Posted on

KWin über den Verlauf der Zeit

Eine der kontroversesten Entwicklungen in KDE in den letzten Jahren war sicherlich das KDE 4.0 Release. Obwohl es klar als Entwickler Release bezeichnet wurde: "KDE 4.0.0 is our "will eat your children" release of KDE4, not the next release of KDE 3.5." (Aaron Seigo, Plasma Maintainer, am 04.01.2008 in talking bluntly), wurde es auch von Anwendern verwendet und nach dem Feedback was man damals gehört hat, waren sie damit nicht wirklich zufrieden. Vor allem Plasma in KDE 4.0 wurde (zu Recht) als unausgereift und nicht stabil bezeichnet. Selbst heute kann man auf heise oder pro-linux kaum einen Artikel zu KDE sehen ohne Hinweis auf das nicht ausgereifte Release von vor drei Jahren.

Immer wieder hieß es, dass KDE sich durch das Release selbst Schaden zugefügt hat und massiv Nutzer verloren hat. Solche Zahlen sind schwer zu erfassen, da es generell keine Statistiken zu den Anzahl Nutzern gibt. Von Entwickler Seite selbst wird das 4.0.0 Release als äußerst wichtig angesehen, alleine schon weil man in der Open Source Welt "Release Early, Release Often" folgt. Persönlich gehöre ich zu den Entwicklern (Hinweis: mein erster Patch für KWin war kurz vor 4.0.0 und wurde am 08.01.2008 eingespielt), die 4.0 unter dem Aspekt was KDE selbst erreichen wollte, als vollen Erfolg ansehen. Als sehr einfache Metrik um das zu bestätigen: in 2008 wurde fast jeden Tag ein neuer SVN Account erstellt.

Nun bleibt die Frage offen, ob KDE wirklich Nutzer verloren hat durch das Release und sich selbst geschadet hat. Wie gesagt ist es schwierig Daten dafür zu bekommen. Als Versuch möchte ich mal die Bug Statistik von KWin über die Geschichte von 1999 bis heute heranziehen. KWin ist hierbei interessant, da es in 3.5 bereits eine sehr ausgereifte und extrem stabile Anwendung war. in KDE 4 kamen die Desktop Effekte dazu, jedoch bis 4.2 standardmäßig deaktiviert und als experimentelles Feature markiert.

Die Statistik beinhaltet leider auch die "Wishlist" (aka Feature Requests), welche die Statistik leicht verfälschen, da sie meistens nicht bestätigt werden oder implementiert werden. Aktuell haben wir ungefähr 320 Feature Requests wovon 270 zu der roten Linie der Unconfirmed Bugs gehört. Eine Entwicklung über die Zeit steht mir hier leider nicht zur Verfügung.

Erklärung der Begriffe:

  • UNCONFIRMED: ein Bug, der noch nicht reproduziert werden konnte oder ein Feature Request
  • NEW: reproduzierbarer Bug
  • ASSIGNED: ein Entwickler hat sich den Bug zugewiesen – wird in KWin eigentlich nicht verwendet
  • REOPENED: ein Bug der als behoben markiert wurde und noch immer vorhanden ist.

Der Graph zeigt sehr deutlich einen massiven Einschnitt in der Entwicklung. Dieser Einschnitt liegt um die Jahreswende 2007/2008. Zwischen 2005 und 2008 haben sich die Zahlen der bestätigten Bugs kaum verändert. Der Abstand zwischen den UNCONFIRMED und NEW bleibt in diesem Zeitraum in etwa gleich. Was hat sich also in 2008 ereignet, dass sich die Zahlen so verändern? In diesen Zeitraum fällt das 4.0 Release. Seit diesem Zeitpunkt hat sich die Gesamtzahl der offenen Bugs mehr als verdoppelt und nehmen etwa einen linearen Verlauf an.

Zuerst einmal stellt sich hier die Frage: wie kann es sein, dass Bugs gemeldet werden, wenn das Release doch in einem Zustand ist, dass es nicht benutzbar ist und der neue Code in der Anwendung standardmäßig nicht aktiviert ist (und damals) auch auf der meisten Hardware nicht funktionierte?

Eine naheliegende Erklärung zu den wachsenden Kurven wäre, dass vor 4.0 die Entwickler neu aufgemachte Bugs behoben haben und ab 4.0 der Bugtracker vermüllt. Diese These kann man mit einer weiteren Statistik widerlegen:

Zuerst die Erklärung der neuen Linien:

  • FIXED: ein behobener Bug
  • DUPLICATE: ein Bug der bereits gemeldet wurde und durch Triaging manuell als Duplikat markiert wurde
  • NEEDSINFO: ein Bug bei dem Rückfragen an den Nutzer gestellt wurden und keine Antwort kam
  • UPSTREAM: Bug in einer Komponente auf die KWin aufbaut – meistens Treiber

Bei den neuen Kurven ist wichtig zu bedenken, dass es zu erwarten ist, dass die Werte immer weiter anwachsen – im Gegensatz zu den offenen Bugs wo man erwartet, dass sie konstant sind oder zurückgehen. Am interessantesten ist die Kurve der FIXED Bugs. In dem Bereich zwischen 2005 und 2008 ist die Kurve fast konstant – genauso wie die Kurven der UNCONFIRMED und NEW Bugs. In 2008 verändert sich die Kurve ebenfalls und nimmt auch ein lineares Wachstum an. In den drei Jahren seit 4.0 haben die KWin Entwickler fast so viele Bugs behoben wie in den acht Jahren zuvor.

Die Kurve zeigt deutlich, dass zum Auslauf von KDE 3.5 keine neuen Bugs mehr gemeldet wurden und seit 4.0 die Anzahl sowohl gemeldeter als auch behobener Bugs deutlich ansteigt. Dies kann man auch an der DUPLICATE Kurve sehen. Diese wächst auch während KDE 3 linear an. D.h. es wurden immer wieder die gleichen Bugs gemeldet. Jedoch verändert auch diese Kurve Ende 2008 ihre Form (Kubuntu mit 4.1 und Effekten standardmäßig aktiviert). Die Kurve hat klar die stärkste Steigung von allen hier gezeigten und scheint gänzlich unbeeinflusst von den anderen Kurven zu sein. Dass wir viele Duplikate bekommen habe ich ja schon vorher gewusst.

Nun will ich noch eine weitere Statistik zeigen: die Entwicklung von KWin. Mit Hilfe von git log und einem kleinen Programm hab ich mir die Anzahl Commits und Anzahl Committer pro Jahr generieren lassen (Achtung: Y-Achse ist logarithmisch):

Der Graph bricht zum Schluss so stark ab, da wir in 2011 noch nicht viele Monate hatten 😉 Was dieser Graph nun zeigt, ist dass in der Zeit in der keine neuen Bugs gemeldet wurden die Enticklungsgeschwindigkeit stark angezogen hatte (KDE 4 Portierung/Entwicklung der Effekte) und auf dem hohen Level sich hat halten können. Das 4.0 Release hat die KWin Entwicklung weder in die eine noch in die andere Richtung beeinflusst. Insbesondere konnte KWin das reduzierte Engagement des Maintainers (immernoch etwa 27 % aller Commits) durch neue Entwickler kompensieren.

Fazit:

  1. Die Aussage, dass es das 4.0 Release brauchte um den Code getestet zu bekommen, ist im Falle von KWin klar bestätigt. Seit 4.0 steigt die Anzahl der gemeldeten und behobenen Bugs stark an.
  2. Zumindest KWin hat durch das 4.0 Release keine Nutzer verloren. Andernfalls hätte die Kurve nicht anziehen können
  3. Die Anzahl der Bugs ist gestiegen obwohl die neue Funktionalität nicht standardmäßig aktiviert war. Dies deutet auf ein Wachstum der Benutzerzahlen hin.
  4. Das spätere standardmäßige Aktivieren der neuen Funktionen hat keine Auswirkungen auf offene und behobene Bugs sondern nur auf Duplikate und Treiber Bugs
  5. Die massive Stabilisierung in KWin hat keine Auswirkung auf die Anzahl der gemeldeten Bugs. Die Rate ist nahezu konstant. Auch dies lässt sich nur mit Anwachsen der Benutzerzahlen erklären.
  6. KWin ist als Projekt sehr gesund.

Ich bitte davon Abstand zu nehmen mich in den Kommentaren daruaf hinzuweisen, dass 4.0 doch ganz fürchterlich war. Das 4.0 Release war vor meiner Zeit als Entwickler und ich bin daher in keinster Weise für das Release verantwortlich.

=-=-=-=-=
Powered by Blogilo

Posted on

Proactive Spam Fighting with Akonadi

One year ago I finished my Master Thesis and I just realized that I never really blogged about it. I had the chance to build my implementation during the Thesis upon the Akonadi framework, which is the reason why I want to share what I worked about. For all who might be more interested in my Thesis, a publisher was so kind to publish it.

The title of my Thesis was "Implementation of Proactive Spam Fighting Techniques" and the Thesis covered the implementation of two orthogonal techniques. Both techniques shared the idea to eliminate Spam before it hits the user’s inbox. Current Spam fighting techniques like for example SpamAssassin are reactive. SpamAssassin uses a rule based approach and a Bayesian filter. Rule based approaches cannot identify Spam messages reliable and have the danger to incorrectly mark ham messages if like last year the date grossly in the future becomes present. Bayesian filtering requires to collect a rather large amount of mail messages before messages can be filtered. This illustrates that reactive Spam fighting is no real solution.

Both implementations required to interact with the user’s inbox and in one case it was required to automatically send out mail on mail reception. The original idea of my tutors at the Laboratory for Dependable Distributed Systems was to develop either a plugin for Mozilla Thunderbird or Microsoft Outlook. With my background of KDE development I had to think of one framework which handles this much better: Akonadi. Thanks to Akonadi I was able to implement the solution in a client and platform independent way. Instead of just supporting Mozilla Thunderbird the solution works on all systems supporting Akonadi and no specific mail client is required. It can even be used by users who just use a web mail client.

Mail-Shake

Mail-Shake is the name of one of the two projects. The concept is difficult to grasp, therefore I just copy the description from my thesis:

The basic idea behind Mail-Shake is to block all mails from unauthenticated senders and to provide senders an easy way to authenticate themselves. The process of authentication is done in a way so that humans are able to participate, while computers – and by that spam bots – are not. After authentication the sender’s address is put on a whitelist. The whitelist is used by Mail-Shake to decide if a mail is authorized or not. By that the concept is proactive as it blocks spam before it is read by the user.

The Mail-Shake process

These initial steps of authentication are illustrated in the Figure above. A sender (User A) has to send a mail to User B’s public email address. All mails sent to the public address are discarded, but answered with a challenge mail containing a unique identifier. User A has to solve the challenge, which reveals User B’s private address. Now User A can resend the original mail with the identifier in the subject. Mail-Shake compares the identifier and puts User A’s address on a whitelist. In future User A can send mails directly to User B’s private address. The authentication step is required only once. As well there is no need to include the identifier in each single mail. Other mails sent to the private address are discarded if the sender address is not on the whitelist.

For the implementation I wrote an Akonadi Agent which monitors a set of public and private collections. Whenever a mail is received at the public address, the agent generates the unique Id, stores it in a database and sends the challenge mail to the sender’s address. The handling of the private collection is more complex. The agent has to decide whether the mail is authorized to be send to the user or not. It first checks the whitelist, if not on the whitelist it has to decide whether it’s a reply to a challenge mail and if it is whether the id is correct and update the whitelist.

The original paper considered that a mail gets just dropped if it is not on the whitelist. That is where I disagreed and implemented an automated mail notification to the sender in the case of not being on the whitelist or incorrect copying of the unique id. Furthermore I did not just delete the mails but used the functionality of Akonadi to allow the mails to be be moved to a different collection.

The concept so far works wonderful for human communication. Humans are able to solve the challenge (in case of my implementation reCAPTCHA Mailhide) and to resend the mail. All spam messages are automatically deleted without disturbing the user. Wait, there is more: communication with machines. I receive far more mails send from machines than from humans: forum, mailing lists, web shops, review board and many, many more. Nobody in the web shop will answer the challenge mail; Mail-Shake will break.

The solution to this problem is to allow users to modify the whitelist by themselves. Of course this is tricky and could make the usage too difficult. Because of that I played again with KDE technology. My Mail-Shake agent uses a StatusNotifier which is hidden most of the time. Whenever a mail is received, which would be discarded, the StatusNotifier changes to the active state and becomes visible. It allows to just add the addresses of the mails in the current to be discarded queue to the whitelist.

Notification on receipt of a not-whitelisted mail

In order to make it even more usable to the user, Mail-Shake can show normal Plasma notifications with an Action to add the address to the whitelist. The idea behind it is, that whenever you expect a mail from an automated system, you turn on the notification through the StatusNotifier and wait for the mail to arrive. (Btw a nice example for a notification which is completely useless when not having actions on it).

After the implementation I had tested Mail-Shake on my own mail system for two months. It illustrated that the implementation and the concept works. Spam mails were successfully removed and humans authenticated themselves. Nevertheless some problems occurred. The first is that the assumption for interaction with web-shops does not hold. Just because you buy at shop foo and receive a mail notification from @foo.com does not mean that further mails (e.g. the bill) will not be sent from @bar.com. This makes interacting with automated systems hardly possible without manually checking the filtered mails, in which case nothing is won in comparison to the existing reactive systems.

The second problem occurred when sending out the response challenge mails. As most mails are sent in reply to Spam mails we can expect that there is no mail system to accept the mails. Our mail transfer agent (MTA) will therefore send us a delivery status notification (DSN). Mail-Shake can handle those and will not send another challenge mail in response to the DSN and remove it, so that the user does not see it.

Mail-Shake supports delivery status notifications as described in RFC 3464, which does not specify a reference to the original mail, which makes it difficult to map the DSN to the sent mail and to decide if the DSN needs to be delivered to the user or not. Luckily during my evaluation each DSN included either the complete original message or at least the header section. So this seems to not be a problem in praxis. But the real problem is that there are MTAs not sending DSNs as described in RFC 3464. They use a custom format. Some of the non-standard conform MTAs are Exim, QMail (at least in older versions) and the MTA used by Google. The mail format is custom which makes it impossible to parse the mail automatically. In case of Exim it seems to be possible to customize the notifications. Mail-Shake is not able to recognize the DSN and starts to go into an infinite mail sending loop.

My personal resume is that Mail-Shake is not in a state to be used in a productive environment. The problem is not in the concept or in the implementation, but in MTAs not supporting RFC 3464. It is unlikely that MTAs not supporting the RFC for years will start to support it and even if, there are enough legacy installations to destroy any hope in having this concept in a workable state. It was rather frustrating to notice that the concept breaks because of the external state of the system.

Mail-Shake Client Integration

In my implementation I relied on reCAPTCHA mailhide API to protect the private mail address. The recipient of a challenge mail has to visit the reCAPTCHA website to reveal the address. This means we encourage users to click on links in mails. That is of course the opposite of what we teach users: never click a link in a mail you did not expect! To solve this issue we need an integration into the mail clients. The user should not notice that he received a challenge mail, the client has to guide the user in solving the challenge.

As at the time of writing my Master Thesis KMail was in heavy porting to Akonadi, I decided to implement the proof-of-concept integration in Mailody (which already supported Akonadi and WebKit at that time). The integration was straight forward and easy to implement. Mailody was a very nice application from that point of view. I extracted the Mail-Shake headers and used them to recognize that the mail is a challenge and integrated a link to solve the challenge. If you clicked the link a dialog opened, connected to reCAPTCHA, extracted the CAPTCHA and offered an input field to enter the text. The solved CAPTCHA was submitted to the web server and the result parsed, providing a localized info whether the challenge was solved or the CAPTCHA incorrect.

Solving a Mail-Shake Challenge in Mailody

The next step would have been to automatically resend the mail with the solved challenge, but I did not implement that. The idea was to just show that a client implementation is possible to solve the mail clicking problem.

Spam Templates

The second part of my Thesis is much smaller, but in my opinion the more interesting one. The laboratory had setup a system to execute Spam Bots in a controlled environment and to intercept the Spam mails it was sending out. The communication to their Master&Control servers was kept alive. The bot and the Master&Control server is by that thinking that they are working correctly. The Spam Bot is regularly fetching new templates from the control server and generates mails based on that template. These mails are intercepted and the template is reverse engineered. The exact process is described in the paper Towards Proactive Spam Filtering.

From the previous existing setup I had a set of regular expression based templates and a set of spam messages which were used to construct the templates. As the system was not up and running I did not have recent templates. Because of that I could only evaluate against the existing messages.

My implementation was based on another Akonadi Agent which reacted on the reception of new mails. Any new mail is checked against the templates and a score is generated from the matched lines of the template. The Agent would receive new templates through an RSS system (thanks to libsyndication). In the ideal situation new templates would be created at the same time as a new Spam campaign is started and distributed to all clients. So at the time the client receives the first Spam message of the new campaign the template would already exist. As the template is only generated from Spam messages it would be impossible to match Ham messages. This has an incredible advantage compared to the existing reactive filtering solutions: you don’t have to collect Spam messages before you are able to filter them.

Unfortunately this is just a research project, so there is no server broadcasting the Spam templates. Which makes my implemented system rather useless and did not allow me to verify if the implementation works correctly – I only know that it does not filter Ham messages.

Where to get the Code?

Of course the complete code has been written as free software, so where can you get it? I used git as version control system, so it has not been available in the KDE repository and after the switch to git I have not yet considered to upload it. The code and packages are available through the openSUSE Build Service and a live CD is available through SUSE Studio (awesome tool). It is possible that some parts are not working anymore as it seems that reCAPTCHA changed their web site (which fails my parser).

What else?

Of course I could only present a small part of my Thesis in one blog post. There are lots more interesting things to find in my Thesis. For example I was able to break the scr.im email address protection system using OpenGL Shading Language and OCR with a reliability of more than 99 %. I also did an investigation on existing CAPTCHA systems which showed me that most systems are in fact completely broken and just an annoyance to the user who has to use them.

Last but not least I want to thanks the KDE PIM team for providing such an awesome framework. Without Akonadi this Thesis would not have been possible. It is just incredible what can be done with Akonadi and I’m looking forward to KMail 2 ever since I started to work with Akonadi. I always feel sorry if I have to read the bad user responses to Akonadi. Don’t get frustrated, keep up the good work!

=-=-=-=-=
Powered by Blogilo

Posted on

Fortschritt im Land der Wabernden Fenster

Ich hab schon länger nicht mehr auf Deutsch gebloggt, was natürlich nicht bedeutet, dass die Entwicklung an KWin eingeschlafen wäre, nein ganz im Gegenteil. Aktuell passiert im Land der wabernden Fenster recht viel und viele Verbesserungen für die Anwender sind in der Implementierungsphase für die nächste Version der KDE Plasma Workspaces.

Über die letzten Monate habe ich hauptsächlich an der Portierung von KWin nach OpenGL ES gearbeitet, womit KWin nun auch auf mobilen Endgeräten verwendet werden kann. Vor kurzem hatten wir dazu eine Meldung auf der KDE Nachrichtenseite (der Artikel enthält auch zwei Videos!). Was aber noch viel wichtiger ist als die Verfügbarkeit von KWin auf mobilen Endgeräten, ist dass mit OpenGL ES der Grundstein für KWin mit Wayland gelegt ist. Wayland erzwingt für Compositing OpenGL ES und somit muss jeder Compositor OpenGL ES verwenden.

Durch die Unterstützung von OpenGL ES wurde der gesamte Compositing Code überarbeitet und KWin unterstützt nun OpenGL 1.x und 2.x als rendering backends. Voreingestellt ist dabei (sofern es der Treiber unterstützt) OpenGL 2. Damit haben wir nicht nur einen bedeutend moderneren Code, sondern auch bessere Performance im gesamten Rendering Stack. Natürlich ist das noch nicht alles, denn wir arbeiten nun daran den Code zu optimieren und selbst ohne Optimierung spürt man schon deutliche Performance Verbesserungen im Vergleich zu 4.6.

Abseits der Programmierung passiert auch richtig viel. Ich habe angefangen im Community Wiki eine KWin Sektion einzurichten um neuen Entwicklern das Leben einfacher zu machen und unsere Bugzilla Komponenten wurden erweitert und die Bugs werden nun besser kategorisiert. Hier ist natürlich die Hilfe der Community mehr als erwünscht 🙂 Bei über 400 Bugs ist das kaum etwas was sich leicht und schnell stemmen lässt.

In unserer Code Struktur arbeite ich auch daran die Effekt API etwas zu erneuern um in Zukunft vielleicht doch irgendwann mal eine stabile API zu haben. Das ist eine Voraussetzung um einige Effekte aus KWin auszulagern. Mittlerweile haben wir mehr als 40 Effekte was sowohl uns als auch den Nutzern die Verwendung erschwert. Ziel ist es in KWin nur noch die wichtigsten Effekte zu haben und den Rest auszulagern. Dies erlaubt uns die Effekte besser zu warten und uns gezielt auf einige Effekte zu konzentrieren.

Demnächst beginnt auch wieder der Google Summer of Code und KWin möchte sich auch daran beteiligen. Wir haben einige interessante Projekte ausgeschrieben. Z.B. eine Modularisierung von KWin Core zur besseren Wartung von KWin Core, dem Aufbau eines Unit Test Frameworks mit Xephyr und KWin’s Scripting Komponente sowie ein Projekt zur Initialen Unterstützung von Wayland Clients. Natürlich kann jeder potentielle GSoC Student auch eigene Ideen bringen.

Wie man sieht sind wir fleißig am Arbeiten und wollen mit KWin in 4.7 das nächste Level des composited Window Managments einläuten. Die Änderungen sind jetzt schon unglaublich performant und meine Erwartung ist, dass wir mit 4.7 den Vergleich mit Compiz endlich nicht mehr scheuen müssen 😉

=-=-=-=-=
Powered by Blogilo

Posted on

Is KWin rocket science?

This weekend I played around with adding a slight animation to the BoxSwitch effect when the selection changes and when new windows are opened or existing closed. Here’s the video (sorry for bad quality):

(OGG Video)

The BoxSwitch effect is now animating the selection when pressing Tab. It’s moving instead of jumping as before. Another change is that all thumbnails adept to changes in their geometry when a new window is added or an existing is removed. They slide to the new position instead of just jumping there. Isn’t that elegant?

On this effect I worked together with a new possible KWin developer who does not know anything about OpenGL. So what do you think how many lines of OpenGL code this new animation contains? 100? 50? 10? or maybe 1? No you don’t need to know any OpenGL to write hardware accelerated animations in KWin. The complete animation code is part of the KWin animation framework and the only change to the code is adding a QTimeLine to the changing geometries. It’s currently just playing around but my plans are to add this new implementation into the rendering engine, so that any changing geometries are automatically animated without the effects to need to worry about.

What I want to highlight is that it is really easy to work on KWin. It’s not rocket science and you do not need to know anything about OpenGL to get awesome effects working. Our effect framework completely abstracts the dirty details making it easy for you to write awesome animations fast. There are lots of small things like here the BoxSwitch where improvements are more than welcome. Just come along to the #kwin channel, drop us a mail on kwin@kde.org and have a look to the Plasma Task Wiki page, which will soon be filled with KWin tasks, too.

=-=-=-=-=
Powered by Blogilo

Posted on

KWin and the unmanageable combinations of drivers

This week I once more noticed the biggest problem of KWin development: the differences between the multiple drivers. Since Monday KWin uses a new OpenGL 2 based coding path as default. The code was mostly written on one of my systems with the nouveau driver and regression testing was mostly done on my second system using fglrx which only supports the "legacy" old coding path by default but can be forced to the new one (but too slow for productive usage).

Around Wednesday notmart reported that the code is broken on NVIDIA proprietary driver. As I had developed on nouveau I had not tested with the blob. So I had to switch the driver again and could confirm the regression (after fixing: it was clearly our bug and not NVIDIA’s). The regression occurred in a code path which seems to be only executed with the NVIDIA blob. It took me hours to figure out what is causing the bug and how to fix it. And it illustrates the big problem in KWin development: without an NVIDIA card and the driver I would not have been able to fix it and I doubt that anyone not knowing the code would have been able to fix it.

Currently KWin supports hardware from the three big vendors: NVIDIA, ATI/AMD and Intel. Each of the vendors has a set of different hardware generations. The biggest difference is between the fixed functionality hardware and the general purpose GPUs. The difference between the hardware generations is so big that assuming that just because it works on one of it, that it works on the others, too, is rather naive.

So we have different hardware vendors with hardware in different generations. Let’s make it more complex: drivers. We also have different drivers for the same hardware. Currently we see the following drivers:

  • NVIDIA proprietary driver
  • Catalyst (aka fglrx) proprietary driver
  • Intel (Mesa driver)
  • Radeon (Mesa driver for AMD/ATI)
  • Radeon/Gallium3D (Mesa driver for AMD/ATI)
  • Nouveau (Mesa driver for NVIDIA, Gallium3D, experimental)

Except for Intel we have different drivers for the same piece of hardware. In case of AMD/ATI even different free drivers (the list is missing radeonhd). The big hope is here in the Gallium3D stack which will hopefully simplify the situation by having one free stack shared by various drivers. As my example above illustrates different drivers for the same hardware show different behavior. So we need not only test with various hardware, but also with the various drivers. So at that point we are somewhere around 30 to 50 combinations of different vendors, hardware generations and the drivers for them.

That’s quite complex already, isn’t it? So let’s make it more complex: driver versions. Obviously each vendor/driver developer is working on their drivers and is fixing bugs and introducing new ones. Of course we need to support all versions properly, so we would need to test with all driver versions. E.g. we know that we see regressions in the latest NVIDIA blobs. Unfortunately I’m still on an older one due to using Debian Testing and not wanting to mess with my setup. So I cannot investigate these issues.

Ok we just made everything more complex with driver versions. Let’s add another complexity factor for the free drivers. The free drivers are more or less bundled together with libdrm, mesa, XServer and the kernel. Some drivers require a specific kernel version or won’t work with another one. This is kind of opening Pandora’s box concerning the possibilities of combinations.

So we have reached the level of combinations which are not being possible to test without recompiling the complete kernel/mesa/X stack. Let’s add another complexity factor: distributions. Distributions do not only ship the upstream drivers, no they "improve" them. We have seen that in the past several times that distro specific code broke KWin. The distribution tries to support a vast diversity of software and it can happen that an optimization for Compiz causes rendering issues in KWin. Also distributions sometimes bundle combinations of drivers and kernel which do not work together. This mostly happens with rolling release distributions. E.g. new Mesa version requiring a new Linux kernel which is not yet released. But rolling release distributions like Arch are also hitting problems before other distributions which can help us to workaround issues before it hits distributions with a larger userbase such as Kubuntu or openSUSE.

All together there are several hundreds of combinations of the stack below KWin. Changes in any of the component can cause regressions. Even trying to test all of them seems to be ridiculous. If I would try to test a reasonable amount of the combinations (vendors * hardware generation * drivers * latest version) I would need something like 30 machines. Of course I don’t have the time for it and that’s also the reason why I declined all hardware offerings so far. It’s just like a drop in a bucket.

But we have users who can test. We need users willing to run the latest KWin master together with the latest driver versions to ensure that we do not run into regressions. And we need users reporting high quality bug reports who are willing to also report them on freedesktop.org bugtracker.

And we need developers: preferable with OpenGL experience. Now is the perfect time to join the KWin development. We switched to a more easily readable coding style and introduced a new and modern OpenGL Shader based compositing backend. You could work on improving the User Experience for all users by writing more awesome effects or optimizing our backend. It’s really fun stuff.

=-=-=-=-=
Powered by Blogilo

Posted on

On the Road to Modern OpenGL (ES)

With KWin GLES shortly before the merge into workspace master (after git transition), it’s time to look back what I promised to deliver half a year ago, when I first talked about the idea of porting KWin to OpenGL ES at Akademy.

At the time of Akademy I had not yet started to write any code as I considered 4.5 more important and waited for the git transition. Obviously I stopped waiting for git or there would not be any code written 😉 My schedule presented at Akademy looked like that:

  • Remove deprecated OpenGL code
  • Remove unused code in Scene and E?ects Lib
  • Drop XGL support
  • Port to OpenGL ES 1.1 till 4.6
  • Port to OpenGL 3 and ES 2.0 till 4.7

Removal of deprecated OpenGL code is mostly done. With GLES KWin has a forward compatible code path, though we still have OpenGL 1.x code for legacy systems and that won’t be dropped in the near future. Considering deprecated OpenGL 1.x code it’s looking quite good, e.g. glBegin/glEnd is removed in the GLES branch. Rendering of geometries is abstracted and the backend performs a runtime check whether legacy or modern rendering is invoked.

For 4.6 I started to port some effects to newer functionality which allowed to drop some then unused code. The remaining functions have been marked as deprecated, but are not yet removed in the GLES branch as they are still used by the Shadow effect (though unavailable in OpenGL ES and broken with OpenGL 2 backend). Given that Shadow is difficult to port, in general broken and unmaintained I consider to just drop it. I am already gathering ideas for a new replacement effect.

The XGL code (XGL was needed to run Compiz before AIGLX was integrated) is still in KWin and should really be dropped. It has been removed from X git tree for quite some time and I doubt anybody has ever run KWin with XGL. There are more things we can now consider to remove: shared memory and "fallback" compositing. Together this will mean a requirement to Texture from Pixmap. That’s pretty well supported nowadays on all drivers and anything else cannot be recommended.

The next bullet point is OpenGL ES 1.1. Well I noticed that it would really clutter the code, so I stopped working on it and switched to going to ES 2.0 directly. My initial plan was to provide 1.1 support at the time of the 4.6 release. Instead we now have ES 2.0 support at the time of 4.6 😀

In 4.7 we will not only have an OpenGL ES 2.0 backend available, it will be integrated into the release. As a side effect it gives us a complete OpenGL 2.x backend which should benefit all users having decent drivers. I’ll soon integrate an option to the advanced settings tab to choose between OpenGL 1.x, 2.x and 3.x. With the 2.x code there, it’s really easy to go to 3.x. We won’t default to it, we will hardly use it (I want to experiment with Geometry Shaders in MagicLamp effect), but I hope that the drivers supporting OpenGL 3 (that’s for us only the NVIDIA blob at the moment, fglrx is only usable with OpenGL 1.x in the composited case) are more efficient if run in a forward compatible profile.

There is still lots to do and I want to concentrate on improving the rendering stack throughout the 4.7 release cycle. It’s quite fortunate that the new rendering stack can be integrated directly after the git transition. So I hope that lots of developers will be using it and can provide feedback for various hardware.

=-=-=-=-=
Powered by Blogilo

Posted on

Bug Statistics for KWin 4.6 Cycle

Once again I used Bugzilla to get some statistics on how many bugs are opened, closed in KWin during the last development cycle (statistics are from Final Tagging 4.5 till today). Overall 437 new bugs have been reported and 425 bugs have been "closed". Many thanks again to the bug day – without it the numbers would look worse. So this means that each day 2.4 bugs are reported. Assuming that we spent 10 minutes on each bug (in general Thomas Lübking and I are reading and responding to each bug comment), we spent 8 days just on bug managing. That sounds reasonable, but is very bad as it’s time spent managing and not fixing the bugs. This can be seen in the table below.

Like in my previous reports about 40 % (165) of the new bugs are duplicates. But this time we also faced the problem of the drivers: 12.8 % (56) of all new bugs have been marked as upstream. I also took the time to go through all duplicate crash reports and found another 52 duplicate driver crashes. About 50 % (213) of all bugs are crashes, but only 3 crashes are confirmed and only 9 have been fixed. 24 of the crashes have not been confirmed yet, meaning that the way to reproduce is not clear. Altogether 21.5 % (94) bugs have not been confirmed yet. Apart from the driver bugs our worst problem has been in Desktop Grid and those crashes were fixed early in the cycle.

Category # New Bugs/% to All # New Crashes/% to New Bugs Overall Closed
Unconfirmed: 94/21.5 24/25.5
New: 17/3.9 3/17.6
Needsinfo: 37/8.4 27/75 55
Fixed: 34/7.8 9/27.3 71
Invalid: 24/5.4 3/12.5 26
Wontfix: 2/0.5 3
Duplicate: 165 (-52)/37.8 110 (-52)/66.6 185 (-52)
Worksforme: 5/1.1 19
Upstream (driver): 56 (+52)/12.8 36 (+52)/64.3 63 (+52)
Downstream: 3/0.7 1/33.3 3
Sum: 437 213/48.7 425

=-=-=-=-=
Powered by Blogilo

Posted on

Down to Five KWin Effects

Today I ported another bunch of effects to GLES which leaves just five more effects to be ported:

  • Blur
  • Logout
  • Explosion
  • Invert
  • Snow

Blur, Logout and Snow are the more difficult ones as they have quite some custom code. Snow especially is calling for a rewrite (it was my first effect and I can clearly see how much my coding improved). Additionally to those effects Shadow and Sharpen are not yet ported and won’t be ported. I want to replace Shadow with BeShadowed or with something completely new to support Oxygen shadows also for menus. The Sharpen effect will be removed for various reasons.

But there is also some work left in other effects. Flip/Cover Switch are missing the multi-monitor adjustments and in Cube effect Sphere and Cylinder are not yet supported. Those knowing a little bit about KWin internals will notice, that mostly effects using Shaders are left. The shader files need to be adjusted due to the fact that the existing ones were emulating fixed-functionality. This is not available in GLES, so the shaders need to be rewritten to not use the emulation parts. I did that today with the Looking Glass effect.

I did a screencast of KWin GLES with the nouveau driver. It shows that resizing is much faster and effects are more smooth than our existing rendering stack. The quality of the video is not so good (I still have problems getting recordmydesktop to work properly) that’s why I do not include it in this post. If you don’t mind it, you can find it on blip.tv.

=-=-=-=-=
Powered by Blogilo

Posted on

This week in KWin-GLES

Another week is gone and my KWin-GLES branch got further polishing. All developing on the OpenGL ES code is nowadays happening on my notebook using KWin-GLES. So it is already so useable that I can develop on it without real problems. Most of the effects are already ported and can be used. Today I added the first 3D effects, that is CoverSwitch and FlipSwitch and a (locally) half finished Cube Desktop Switch animation. Quite nice to see the new code working as expected 😉 Only eleven effects are not yet ported, but these include some effects which won’t be ported, such as FPS, Sharpen or Shadow effect.

KWin has now three default shipped shaders, which can be used by both effects and the compositing scene. There is one shader to render simple 2D textured graphics. It’s a straight forward simple shader for the normal tasks. A second, more generic shader is available for all kind of transformations: translating, scaling and rotating of the complete screen or windows. Last but not least there is one shader to render simple colored geometries. I noticed I need that one, when I started to port the ShowPaint effect (one of the most important ones).

Going on ES means that everything has to be rendered with shaders and geometries have to be passed using a vertex buffer object (VBO). For 4.6 I added an abstraction to VBOs which is only used by the EffectFrames (overlays in e.g. PresentWindows effect). Now all effects and the scene rendering has to use those VBOs. The way KWin works it’s mostly streamed data, so there is one VBO which can be shared by all effects.

The old code looked like that:

glPushAttrib( GL_CURRENT_BIT | GL_ENABLE_BIT );
glEnable( GL_BLEND );
glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
float alpha = 0.2;
const QColor& color = colors[ color_index ];
glColor4f( color.red() / 255., color.green() / 255., color.blue() / 255., alpha );
glBegin( GL_QUADS );
foreach( const QRect &r, painted.rects())
    {
    glVertex2i( r.x(), r.y());
    glVertex2i( r.x() + r.width(), r.y());
    glVertex2i( r.x() + r.width(), r.y() + r.height());
    glVertex2i( r.x(), r.y() + r.height());
    }
glEnd();
glPopAttrib();

We see here quite some deprecated stuff like glBegin/glEnd, this needs to be replaced by a VBO. Each quad has to become two triangles and of course it’s not allowed to set the color (that has to be done in a shader). Last but not least the attribute stack is not available which means we need to explicitly disable blending. The new code looks like that:

#ifndef KWIN_HAVE_OPENGLES
glPushAttrib( GL_CURRENT_BIT | GL_ENABLE_BIT );
#endif
GLVertexBuffer *vbo = GLVertexBuffer::streamingBuffer();
vbo->reset();
vbo->setUseColor(true);
if (ShaderManager::instance()->isValid()) {
    ShaderManager::instance()->pushShader(ShaderManager::ColorShader);
    vbo->setUseShader(true);
}
glEnable( GL_BLEND );
glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
QColor color = colors[ color_index ];
color.setAlphaF(0.2);
vbo->setColor(color);
QVector<float> verts;
verts.reserve(painted.rects().count()*12);
foreach (const QRect &r, painted.rects()) {
    verts << r.x() + r.width() << r.y();
    verts << r.x() << r.y();
    verts << r.x() << r.y() + r.height();
    verts << r.x() << r.y() + r.height();
    verts << r.x() + r.width() << r.y() + r.height();
    verts << r.x() + r.width() << r.y();
}
vbo->setData(verts.size()/2, 2, verts.data(), NULL);
vbo->render(GL_TRIANGLES);
if (ShaderManager::instance()->isValid()) {
    ShaderManager::instance()->popShader();
}
glDisable( GL_BLEND );
#ifndef KWIN_HAVE_OPENGLES
glPopAttrib();
#endif

Unfortunately we currently need to ifdef the attribute stack. I am planning to go through the complete KWin code and remove all of those later on. The VBO for streaming is used to set the vertices. Also the color is attached to the VBO. When rendering is performed the VBO either uses the legacy glColor or sets a uniform of a bound shader. The new ShaderManager is used to bind the shader for colored geometries. This manager takes care of restoring a previous bound shader when unbinding the shader. As we can see the shader is only bound if it is valid. So this code works for both the legacy rendering as well as the modern shader based rendering. The effect itself does not even need to care about it. All the differences for passing the geometries, setting the color, etc. are abstracted away in the VBO. With these changes porting an effect is mostly straightforward.

Now with most effects and functionality ported I can start looking at the finishing touch. I split several methods in a glx and egl specific variant and noticed later on that the differences are rather small. Those have to be merged again and I have to ensure that everything is still working when not using GLES. I am still aiming for importing everything into master as soon as kdebase-workspace migrates to git. Maybe I will give it one week or two to settle down before merging into master, but I’m quite confident that we will see this in master very soon.

=-=-=-=-=
Powered by Blogilo