Down to Five KWin Effects

Today I ported another bunch of effects to GLES which leaves just five more effects to be ported:

  • Blur
  • Logout
  • Explosion
  • Invert
  • Snow

Blur, Logout and Snow are the more difficult ones as they have quite some custom code. Snow especially is calling for a rewrite (it was my first effect and I can clearly see how much my coding improved). Additionally to those effects Shadow and Sharpen are not yet ported and won’t be ported. I want to replace Shadow with BeShadowed or with something completely new to support Oxygen shadows also for menus. The Sharpen effect will be removed for various reasons.

But there is also some work left in other effects. Flip/Cover Switch are missing the multi-monitor adjustments and in Cube effect Sphere and Cylinder are not yet supported. Those knowing a little bit about KWin internals will notice, that mostly effects using Shaders are left. The shader files need to be adjusted due to the fact that the existing ones were emulating fixed-functionality. This is not available in GLES, so the shaders need to be rewritten to not use the emulation parts. I did that today with the Looking Glass effect.

I did a screencast of KWin GLES with the nouveau driver. It shows that resizing is much faster and effects are more smooth than our existing rendering stack. The quality of the video is not so good (I still have problems getting recordmydesktop to work properly) that’s why I do not include it in this post. If you don’t mind it, you can find it on blip.tv.

=-=-=-=-=
Powered by Blogilo

This week in KWin-GLES

Another week is gone and my KWin-GLES branch got further polishing. All developing on the OpenGL ES code is nowadays happening on my notebook using KWin-GLES. So it is already so useable that I can develop on it without real problems. Most of the effects are already ported and can be used. Today I added the first 3D effects, that is CoverSwitch and FlipSwitch and a (locally) half finished Cube Desktop Switch animation. Quite nice to see the new code working as expected 😉 Only eleven effects are not yet ported, but these include some effects which won’t be ported, such as FPS, Sharpen or Shadow effect.

KWin has now three default shipped shaders, which can be used by both effects and the compositing scene. There is one shader to render simple 2D textured graphics. It’s a straight forward simple shader for the normal tasks. A second, more generic shader is available for all kind of transformations: translating, scaling and rotating of the complete screen or windows. Last but not least there is one shader to render simple colored geometries. I noticed I need that one, when I started to port the ShowPaint effect (one of the most important ones).

Going on ES means that everything has to be rendered with shaders and geometries have to be passed using a vertex buffer object (VBO). For 4.6 I added an abstraction to VBOs which is only used by the EffectFrames (overlays in e.g. PresentWindows effect). Now all effects and the scene rendering has to use those VBOs. The way KWin works it’s mostly streamed data, so there is one VBO which can be shared by all effects.

The old code looked like that:

glPushAttrib( GL_CURRENT_BIT | GL_ENABLE_BIT );
glEnable( GL_BLEND );
glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
float alpha = 0.2;
const QColor& color = colors[ color_index ];
glColor4f( color.red() / 255., color.green() / 255., color.blue() / 255., alpha );
glBegin( GL_QUADS );
foreach( const QRect &r, painted.rects())
    {
    glVertex2i( r.x(), r.y());
    glVertex2i( r.x() + r.width(), r.y());
    glVertex2i( r.x() + r.width(), r.y() + r.height());
    glVertex2i( r.x(), r.y() + r.height());
    }
glEnd();
glPopAttrib();

We see here quite some deprecated stuff like glBegin/glEnd, this needs to be replaced by a VBO. Each quad has to become two triangles and of course it’s not allowed to set the color (that has to be done in a shader). Last but not least the attribute stack is not available which means we need to explicitly disable blending. The new code looks like that:

#ifndef KWIN_HAVE_OPENGLES
glPushAttrib( GL_CURRENT_BIT | GL_ENABLE_BIT );
#endif
GLVertexBuffer *vbo = GLVertexBuffer::streamingBuffer();
vbo->reset();
vbo->setUseColor(true);
if (ShaderManager::instance()->isValid()) {
    ShaderManager::instance()->pushShader(ShaderManager::ColorShader);
    vbo->setUseShader(true);
}
glEnable( GL_BLEND );
glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
QColor color = colors[ color_index ];
color.setAlphaF(0.2);
vbo->setColor(color);
QVector<float> verts;
verts.reserve(painted.rects().count()*12);
foreach (const QRect &r, painted.rects()) {
    verts << r.x() + r.width() << r.y();
    verts << r.x() << r.y();
    verts << r.x() << r.y() + r.height();
    verts << r.x() << r.y() + r.height();
    verts << r.x() + r.width() << r.y() + r.height();
    verts << r.x() + r.width() << r.y();
}
vbo->setData(verts.size()/2, 2, verts.data(), NULL);
vbo->render(GL_TRIANGLES);
if (ShaderManager::instance()->isValid()) {
    ShaderManager::instance()->popShader();
}
glDisable( GL_BLEND );
#ifndef KWIN_HAVE_OPENGLES
glPopAttrib();
#endif

Unfortunately we currently need to ifdef the attribute stack. I am planning to go through the complete KWin code and remove all of those later on. The VBO for streaming is used to set the vertices. Also the color is attached to the VBO. When rendering is performed the VBO either uses the legacy glColor or sets a uniform of a bound shader. The new ShaderManager is used to bind the shader for colored geometries. This manager takes care of restoring a previous bound shader when unbinding the shader. As we can see the shader is only bound if it is valid. So this code works for both the legacy rendering as well as the modern shader based rendering. The effect itself does not even need to care about it. All the differences for passing the geometries, setting the color, etc. are abstracted away in the VBO. With these changes porting an effect is mostly straightforward.

Now with most effects and functionality ported I can start looking at the finishing touch. I split several methods in a glx and egl specific variant and noticed later on that the differences are rather small. Those have to be merged again and I have to ensure that everything is still working when not using GLES. I am still aiming for importing everything into master as soon as kdebase-workspace migrates to git. Maybe I will give it one week or two to settle down before merging into master, but I’m quite confident that we will see this in master very soon.

=-=-=-=-=
Powered by Blogilo

KWin OpenGL ES branch imported into git

There were some requests to be able to try out my work on getting KWin on OpenGL ES, so I just set up a scratch repo based on kdebase-workspace with my changes to CMake and KWin till kdebase migrates to git. 

Disclaimer:

Just to make it sure: I won’t backport anything from current svn trunk to this git branch, I will delete the repo as soon as workspace is in git and we don’t accept bug reports for it. The ES branch is still WIP and is currently tested only on one piece of hardware, so it is possible that it does not work at all on other systems.

In order to test it you need GLES2 and EGL libraries with the GL_OES_EGL_image and EGL_KHR_image_pixmap extensions. Those are provided by mesa. Several parts of KWin are currently excluded from the build, so for example the kcm for desktop effects is not included. To change effects or disable compositing you have to edit the configuration file manually.
In general the ES branch is working pretty well. This blog post is just being written using the ES backend and yesterday I programmed on this system directly (before I used X’s network transparency to forward kate to my main system). It feels much smoother than the glx backend, memory usage of X.org and kwin seem better, and the whole thing is not yet optimized. Since yesterday all effects not depending on OpenGL are supported, e.g. Present Windows or Desktop Grid, while Desktop Cube or CoverSwitch are not yet available. I hope to have them also ported soon.

=-=-=-=-=
Powered by Blogilo

KWin runs on OpenGL ES

Last weekend I could announce that KWin compiles with OpenGL ES headers. This weekend I was able to proceed even more: I got the first windows composited using OpenGL ES 2.0. Not everything is working and there is still lot’s of work to be done and it has not yet been tried on actual devices (yes you can use OpenGL ES on a desktop), but nevertheless it’s a very important step.

As it just looks like normal desktop compositing with rendering artefacts, I don’t show a screenshot. This has to wait till Present Windows (or preferable Cube) works on a mobile device 🙂

I think it is important to give some credits to those who helped me directly or indirectly:

  • Fredrik for providing a nice example application showing how texture from pixmap works on OpenGL ES and for all the help on IRC
  • Mesa/Gallium for providing ES libraries for desktop systems
  • Nouveau for making it possible to use ES on my notebook. Without Nouveau it would have been very difficult to do any testing on ES.
  • Debian for packaging Nouveau with 3D support in the experimental repository
  • Linaro for investigating about the possibilities for texture from pixmap on OpenGL ES
  • Addison Wesley for donating a copy of the fifth edition of the OpenGL SuperBible. Really a great help if you want to develop for the core profile.
  • Marco and Alexis for nagging about missing ES support
  • Marco for promising to trying out the patches on actual devices

It is incredible how clean the ES code looks compared to the glx backend. I’m really looking forward to be able to drop the legacy OpenGL code and I hope to have the ES port in a state that users can use it as a runtime replacement on desktop systems as well. It would be nice to provide users a modern compositing backend.

=-=-=-=-=
Powered by Blogilo

KWin compiles for OpenGL ES

While others are having fun at the KDE Mobile Sprint in Berlin, I am at home and hacking on KWin mobile and could achieve a huge milestone today: KWin compiles for OpenGL ES 2!

This does not yet mean that you can run KWin on an ES powered MeeGo device, it just means that it compiles. There is still lots of work to be done, so there is no compositing code at all. The code for binding textures from pixmap or performing the actual painting is completely disabled. It’s lots of ifdefs currently and that has to be cleaned up first.

Of course nothing of it will hit 4.6, it’s all 4.7 material and the code lives only in my private git branch. Nevertheless if anybody is interested in the code I can provide the diff (or complain loud enough so that we need git :-P).

The impact to the codebase is not as bad as I feared. After dropping the idea of going to OpenGL ES 1.1 first and writing the required shader for the desktop edition, the changes are rather small. In most cases it’s just a huge set of legacy drawing commands which can be easily removed.

Oh and currently all effects are removed from the build as my shader does not yet support the required transformations and many effects use legacy code, which would be too much work to get it compiling in the current state. We will have to see which effects we want on a mobile profile and enable the build of just those (e.g. Present Windows).

So the next steps are setting up a build environment where I actually can test the stuff. KDE Mobile team: I need your help!

=-=-=-=-=
Powered by Blogilo

Debugging Rendering Code visuell

Ich bin aktuell dabei den Rendering Code von KWin neuzuschreiben für OpenGL ES und auch OpenGL 3. Das stellt einen natürlich vor Probleme: wie überprüft man, dass der neue Code ausgeführt wird und, dass der alte noch funktioniert?

Der neue Zeichen Code verwendet nun einen Shader – hier ist es einfach einen Test einzubauen. Aktuell färbe ich einfach alle Pixel grün ein, wenn der Shader verwendet wird. Somit sieht man sehr leicht, welche Bereiche den neuen Pfad verwenden. Neben dem eigentlichen Zeichnen, ist es auch noch interessant zu sehen wie die Geometrie aussieht, die da eigentlich gerendert wird. Vor einiger Zeit hatte mir ein Compiz Entwickler einen Screenshot gezeigt, wo das zu sehen war und ich dachte mir, irgendwie nützlich. Heute abend hab ich es mal schnell eingebaut.

Ich lese zur Zeit die fünfte Auflage der OpenGL SuperBible (dazu mehr mal in einem anderen Post) und zwar von vorne ohne größere Grundlagenbereiche auszulassen. Somit bin ich heute auf eine Stelle gestossen, die beschreibt wie das geht. Natürlich kannte ich die Technik schon, nur ich hatte nicht daran gedacht. Beim Lesen dachte ich mir: ach das war doch das was Compiz auch hat, wäre ganz nützlich einzubauen. Das sieht dann insgesamt eigentlich schon sehr häßlich aus 😉

Debug OpenGL rendering
Debug OpenGL rendering

Was man sehr schön sieht, ist wie groß der Schatten der Fensterdekoration doch ist.. Den debug Modus kann man in Zukunft über die Umgebungsvariable KWIN_GL_DEBUG aktivieren. Da wir ja schon im Feature Freeze sind, sprechen wir hier von 4.7 im Juli 2011. Wer trotzdem den Bildschirm bunt sehen will, kann ja den Show Paint Effekt verwenden.

=-=-=-=-=
Powered by Blogilo

Multihead KWin needs YOU!

Aaron wrote a nice blog post about multihead in Plasma and mentions that "there are still some KWin issues" and he is right. I don’t expect these issues to go away in the foreseeable future. Multihead in KWin is unmaintained as long as I have been developing for KWin and considering the rather small amount of bug reports we get for multihead it seems to be not worth spending time on it. Sorry.

So if anybody wants to use multihead and thinks that he needs multihead (multi-screen and multi-seat are working fine), he would have to sit down and investigate what is working/missing in KWin. I assume that it is not much which needs to be changed – probably working around the Kephal issue mentioned in Aaron’s blog.

So if you test on the Plasma desktop changes, please also have a look into what’s missing in KWin and report it to bug 256242 (it incorrectly mentions multi-screen in the title).

=-=-=-=-=
Powered by Blogilo

Going down the programmable pipeline road

As you might know OpenGL comes in two flavors: fixed functionality and programmable pipeline. With the fixed functionality you have to use API calls to influence the execution of each stage of the rendering pipeline. It is a very powerful API allowing you to do most of the stuff we use in KWin. The programmable pipeline allows to directly execute code (called a "Shader") to do vertex and fragment processing. For example we are able to saturate a complete window as a whole with fixed functionality, but we need a fragment shader to be able to change the color of each pixel depending on the input color. This is for example used in the invert effect. A vertex shader can be used to influence the geometry. E.g. we could use it to transform a cube into a sphere. OpenGL 1.x is completely fixed functionality, in OpenGL 2 the programmable pipeline was introduced to exchange parts of the rendering stack, but fixed functionality was still around. With OpenGL 3 everyone expected the fixed functionality to be removed, but it was only deprecated and you can still use it. All the modern calls have been moved into a "core profile".

KWin uses OpenGL 2, that is most of our code is fixed functionality as it was around with OpenGL 1, but we can use shaders to exchange parts of the rendering pipeline to e.g. invert a window. As our shaders are used to emulate fixed functionality the code they use is mostly deprecated as well, KWin does not yet use the core profile.

To make it all more complicated there is also an OpenGL version for Embedded Systems (ES/EGL), which also comes in a flavor for fixed functionality (1.1) and programmable pipeline (2.0). In opposite to the desktop version of OpenGL, the ES version does not include the fixed functionality API calls in ES 2. This means that our source code cannot be compiled directly for ES.

Up to now I only blogged about that I want to port KWin to ES. I started working on it and in 4.6 there is some code already which was written to ease the porting effort. The rendering code has been partly split into legacy painting and modern painting, which makes it easier to just #ifdef away the unsupported calls when compiling for mobile. My initial plan was to first port KWin to ES 1.1 as the code base is closer to 1.1 than to 2.0. The work on it is quite boring and basically stalled. So I decided that this is no solution and instead want to add a core profile compatible rendering path to KWin – this would work both with OpenGL 3 and OpenGL ES 2. Of course no existing code will be removed and most likely it will become opt-in as I expect the changes to break some current effects when using the core profile (at least I can see that cube is partly broken in my special kwin version I’m running here).

With the feature freeze for 4.6 behind us, it’s the perfect time to play around with a new rendering path as it can mature on my systems before it is made available to a broader developer testing. So yesterday I set down and implemented a core profile compatible shader for basic rendering without transformations. This is to be used when no effect is active and only parts of the screen need to be repainted (visible with show paint effect). The first results of this change look promising as our EffectFrame overlays can already be rendered with this new coding path. I would like to present a screenshot, but it is boring as it just looks as always. If you want beautiful pictures, go to Nuno’s blog 😉 As soon as the window rendering is also based on this new shader I have a base to use it in the ES porting. Luckily I can reuse the work I spent for porting to ES 1.1, as the same code has to be removed there. So the initial version for mobile will be based on this code and by that just support basic compositing of windows without any effects as those might do transformations or use own GL calls, which have to be ported, too.

Unfortunately the state of the OpenGL API with all the deprecated function calls make it unnecessary difficult to port to the core profile. When passing the geometry to the graphics card we have to either use the legacy code path (for OpenGL 1) or a modern one. Now the modern one we already have, is also deprecated and the code path which has to be used with the core profile and core shaders is incompatible with our existing shaders replacing fixed functionality and the no shader case (but not legacy). Changing all shaders is a lot of work without any gain and the risk to break them. This means that the easy check based on is a shader used, is impossible to decide which code path to use and we have to introduce states into our rendering code to decide which path to use.

These changes will allow us to optionally use OpenGL 3 in KWin. I would like to have at least one effect using a geometry shader in 4.7. And it is required to get KWin on OpenGL ES 2. While that used to be a nice to have feature with a low priority up to now, it looks like it will become very important in the future as Wayland requires ES. This is a very sane decision by the wayland developers. As everybody is writing about wayland nowadays I had to mention it at least once 😛 I am very interested in the topic and hope that it will happen.

And now to something completely different: There is an old saying that something didn’t happen, till you blogged about it. So… Lubos stepped down as KWin Maintainer this week and asked me to become the new Maintainer. I want to thank Lubos for the long time as maintainer and all the work he has done for KWin to make it the extraordinary window manager it is today.

=-=-=-=-=
Powered by Blogilo

Optimization in KWin 4.6

Apart from the scripting interface KWin will not ship any major new feature in 4.6. Most of the work I did for 4.6 was to improve the overall performance of our compositing engine. One of the first improvements I added was for the text and icon overlays in our effects. Now the texture and geometry is always cached and Texture from Pixmap (TFP) is used instead of costly QPixmap/QImage conversations.

Another area which saw great improvements are effects which transform one window. These effects used to cause the whole screen to repaint even if only a small region changed. Most prominent visible is this with the Show Paint effect enabled and for example Sliding Popups or Wobbly Windows. Those effects are now repainting only the changed area which is a great improvement, especially considering that there is less blurring involved. The changes seem to make a difference as developers running trunk asked in #kwin if we changed something in Wobbly Windows. Of course not each effect does benefit from this change. It is an ongoing issue and one effect after another has to be improved.

Another area for optimization is not directly in the compositing engine but more in making use of the advantages of compositing in general. 4.6 will include a new effect to render the startup notification ("bouncing cursor"). The current way involves resizing and shaping a window and moving this window. With compositing enabled this can be a costly operation (for me the performance was so bad, that I disabled the notification completely). The new effect does the same animation as the legacy system without using a window at all. It is just a texture rendered on the screen. This means the whole window transformations are not needed and also the costly TFP operations are not required. Another nice side-effect is that the icon can now use an alpha channel – the old animation transformed each icon to a RGB-only icon. If compositing is not used the system will fall back to the legacy rendering which is btw. not part of KWin.

During the last week I worked on another optimization for our Lanczos filter. Especially in Present Windows the filter was causing some performance regressions. Highlighting one of the windows causes the complete screen to repaint (optimization opportunities see above) which caused a new Lanczos filtering for each visible window. The highlighting animation lasts for something like 200 msec with constant fullscreen repaints. It is obvious why this caused a performance regression. My changes cache each generated texture and use this whenever a repaint without change of content is required. So the highlighting does not require a recreation of the texture (the highlighting is added in a later step). So Present Windows in 4.6 will have the performance as it used to be in 4.4 with the improved thumbnails from 4.5. Of course the cache is useful to all places where the Lanczos sampling is used – be it Box Switch or Taskbar Thumbnails.

I hope that with this change the blacklist is no longer required for performance problems but can be used for driver problems only as intended. Also this allows to integrate the Lanczos improved thumbnails in more effects – e.g. Desktop Grid as in the image.

Improving the performance is of course an ongoing issue and there are still some areas where we can get some clever caching in place. For example blur might be a good candidate for improvements. But this is topic for a blog post "Optimization in KWin 4.7".

=-=-=-=-=
Powered by Blogilo

KWin at UDS

The last week I had the opportunity to attend the Ubuntu Developer Summit in Orlando, Florida. I want to thank Canonical for making this possible. UDS is quite different from other Open Source conferences I have been so far (Akademy, Desktop Summit, Plasma sprints). UDS is really centered about the idea to get things done. The schedule is very exhausting with several one hour sessions per day and up to 14 parallel tracks.

Like Desktop Summit UDS offers the possibility to talk with developers from the "other side". But in addition to the GNOME folks you also have the X, kernel, etc. people around. This is especially from a KWin point of view a real advantage.

I am really glad that I had the opportunity to talk to the people working on Unity. This new desktop shell is a great challenge but also a great opportunity for the free desktops. There is lots to collaborate on and I noticed that Canonical wants to collaborate and is thankful that we as an upstream are collaborative – for example with the Status Notifier/Indicators. This brings me to KWin. As you might have heard of Unity will become the primary desktop shell in Ubuntu 11.04 with Compiz instead of Mutter as the Compositing Manager. This change (which I was aware of for quite some time) is in my opinion the right decision and like Sam I am glad to see development effort in Compiz increase and not Compiz becoming obsoleted by us and GNOME Shell. For us as KWin this brings great opportunities. Canonical is also aware of the driver regressions we run into our latest release. The hardware requirements for Unity are slightly higher than ours. Unity requires Framebuffer Objects (FBO), which we use in various effects (e.g. Blur). Canonical will set up a great testing environment to ensure that the quality of drivers does not regress. With KWin having a similar set of requirements chances are very good that we can profit from that effort. So at least for Kubuntu I am quite confident that 11.04 will be a great release for our users.

Apart from the collaboration on driver requirements the switch to Compiz will hopefully benefit us further. Canonical is really interested in improving the already good collaboration between KWin and Compiz further. Getting us exchanging more ideas and code will help us both and provide our user bases the best window management and compositing experience. The ideas of Canonical are very interesting and I am looking forward to see them happen. So great times are hopefully before us 🙂

One of the first areas we are going to collaborate is the future of window decorations. The idea of client side decorations is dead and we will work on improving what has be started at KDE with allowing the decorations to paint the background of the windows. This will hopefully result in a new theme specification which is finally worked out by the major window managers and cross desktop.

Apart from KWin/Unity I attended quite some Kubuntu sessions and I hope I could give some ideas and feedback to the developers. There is great stuff going on, but I keep that to other bloggers 🙂

=-=-=-=-=
Powered by Blogilo