Martin's Blog

With KWin GLES shortly before the merge into workspace master (after git transition), it’s time to look back what I promised to deliver half a year ago, when I first talked about the idea of porting KWin to OpenGL ES at Akademy.

At the time of Akademy I had not yet started to write any code as I considered 4.5 more important and waited for the git transition. Obviously I stopped waiting for git or there would not be any code written 😉 My schedule presented at Akademy looked like that:

• Remove deprecated OpenGL code
• Remove unused code in Scene and E?ects Lib
• Drop XGL support
• Port to OpenGL ES 1.1 till 4.6
• Port to OpenGL 3 and ES 2.0 till 4.7

Removal of deprecated OpenGL code is mostly done. With GLES KWin has a forward compatible code path, though we still have OpenGL 1.x code for legacy systems and that won’t be dropped in the near future. Considering deprecated OpenGL 1.x code it’s looking quite good, e.g. glBegin/glEnd is removed in the GLES branch. Rendering of geometries is abstracted and the backend performs a runtime check whether legacy or modern rendering is invoked.

For 4.6 I started to port some effects to newer functionality which allowed to drop some then unused code. The remaining functions have been marked as deprecated, but are not yet removed in the GLES branch as they are still used by the Shadow effect (though unavailable in OpenGL ES and broken with OpenGL 2 backend). Given that Shadow is difficult to port, in general broken and unmaintained I consider to just drop it. I am already gathering ideas for a new replacement effect.

The XGL code (XGL was needed to run Compiz before AIGLX was integrated) is still in KWin and should really be dropped. It has been removed from X git tree for quite some time and I doubt anybody has ever run KWin with XGL. There are more things we can now consider to remove: shared memory and "fallback" compositing. Together this will mean a requirement to Texture from Pixmap. That’s pretty well supported nowadays on all drivers and anything else cannot be recommended.

The next bullet point is OpenGL ES 1.1. Well I noticed that it would really clutter the code, so I stopped working on it and switched to going to ES 2.0 directly. My initial plan was to provide 1.1 support at the time of the 4.6 release. Instead we now have ES 2.0 support at the time of 4.6 😀

In 4.7 we will not only have an OpenGL ES 2.0 backend available, it will be integrated into the release. As a side effect it gives us a complete OpenGL 2.x backend which should benefit all users having decent drivers. I’ll soon integrate an option to the advanced settings tab to choose between OpenGL 1.x, 2.x and 3.x. With the 2.x code there, it’s really easy to go to 3.x. We won’t default to it, we will hardly use it (I want to experiment with Geometry Shaders in MagicLamp effect), but I hope that the drivers supporting OpenGL 3 (that’s for us only the NVIDIA blob at the moment, fglrx is only usable with OpenGL 1.x in the composited case) are more efficient if run in a forward compatible profile.

There is still lots to do and I want to concentrate on improving the rendering stack throughout the 4.7 release cycle. It’s quite fortunate that the new rendering stack can be integrated directly after the git transition. So I hope that lots of developers will be using it and can provide feedback for various hardware.

=-=-=-=-=
Powered by Blogilo

Today I ported another bunch of effects to GLES which leaves just five more effects to be ported:

• Blur
• Logout
• Explosion
• Invert
• Snow

Blur, Logout and Snow are the more difficult ones as they have quite some custom code. Snow especially is calling for a rewrite (it was my first effect and I can clearly see how much my coding improved). Additionally to those effects Shadow and Sharpen are not yet ported and won’t be ported. I want to replace Shadow with BeShadowed or with something completely new to support Oxygen shadows also for menus. The Sharpen effect will be removed for various reasons.

But there is also some work left in other effects. Flip/Cover Switch are missing the multi-monitor adjustments and in Cube effect Sphere and Cylinder are not yet supported. Those knowing a little bit about KWin internals will notice, that mostly effects using Shaders are left. The shader files need to be adjusted due to the fact that the existing ones were emulating fixed-functionality. This is not available in GLES, so the shaders need to be rewritten to not use the emulation parts. I did that today with the Looking Glass effect.

I did a screencast of KWin GLES with the nouveau driver. It shows that resizing is much faster and effects are more smooth than our existing rendering stack. The quality of the video is not so good (I still have problems getting recordmydesktop to work properly) that’s why I do not include it in this post. If you don’t mind it, you can find it on blip.tv.

=-=-=-=-=
Powered by Blogilo

Another week is gone and my KWin-GLES branch got further polishing. All developing on the OpenGL ES code is nowadays happening on my notebook using KWin-GLES. So it is already so useable that I can develop on it without real problems. Most of the effects are already ported and can be used. Today I added the first 3D effects, that is CoverSwitch and FlipSwitch and a (locally) half finished Cube Desktop Switch animation. Quite nice to see the new code working as expected 😉 Only eleven effects are not yet ported, but these include some effects which won’t be ported, such as FPS, Sharpen or Shadow effect.

KWin has now three default shipped shaders, which can be used by both effects and the compositing scene. There is one shader to render simple 2D textured graphics. It’s a straight forward simple shader for the normal tasks. A second, more generic shader is available for all kind of transformations: translating, scaling and rotating of the complete screen or windows. Last but not least there is one shader to render simple colored geometries. I noticed I need that one, when I started to port the ShowPaint effect (one of the most important ones).

Going on ES means that everything has to be rendered with shaders and geometries have to be passed using a vertex buffer object (VBO). For 4.6 I added an abstraction to VBOs which is only used by the EffectFrames (overlays in e.g. PresentWindows effect). Now all effects and the scene rendering has to use those VBOs. The way KWin works it’s mostly streamed data, so there is one VBO which can be shared by all effects.

The old code looked like that:

glPushAttrib( GL_CURRENT_BIT | GL_ENABLE_BIT );
glEnable( GL_BLEND );
glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
float alpha = 0.2;
const QColor& color = colors[ color_index ];
glColor4f( color.red() / 255., color.green() / 255., color.blue() / 255., alpha );
glBegin( GL_QUADS );
foreach( const QRect &r, painted.rects())
    {
    glVertex2i( r.x(), r.y());
    glVertex2i( r.x() + r.width(), r.y());
    glVertex2i( r.x() + r.width(), r.y() + r.height());
    glVertex2i( r.x(), r.y() + r.height());
    }
glEnd();
glPopAttrib();

We see here quite some deprecated stuff like glBegin/glEnd, this needs to be replaced by a VBO. Each quad has to become two triangles and of course it’s not allowed to set the color (that has to be done in a shader). Last but not least the attribute stack is not available which means we need to explicitly disable blending. The new code looks like that:

#ifndef KWIN_HAVE_OPENGLES
glPushAttrib( GL_CURRENT_BIT | GL_ENABLE_BIT );
#endif
GLVertexBuffer *vbo = GLVertexBuffer::streamingBuffer();
vbo->reset();
vbo->setUseColor(true);
if (ShaderManager::instance()->isValid()) {
    ShaderManager::instance()->pushShader(ShaderManager::ColorShader);
    vbo->setUseShader(true);
}
glEnable( GL_BLEND );
glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
QColor color = colors[ color_index ];
color.setAlphaF(0.2);
vbo->setColor(color);
QVector<float> verts;
verts.reserve(painted.rects().count()*12);
foreach (const QRect &r, painted.rects()) {
    verts << r.x() + r.width() << r.y();
    verts << r.x() << r.y();
    verts << r.x() << r.y() + r.height();
    verts << r.x() << r.y() + r.height();
    verts << r.x() + r.width() << r.y() + r.height();
    verts << r.x() + r.width() << r.y();
}
vbo->setData(verts.size()/2, 2, verts.data(), NULL);
vbo->render(GL_TRIANGLES);
if (ShaderManager::instance()->isValid()) {
    ShaderManager::instance()->popShader();
}
glDisable( GL_BLEND );
#ifndef KWIN_HAVE_OPENGLES
glPopAttrib();
#endif

Unfortunately we currently need to ifdef the attribute stack. I am planning to go through the complete KWin code and remove all of those later on. The VBO for streaming is used to set the vertices. Also the color is attached to the VBO. When rendering is performed the VBO either uses the legacy glColor or sets a uniform of a bound shader. The new ShaderManager is used to bind the shader for colored geometries. This manager takes care of restoring a previous bound shader when unbinding the shader. As we can see the shader is only bound if it is valid. So this code works for both the legacy rendering as well as the modern shader based rendering. The effect itself does not even need to care about it. All the differences for passing the geometries, setting the color, etc. are abstracted away in the VBO. With these changes porting an effect is mostly straightforward.

Now with most effects and functionality ported I can start looking at the finishing touch. I split several methods in a glx and egl specific variant and noticed later on that the differences are rather small. Those have to be merged again and I have to ensure that everything is still working when not using GLES. I am still aiming for importing everything into master as soon as kdebase-workspace migrates to git. Maybe I will give it one week or two to settle down before merging into master, but I’m quite confident that we will see this in master very soon.

=-=-=-=-=
Powered by Blogilo

Last weekend I could announce that KWin compiles with OpenGL ES headers. This weekend I was able to proceed even more: I got the first windows composited using OpenGL ES 2.0. Not everything is working and there is still lot’s of work to be done and it has not yet been tried on actual devices (yes you can use OpenGL ES on a desktop), but nevertheless it’s a very important step.

As it just looks like normal desktop compositing with rendering artefacts, I don’t show a screenshot. This has to wait till Present Windows (or preferable Cube) works on a mobile device 🙂

I think it is important to give some credits to those who helped me directly or indirectly:

• Fredrik for providing a nice example application showing how texture from pixmap works on OpenGL ES and for all the help on IRC
• Mesa/Gallium for providing ES libraries for desktop systems
• Nouveau for making it possible to use ES on my notebook. Without Nouveau it would have been very difficult to do any testing on ES.
• Debian for packaging Nouveau with 3D support in the experimental repository
• Linaro for investigating about the possibilities for texture from pixmap on OpenGL ES
• Addison Wesley for donating a copy of the fifth edition of the OpenGL SuperBible. Really a great help if you want to develop for the core profile.
• Marco and Alexis for nagging about missing ES support
• Marco for promising to trying out the patches on actual devices

It is incredible how clean the ES code looks compared to the glx backend. I’m really looking forward to be able to drop the legacy OpenGL code and I hope to have the ES port in a state that users can use it as a runtime replacement on desktop systems as well. It would be nice to provide users a modern compositing backend.

=-=-=-=-=
Powered by Blogilo

While others are having fun at the KDE Mobile Sprint in Berlin, I am at home and hacking on KWin mobile and could achieve a huge milestone today: KWin compiles for OpenGL ES 2!

This does not yet mean that you can run KWin on an ES powered MeeGo device, it just means that it compiles. There is still lots of work to be done, so there is no compositing code at all. The code for binding textures from pixmap or performing the actual painting is completely disabled. It’s lots of ifdefs currently and that has to be cleaned up first.

Of course nothing of it will hit 4.6, it’s all 4.7 material and the code lives only in my private git branch. Nevertheless if anybody is interested in the code I can provide the diff (or complain loud enough so that we need git :-P).

The impact to the codebase is not as bad as I feared. After dropping the idea of going to OpenGL ES 1.1 first and writing the required shader for the desktop edition, the changes are rather small. In most cases it’s just a huge set of legacy drawing commands which can be easily removed.

Oh and currently all effects are removed from the build as my shader does not yet support the required transformations and many effects use legacy code, which would be too much work to get it compiling in the current state. We will have to see which effects we want on a mobile profile and enable the build of just those (e.g. Present Windows).

So the next steps are setting up a build environment where I actually can test the stuff. KDE Mobile team: I need your help!

=-=-=-=-=
Powered by Blogilo