Tag: OpenGL

I’m very excited about a change I have been working on for KWin since yesterday and which has entered the review process today. The result of it can be seen in this debug output of KWin:

kwin(17876) KWin::Compositor::slotCompositingOptionsInitialized: Initializing OpenGL compositing
kwin(17876) KWin::SceneOpenGL::createScene: Forcing EGL Windowing System through environment variable
kwin(17876) KWin::EglOnXBackend::initRenderingContext: EGL version:  1 . 4
OpenGL vendor string:                   X.Org
OpenGL renderer string:                 Gallium 0.4 on AMD TURKS
OpenGL version string:                  2.1 Mesa 8.0.4
OpenGL shading language version string: 1.20
Driver:                                 R600G
GPU class:                              NI
OpenGL version:                         2.1
GLSL version:                           1.20
Mesa version:                           8.0.4
X server version:                       1.12.3
Linux kernel version:                   3.2
Direct rendering:                       yes
Requires strict binding:                no
GLSL shaders:                           yes
Texture NPOT support:                   yes
kwin(17876) KWin::ShaderManager::initShaders: Ortho Shader is valid
kwin(17876) KWin::ShaderManager::initShaders: Generic Shader is valid
kwin(17876) KWin::ShaderManager::initShaders: Color Shader is valid
kwin(17876) KWin::SceneOpenGL2::SceneOpenGL2: OpenGL 2 compositing successfully initialized

Everything looks quite normal. OpenGL 2 based compositing, everything works fine, all effects load. But still there is something very exciting going on. Instead of GLX, the EGL backend is used, which has been written for the OpenGL ES 2.0 compositor. But this is the normal KWin, not the kwin_gles. We run OpenGL over EGL. To do so the current patch uses an environment variable KWIN_OPENGL_WS which can be set to egl. By default we still use GLX as that’s the safest what we can get at the moment with the available driver collection (and our egl backend needs some more love to be honest). It shows how important the refactoring which I have blogged about last week has been: without it this change would not have been possible.

This is a very exciting change as it means we have one backend to serve both OpenGL and OpenGL ES 2.0, it is also very exciting as it means that KWin is already prepared for the proposed new OpenGL ABI, which will deprecate GLX. The current OpenGL ABI pulls in GLX even if you don’t want GLX at all. I’m very happy that we have this EGL backend as this can turn out to be very important for the adoption of the new OpenGL ABI. It has been noted during the discussion at XDC that there is a chicken and the egg problem by distributions not providing EGL and software therefore not using EGL which means distros do not provide EGL. We can help here as we are a component which most distributions ship and which now requires (currently still optionally) EGL.

As a free software user I’m also excited that this is a change fully built on top of the free OpenGL stack. For a long time quite some functionality of KWin had only been available with proprietary drivers. Now not only the free OpenGL stack provides all we need, it also allows us to move into areas where the proprietary drivers are not yet. With my KWin developer hat on, I of course hope that the proprietary drivers will start to provide EGL, too.

Last but not least I am excited about this change as it is another small step on our long road through the country. It means that changes which will need to happen in the future can also be provided to the OpenGL version of KWin even if we do no longer use GLX as the primary backend.

Today we expect the release of the KDE Plasma Workspaces in version 4.7 which is the first release including an OpenGL ES 2.0/EGL backend in KWin. This does not only allow us to run KWin on OpenGL ES powered devices (I am particular looking forward to see KWin on Tegra 2 devices), but also gives us a much better compositing experience on the desktop systems. Thanks to the work on OpenGL ES 2.0 our default compositing backend is now OpenGL 2.x based instead of OpenGL 1.x as it was till 4.6.

Blogilo

With KWin GLES shortly before the merge into workspace master (after git transition), it’s time to look back what I promised to deliver half a year ago, when I first talked about the idea of porting KWin to OpenGL ES at Akademy.

At the time of Akademy I had not yet started to write any code as I considered 4.5 more important and waited for the git transition. Obviously I stopped waiting for git or there would not be any code written 😉 My schedule presented at Akademy looked like that:

• Remove deprecated OpenGL code
• Remove unused code in Scene and E?ects Lib
• Drop XGL support
• Port to OpenGL ES 1.1 till 4.6
• Port to OpenGL 3 and ES 2.0 till 4.7

Removal of deprecated OpenGL code is mostly done. With GLES KWin has a forward compatible code path, though we still have OpenGL 1.x code for legacy systems and that won’t be dropped in the near future. Considering deprecated OpenGL 1.x code it’s looking quite good, e.g. glBegin/glEnd is removed in the GLES branch. Rendering of geometries is abstracted and the backend performs a runtime check whether legacy or modern rendering is invoked.

For 4.6 I started to port some effects to newer functionality which allowed to drop some then unused code. The remaining functions have been marked as deprecated, but are not yet removed in the GLES branch as they are still used by the Shadow effect (though unavailable in OpenGL ES and broken with OpenGL 2 backend). Given that Shadow is difficult to port, in general broken and unmaintained I consider to just drop it. I am already gathering ideas for a new replacement effect.

The XGL code (XGL was needed to run Compiz before AIGLX was integrated) is still in KWin and should really be dropped. It has been removed from X git tree for quite some time and I doubt anybody has ever run KWin with XGL. There are more things we can now consider to remove: shared memory and "fallback" compositing. Together this will mean a requirement to Texture from Pixmap. That’s pretty well supported nowadays on all drivers and anything else cannot be recommended.

The next bullet point is OpenGL ES 1.1. Well I noticed that it would really clutter the code, so I stopped working on it and switched to going to ES 2.0 directly. My initial plan was to provide 1.1 support at the time of the 4.6 release. Instead we now have ES 2.0 support at the time of 4.6 😀

In 4.7 we will not only have an OpenGL ES 2.0 backend available, it will be integrated into the release. As a side effect it gives us a complete OpenGL 2.x backend which should benefit all users having decent drivers. I’ll soon integrate an option to the advanced settings tab to choose between OpenGL 1.x, 2.x and 3.x. With the 2.x code there, it’s really easy to go to 3.x. We won’t default to it, we will hardly use it (I want to experiment with Geometry Shaders in MagicLamp effect), but I hope that the drivers supporting OpenGL 3 (that’s for us only the NVIDIA blob at the moment, fglrx is only usable with OpenGL 1.x in the composited case) are more efficient if run in a forward compatible profile.

There is still lots to do and I want to concentrate on improving the rendering stack throughout the 4.7 release cycle. It’s quite fortunate that the new rendering stack can be integrated directly after the git transition. So I hope that lots of developers will be using it and can provide feedback for various hardware.

=-=-=-=-=
Powered by Blogilo

Today I ported another bunch of effects to GLES which leaves just five more effects to be ported:

• Blur
• Logout
• Explosion
• Invert
• Snow

Blur, Logout and Snow are the more difficult ones as they have quite some custom code. Snow especially is calling for a rewrite (it was my first effect and I can clearly see how much my coding improved). Additionally to those effects Shadow and Sharpen are not yet ported and won’t be ported. I want to replace Shadow with BeShadowed or with something completely new to support Oxygen shadows also for menus. The Sharpen effect will be removed for various reasons.

But there is also some work left in other effects. Flip/Cover Switch are missing the multi-monitor adjustments and in Cube effect Sphere and Cylinder are not yet supported. Those knowing a little bit about KWin internals will notice, that mostly effects using Shaders are left. The shader files need to be adjusted due to the fact that the existing ones were emulating fixed-functionality. This is not available in GLES, so the shaders need to be rewritten to not use the emulation parts. I did that today with the Looking Glass effect.

I did a screencast of KWin GLES with the nouveau driver. It shows that resizing is much faster and effects are more smooth than our existing rendering stack. The quality of the video is not so good (I still have problems getting recordmydesktop to work properly) that’s why I do not include it in this post. If you don’t mind it, you can find it on blip.tv.

=-=-=-=-=
Powered by Blogilo

Another week is gone and my KWin-GLES branch got further polishing. All developing on the OpenGL ES code is nowadays happening on my notebook using KWin-GLES. So it is already so useable that I can develop on it without real problems. Most of the effects are already ported and can be used. Today I added the first 3D effects, that is CoverSwitch and FlipSwitch and a (locally) half finished Cube Desktop Switch animation. Quite nice to see the new code working as expected 😉 Only eleven effects are not yet ported, but these include some effects which won’t be ported, such as FPS, Sharpen or Shadow effect.

KWin has now three default shipped shaders, which can be used by both effects and the compositing scene. There is one shader to render simple 2D textured graphics. It’s a straight forward simple shader for the normal tasks. A second, more generic shader is available for all kind of transformations: translating, scaling and rotating of the complete screen or windows. Last but not least there is one shader to render simple colored geometries. I noticed I need that one, when I started to port the ShowPaint effect (one of the most important ones).

Going on ES means that everything has to be rendered with shaders and geometries have to be passed using a vertex buffer object (VBO). For 4.6 I added an abstraction to VBOs which is only used by the EffectFrames (overlays in e.g. PresentWindows effect). Now all effects and the scene rendering has to use those VBOs. The way KWin works it’s mostly streamed data, so there is one VBO which can be shared by all effects.

The old code looked like that:

glPushAttrib( GL_CURRENT_BIT | GL_ENABLE_BIT );
glEnable( GL_BLEND );
glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
float alpha = 0.2;
const QColor& color = colors[ color_index ];
glColor4f( color.red() / 255., color.green() / 255., color.blue() / 255., alpha );
glBegin( GL_QUADS );
foreach( const QRect &r, painted.rects())
    {
    glVertex2i( r.x(), r.y());
    glVertex2i( r.x() + r.width(), r.y());
    glVertex2i( r.x() + r.width(), r.y() + r.height());
    glVertex2i( r.x(), r.y() + r.height());
    }
glEnd();
glPopAttrib();

We see here quite some deprecated stuff like glBegin/glEnd, this needs to be replaced by a VBO. Each quad has to become two triangles and of course it’s not allowed to set the color (that has to be done in a shader). Last but not least the attribute stack is not available which means we need to explicitly disable blending. The new code looks like that:

#ifndef KWIN_HAVE_OPENGLES
glPushAttrib( GL_CURRENT_BIT | GL_ENABLE_BIT );
#endif
GLVertexBuffer *vbo = GLVertexBuffer::streamingBuffer();
vbo->reset();
vbo->setUseColor(true);
if (ShaderManager::instance()->isValid()) {
    ShaderManager::instance()->pushShader(ShaderManager::ColorShader);
    vbo->setUseShader(true);
}
glEnable( GL_BLEND );
glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
QColor color = colors[ color_index ];
color.setAlphaF(0.2);
vbo->setColor(color);
QVector<float> verts;
verts.reserve(painted.rects().count()*12);
foreach (const QRect &r, painted.rects()) {
    verts << r.x() + r.width() << r.y();
    verts << r.x() << r.y();
    verts << r.x() << r.y() + r.height();
    verts << r.x() << r.y() + r.height();
    verts << r.x() + r.width() << r.y() + r.height();
    verts << r.x() + r.width() << r.y();
}
vbo->setData(verts.size()/2, 2, verts.data(), NULL);
vbo->render(GL_TRIANGLES);
if (ShaderManager::instance()->isValid()) {
    ShaderManager::instance()->popShader();
}
glDisable( GL_BLEND );
#ifndef KWIN_HAVE_OPENGLES
glPopAttrib();
#endif

Unfortunately we currently need to ifdef the attribute stack. I am planning to go through the complete KWin code and remove all of those later on. The VBO for streaming is used to set the vertices. Also the color is attached to the VBO. When rendering is performed the VBO either uses the legacy glColor or sets a uniform of a bound shader. The new ShaderManager is used to bind the shader for colored geometries. This manager takes care of restoring a previous bound shader when unbinding the shader. As we can see the shader is only bound if it is valid. So this code works for both the legacy rendering as well as the modern shader based rendering. The effect itself does not even need to care about it. All the differences for passing the geometries, setting the color, etc. are abstracted away in the VBO. With these changes porting an effect is mostly straightforward.

Now with most effects and functionality ported I can start looking at the finishing touch. I split several methods in a glx and egl specific variant and noticed later on that the differences are rather small. Those have to be merged again and I have to ensure that everything is still working when not using GLES. I am still aiming for importing everything into master as soon as kdebase-workspace migrates to git. Maybe I will give it one week or two to settle down before merging into master, but I’m quite confident that we will see this in master very soon.

=-=-=-=-=
Powered by Blogilo

Last weekend I could announce that KWin compiles with OpenGL ES headers. This weekend I was able to proceed even more: I got the first windows composited using OpenGL ES 2.0. Not everything is working and there is still lot’s of work to be done and it has not yet been tried on actual devices (yes you can use OpenGL ES on a desktop), but nevertheless it’s a very important step.

As it just looks like normal desktop compositing with rendering artefacts, I don’t show a screenshot. This has to wait till Present Windows (or preferable Cube) works on a mobile device 🙂

I think it is important to give some credits to those who helped me directly or indirectly:

• Fredrik for providing a nice example application showing how texture from pixmap works on OpenGL ES and for all the help on IRC
• Mesa/Gallium for providing ES libraries for desktop systems
• Nouveau for making it possible to use ES on my notebook. Without Nouveau it would have been very difficult to do any testing on ES.
• Debian for packaging Nouveau with 3D support in the experimental repository
• Linaro for investigating about the possibilities for texture from pixmap on OpenGL ES
• Addison Wesley for donating a copy of the fifth edition of the OpenGL SuperBible. Really a great help if you want to develop for the core profile.
• Marco and Alexis for nagging about missing ES support
• Marco for promising to trying out the patches on actual devices

It is incredible how clean the ES code looks compared to the glx backend. I’m really looking forward to be able to drop the legacy OpenGL code and I hope to have the ES port in a state that users can use it as a runtime replacement on desktop systems as well. It would be nice to provide users a modern compositing backend.

=-=-=-=-=
Powered by Blogilo

While others are having fun at the KDE Mobile Sprint in Berlin, I am at home and hacking on KWin mobile and could achieve a huge milestone today: KWin compiles for OpenGL ES 2!

This does not yet mean that you can run KWin on an ES powered MeeGo device, it just means that it compiles. There is still lots of work to be done, so there is no compositing code at all. The code for binding textures from pixmap or performing the actual painting is completely disabled. It’s lots of ifdefs currently and that has to be cleaned up first.

Of course nothing of it will hit 4.6, it’s all 4.7 material and the code lives only in my private git branch. Nevertheless if anybody is interested in the code I can provide the diff (or complain loud enough so that we need git :-P).

The impact to the codebase is not as bad as I feared. After dropping the idea of going to OpenGL ES 1.1 first and writing the required shader for the desktop edition, the changes are rather small. In most cases it’s just a huge set of legacy drawing commands which can be easily removed.

Oh and currently all effects are removed from the build as my shader does not yet support the required transformations and many effects use legacy code, which would be too much work to get it compiling in the current state. We will have to see which effects we want on a mobile profile and enable the build of just those (e.g. Present Windows).

So the next steps are setting up a build environment where I actually can test the stuff. KDE Mobile team: I need your help!

=-=-=-=-=
Powered by Blogilo

As you might know OpenGL comes in two flavors: fixed functionality and programmable pipeline. With the fixed functionality you have to use API calls to influence the execution of each stage of the rendering pipeline. It is a very powerful API allowing you to do most of the stuff we use in KWin. The programmable pipeline allows to directly execute code (called a "Shader") to do vertex and fragment processing. For example we are able to saturate a complete window as a whole with fixed functionality, but we need a fragment shader to be able to change the color of each pixel depending on the input color. This is for example used in the invert effect. A vertex shader can be used to influence the geometry. E.g. we could use it to transform a cube into a sphere. OpenGL 1.x is completely fixed functionality, in OpenGL 2 the programmable pipeline was introduced to exchange parts of the rendering stack, but fixed functionality was still around. With OpenGL 3 everyone expected the fixed functionality to be removed, but it was only deprecated and you can still use it. All the modern calls have been moved into a "core profile".

KWin uses OpenGL 2, that is most of our code is fixed functionality as it was around with OpenGL 1, but we can use shaders to exchange parts of the rendering pipeline to e.g. invert a window. As our shaders are used to emulate fixed functionality the code they use is mostly deprecated as well, KWin does not yet use the core profile.

To make it all more complicated there is also an OpenGL version for Embedded Systems (ES/EGL), which also comes in a flavor for fixed functionality (1.1) and programmable pipeline (2.0). In opposite to the desktop version of OpenGL, the ES version does not include the fixed functionality API calls in ES 2. This means that our source code cannot be compiled directly for ES.

Up to now I only blogged about that I want to port KWin to ES. I started working on it and in 4.6 there is some code already which was written to ease the porting effort. The rendering code has been partly split into legacy painting and modern painting, which makes it easier to just #ifdef away the unsupported calls when compiling for mobile. My initial plan was to first port KWin to ES 1.1 as the code base is closer to 1.1 than to 2.0. The work on it is quite boring and basically stalled. So I decided that this is no solution and instead want to add a core profile compatible rendering path to KWin – this would work both with OpenGL 3 and OpenGL ES 2. Of course no existing code will be removed and most likely it will become opt-in as I expect the changes to break some current effects when using the core profile (at least I can see that cube is partly broken in my special kwin version I’m running here).

With the feature freeze for 4.6 behind us, it’s the perfect time to play around with a new rendering path as it can mature on my systems before it is made available to a broader developer testing. So yesterday I set down and implemented a core profile compatible shader for basic rendering without transformations. This is to be used when no effect is active and only parts of the screen need to be repainted (visible with show paint effect). The first results of this change look promising as our EffectFrame overlays can already be rendered with this new coding path. I would like to present a screenshot, but it is boring as it just looks as always. If you want beautiful pictures, go to Nuno’s blog 😉 As soon as the window rendering is also based on this new shader I have a base to use it in the ES porting. Luckily I can reuse the work I spent for porting to ES 1.1, as the same code has to be removed there. So the initial version for mobile will be based on this code and by that just support basic compositing of windows without any effects as those might do transformations or use own GL calls, which have to be ported, too.

Unfortunately the state of the OpenGL API with all the deprecated function calls make it unnecessary difficult to port to the core profile. When passing the geometry to the graphics card we have to either use the legacy code path (for OpenGL 1) or a modern one. Now the modern one we already have, is also deprecated and the code path which has to be used with the core profile and core shaders is incompatible with our existing shaders replacing fixed functionality and the no shader case (but not legacy). Changing all shaders is a lot of work without any gain and the risk to break them. This means that the easy check based on is a shader used, is impossible to decide which code path to use and we have to introduce states into our rendering code to decide which path to use.

These changes will allow us to optionally use OpenGL 3 in KWin. I would like to have at least one effect using a geometry shader in 4.7. And it is required to get KWin on OpenGL ES 2. While that used to be a nice to have feature with a low priority up to now, it looks like it will become very important in the future as Wayland requires ES. This is a very sane decision by the wayland developers. As everybody is writing about wayland nowadays I had to mention it at least once 😛 I am very interested in the topic and hope that it will happen.

And now to something completely different: There is an old saying that something didn’t happen, till you blogged about it. So… Lubos stepped down as KWin Maintainer this week and asked me to become the new Maintainer. I want to thank Lubos for the long time as maintainer and all the work he has done for KWin to make it the extraordinary window manager it is today.

=-=-=-=-=
Powered by Blogilo