Tag Archives: OpenGL

KWin/5, QtQuick 2 and the caused changes for OpenGL

In the KWin/4 days our OpenGL implementation could be based on some assumptions. First of all only KWin creates an OpenGL context, none of the libraries, KWin depends on, uses OpenGL. From this it is possible to derive more assumptions, for example our compositing OpenGL context is always current. Which means that it’s possible to run any OpenGL code in any effect. No matter how the code path got triggered. Also we can link KWin against OpenGL or OpenGL ES. We don’t have to check with other libraries to prevent conflicts.

With KWin/5 these assumptions no longer hold. KWin uses QtQuick 2 and with that we pull in the Qt OpenGL module. One of the direct implications of this change is, that we are no longer able to provide kwin/OpenGL and kwin/OpenGLES at the same time. The compilation of KWin has to follow the compilation of the Qt OpenGL module. So compiling against OpenGLES is only possible if Qt is compiled for OpenGLES, which means that the option to run KWin on OpenGLES is probably non-existing on classic desktop computers and the option to run KWin on OpenGL is probably non-existing on ARM systems such as the Improv. Given that it’s no longer possible to compile both versions at the same time, the binary kwin_gles got removed. A kwin compiled against GLES is just called kwin.

With QtQuick also our assumption that only KWin creates an OpenGL context and makes it current doesn’t hold any more. In fact it could be that at any time Qt creates an OpenGL context and makes it current in the main GUI thread. Now people probably know that QtQuick 2 uses a rendering thread, so that should be fine, right? Well not quite. To decide whether QtQuick uses a rendering thread it creates a dummy context in the main gui thread and makes it current. So our assumption that our context is created once and then kept current, doesn’t hold any more. The first solution to this problem was to make our context current whenever we go into the repaint loop. That worked quite well, till I tested KWin/5 on my SandyBridge notebook.

The problem I stumbled upon is that Qt doesn’t use a rendering thread in all cases. For some hardware it prefers to use the main gui thread. One of them is SandyBridge Mobile, the hardware I am using on my notebook. This showed that the initial solution was not elaborated enough. With another context rendering in the same thread it showed that it’s quite likely that we hit conditions where our compositing context is not current. Resulting in e.g. not rendered window decorations, effects not able to load their textures, etc. etc.

These problems are hopefully solved, now. The effects API is extended by calls to make the context current and I tried to hunt down all effects which do OpenGL calls outside the repaint loop. Unfortunately given the large number of effects it’s still possible that some effects use it incorrectly. It will be difficult to track those down: so please test.

The case when QtQuick uses the main GUI thread for rendering illustrates that we in KWin are not the only ones having incorrect assumptions on OpenGL. QOpenGLContext assumes that every OpenGL context in a Qt application has been created through QOpenGLContext and that an OpenGL context is only made current on the current thread through the QOpenGLContext API. Especially if one uses glx or egl directly to make a context current QOpenGLContext doesn’t notice this and assumes that its own context is still current which can result in nastiness. This is circumvented in KWin now by making sure that QOpenGLContext has correct information once the compositing context is made current. Nevertheless we are still hitting a bug causing a crash. This one is also currently worked around in the development version by enforcing XRender based compositing on the hardware which uses the main GUI thread for rendering. On SandyBridge one can also use the environment variable QT_OPENGL_NO_SANITY_CHECK to force QtQuick to use a dedicated rendering thread as the problem why Qt uses the main gui thread is not relevant to KWin’s usage of QtQuick. KWin also checks for this environment variable and doesn’t force to XRender, if it is set.

Obviously one could question why we are not using QOpenGLContext given that this seems to conflict. We haven’t used Qt’s OpenGL module mostly for historic reasons. Of course I evaluated the option of using QOpenGLContext when investigating this issue and right now in Qt 5.2 it would not be an appropriate solution for the usage in KWin. It’s not possible to create a QOpenGLContext from a native context and even if it were possible our implementation is more flexible. KWin can be compiled against both egl and glx allowing to switch through an environment variable. Qt on the other hand supports either egl or glx, but not both at the same time. If I find time for this, I intend to improve the situation for Qt 5.3, so that we can consider the usage of QOpenGLContext once we depend on Qt 5.3. Switching to Qt’s OpenGL context wrapper would allow us to get rid of a considerable amount of code. I’m especially interested in the QOpenGLFunctions. Obviously that will only be a solution if KWin uses the same windowing system as Qt’s platform plugin. But that’s a problem for another day ;-)

Compositing and “lightweight” desktops

In the general discussion about “lightweight” desktop environments I have read a few times that one should disable Compositing in KWin. That’s done in Kubuntu’s low-fat settings package and also something Jos talked about in the context of Klyde.

I have never seen an explanation on why Compositing should matter at all. It mostly boils down to “OpenGL is evil” and “I don’t want 3D”. So let’s leave the “educated guesses” behind us and have a proper look to the question whether Compositing matters for “lightweight”. (Remember: lightweight is a buzz-word without any meaning.)

Let’s start from one of the “lightweight” aspects: Memory. For this it’s important to know that KWin is a Window Manager and Compositor. You cannot just have the Window Manager – even if compositing is disabled. It’s not like xcompmgr which runs in another process. For the binary that gets loaded into memory it doesn’t matter whether compositing is enabled or not. The pure binary containing both window manager and compositor has a size of 1.3 M (on Debian testing, that should be KDE SC 4.8). But when you enable compositing an additional library gets loaded containing the Effects. They are all in one library to not have to open 40 different files. (Again on Debian testing) This library has a size of 783 K. The effects do not link any library which is not already linked by KWin core, so that’s what we are talking about from additional memory costs when using any desktop effect from binary size: 783 K!

Obviously the binary mapped into memory is just one aspect. There are also additional data structures which need to be created on the heap. I have no idea how much is really needed and cannot properly look at my system because I have a KWin with debugging symbols loaded. What’s important to know is that only Effects one uses will get loaded and use memory for their data structures. Also the Effects do not load everything they need but wait till they get activated. E.g. the cube effect will only load a background texture if the cube gets activated. But of course effects do not have anything to do with just using compositing. If you don’t want an effect, just disable it.

The biggest cost from compositing in the memory perspective is probably using the XComposite extension which redirects the rendering into an X pixmap. That’s something which will go away once we are on Wayland as application and compositor share the buffer the application is rendering to.

But even so the usage of the additional memory is not just something which is thrown out. It’s just another case of the time-memory tradeoff. Unrelated windows do not need to repaint if a window get’s unmapped, moved or resized. Just look at this comment from the non-composited switching of virtual desktops:

During virt. desktop switching, desktop areas covered by windows that are going to be hidden are first obscured by new windows with no background ( i.e. transparent ) placed right below the windows. These invisible windows are removed after the switch is complete. Reduces desktop ( wallpaper ) repaints during desktop switching

Without compositing we move the time-memory tradeoff towards using more CPU (create those windows) to prevent even more CPU usage. With compositing we don’t have to do anything of it. So disabling compositing is obviously not the silver bullet for being “lightweight”. It just means moving the time-memory tradeoff slider towards CPU.

What remains is the OpenGL question. I know that many people think it’s about having a “3D desktop”. But the desktop I’m using is quite 2D-ish. There are people saying that 2D is just a special case of 3D. Looking at OpenGL that is certainly the case. As long as you don’t use any effect which needs the “3D aspect” of OpenGL, KWin just does 2D with OpenGL. Using OpenGL means to make use of hardware which is designed to do these operations instead of doing it on the CPU which is not designed for it. But what if one doesn’t have the hardware for it? A valid question, but just not in the context of KWin. If your system only provides software rendering KWin automatically falls back to XRender based compositing.

The last aspect to mention in the context of OpenGL is texture from pixmap. I don’t know how it’s implemented in the drivers so I don’t know whether that needs additional memory or not. But if you are concerned about it: XRender does not need a texture from pixmap.

So as you can see compositing comes with a little bit more memory usage but reduces CPU usage. It’s nothing I could say which one is generally better, but less CPU usage means a sleeping CPU. Granted some effects do animations – that also requires CPU usage. If you are concerned about that it’s better to disable the effects or set the global animation speed to instant than to just disable compositing completely.

And if you try to be “lightweight” and play with the compositing settings: one unrelated change can ruin all your benefit. Just imagine you would disable compositing because you want to be “lightweight” and then you enable a window decoration which uses QML (that is interpreted code) and does animations?

Fallback mode in KDE Plasma Workspaces

Recently there has been a lot of buzz about non-composited fallback modes in various Desktop Shells and of course I have been asked several times about the fallback modes in KDE Plasma workspaces and whether they would be removed, too. Now instead of answering the same question again and again I decided to write a blog post to discuss the situation in more detail.

The first thing to notice is that KDE Plasma workspaces do not have a non-composited fallback mode in the way GNOME Shell or Unity used to have. The main difference is that our window manager (KWin) is able to act as a non-composited, XRender based compositor and OpenGL (ES) based compositor. This means that we do not have to maintain two window managers in order to provide non-composited setups.

The second major difference is that the Desktop Shell (either KDE Plasma Desktop, KDE Plasma Netbook or KDE Plasma Active) is not a plugin to the compositor but a separate application running in an own process. This allows to use a completely different window manager together with the Desktop Shell. Some years ago it was not that uncommon to use Compiz together with Plasma, though we see less and less such usage patterns (mostly caused by Compiz no longer being available in most major distributions except Ubuntu). This is a design decision which served us well and we do not plan to change it. Some time ago I brought up the topic, but more experienced developers (mainly Aaron) illustrated nicely the advantages of our design choices.

Of course we need some code to be able to adapt to non-composited mode inside the Plasma workspaces. But this code would be needed anyway. Why? To answer this question we have to go back in time to when the development of Plasma had started – that is around 2006/2007. Back then OpenGL based compositing had been a rather new topic (I think I first heard about Compiz in 2004 or 2005) and it had only been possible with the binary blobs and hacks such as Xgl. The Intel hardware back then was not powerful enough and drivers were lacking, too. With other words: going an OpenGL-based only path seemed not feasible at that time and consider that the fallback modes in Unity and GNOME Shell got only removed in late 2012/early 2013. This is more than half a decade later – an eternity in IT. Also KWin did not enable compositing support by default till 4.2, so the first releases of KDE Plasma were non-composited by default.

The solution inside Plasma was to make our themes aware of compositing. That is each theme contains an opaque element set which will be used when in non-composited mode. The fact whether compositing is used, is globally available through a standardized X11 manager selection. So whenever the compositing state is changed Plasma is notified and switches the used elements. Simple and elegant. This functionality served us quite well when we introduced the blur effect. Translucency is actually a very difficult topic. If you choose a too strong translucency level the text is no longer readable, if you choose a translucency fitted for text readability the translucency is hardly visible. The solution to this problem is the blur effect. It provides easily readable text even with strong translucency levels. But even today we do not enable the blur effect by default on all hardware as it is an expensive effect and we would not dare to enable it on embedded devices (personally I doubt that Windows 8 removed the blur because the designers disliked it). So we see we need themes with different levels of translucency depending on whether blur is available or not. The design decisions for the Plasma themes provided us the solution directly. As well it means that even if we would drop the non-composited mode we would have to keep our design to support the blur selection.

For most of the applications it hardly matters whether the desktop is composited or not. Only very few applications use the alpha channel in the first place and if they want to use it, everything’s nicely abstracted inside Qt.

Inside the window manager it also doesn’t really matter. Of course our window decorations are aware of whether compositing is used or not and especially Oxygen makes use of it by providing an adjusted look for the non-composited case, so there we have some overhead, but except that there are hardly any adjustments.

This also means that we would not gain anything from requiring compositing. The compositor is a separate module inside the window manager which just gets started by the window manager on start-up. It lives more or less for itself and is just notified when a new window is added to the window manager and so on. Nothing inside the window manager really depends on the compositor. That means it would be more work to get KWin requiring compositing than to keep supporting the non-composited mode. And having the non-composited mode around allows us to do things like turning compositing off when running games or heavy OpenGL based applications such as Blender. So if you want to get some of the now finally available games for Linux, KDE Plasma should be your primary choice to enjoy the game. I have also heard of users switching to KDE Plasma because we still provide non OpenGL based setups.

What remains is the question of what will change with Qt 5? Obviously not much. Plasma will continue to provide a non-composited and a composited theme, but Plasma itself will require OpenGL due to being based on QtQuick 2. I do hope that llvmpipe will be sufficient for Plasma in case of non available drivers. The same is true for KWin: our QtQuick based elements will require OpenGL 2, but that does not mean that we will require OpenGL based compositing. We will still provide non-composited mode and if the XRender based compositor get’s ported to XCB (we have lots of work there to get involved ;-) will continue to provide XRender based compositing as well as OpenGL 1 and OpenGL 2 based. Since the refactoring of the OpenGL based compositor I do not see much difficulties with keeping the OpenGL 1 based compositor around, though if it turns out to not be used anymore, it might be removed.

The last remaining question is what will change with Wayland? Obviously Wayland requires to have a compositor, but it does not require us to stop supporting X11. KWin will continue to be an X11 based window manager allowing you to not use Wayland at all. And even if you choose to use a Wayland based KWin nothing says that it needs to be a compositor on top of OpenGL. Whether we will provide non-OpenGL based compositors for Wayland is a question for the future, though.

A final remark: this blog post describes the situation inside the KDE Plasma workspaces and the design decisions we have done. This is not a critic at the design decisions other projects have done and I am not able to judge whether their decision is right or wrong and it is completely irrelevant what my opinion is on their decisions.

A journey through virtualization

This weekend I had a look on enabling the OpenGL compositor when running in a virtual machine. My assumption was that given that the next version of Ubuntu is going to require OpenGL, there should be no problem with getting OpenGL running in a virtual machine.

I normally use KVM when I have need for a virtual machine running a Linux guest. As far as I know you cannot get OpenGL with this technology, so I hadn’t looked into it for quite some time.

My first attempt was to just use the project neon weekly build. Unfortunately I failed to convert it to a VirtualBox image, nevertheless I decided to run it and verify that KVM does not support OpenGL. Well I was rather surprised once the machine was up to see that OpenGL was functional, but well it was slow. Restarting KWin showed me that there is a missing slot (which I am aware of and is already fixed in one of the pending reviews) in master which makes the fallback to XRender not work, but KWin just uses llvmpipe. OK, result incorrect, but at least validated that KVM is not working.

I then decided to try VirtualBox with an install of Kubuntu Natty. Install went well but no OpenGL available. The Xorg log file told me that VirtualBox does not know the XServer version. Installed the latest version of VirtualBox, installed the latest version of guest additions in the Kubuntu guest, but no success: driver does not work.

So I decided to test with an outdated system, aka Debian Testing as I’m also running this on my workstation. Here I was more successful. After installing the system in the virtual machine OpenGL worked out-of-the-box. And what surprised me even more was that OpenGL in KWin worked when using KWIN_DIRECT_GL=1 and KWIN_COMPOSE=O. This means OpenGL works in general and it’s only blocked by KWin not knowing the driver.

KWin with OpenGL in VirtualBox

The main reason is that KWin does not use direct rendering if it finds an unknown driver and VirtualBox only provides OpenGL in direct rendering mode. If LIBGL_ALWAYS_INDIRECT is set, it falls back to Mesa’s software rasterization and KWin sets this environment variable if it doesn’t know the driver.

This meant I had to perform some coding to recognize the driver and had to test it in the virtual machine. Thanks to host and guest running the same version of Debian I was able to setup a NFS share and just use my normal development version of KWin.

After a little bit of hacking and fighting with NFS I got the driver reported correctly and KWin running with OpenGL without using environment variable hacks. But VirtualBox’s driver is not complete. Some methods we (optionally) use from GLX 1.3 are not implemented and trying to use it resulted in warnings. That unfortunately needed a hack inside KWin (it’s small and well contained, so it’s not much of an issue).

With VirtualBox working I decided to also look into VMware and have installed VMware Player for the first time in years. Converted my Debian image into a VMware compatible format, enabled 3D acceleration and started the system, just to be presented with warnings that 3D is not working.

The Internet told me to add

mks.gl.allowBlacklistedDrivers = TRUE

to the vmx file. But that did not resolve the issue. The two warnings went down to one, but still a warning. Apparently one has to install libtxc-dxtn-s2tc0 in order to get 3D support.

Installed and no warnings: yeah. Started the system, but no OpenGL. Installed the guest additions, but it told me that the system already has X11 integration. Looked into the Xorg log and yes it complained that something is wrong with the module. So no OpenGL inside Debian guest.

KWin with OpenGL in VMWare player

Resolved to my previous plan of trying Kubuntu 12.10. Started it in a live session and KWin is not composited. But glxinfo reported a driver, so I restarted KWin with KWIN_COMPOSE=O and there it was: OpenGL running also in VMware.

Looking at KWin’s output I was quite happy to see that it is a Gallium3D driver, which means we don’t need any adjustments at all. KWin uses OpenGL out-of-the-box, it was only not running, because of the live session of Kubuntu. So all that was needed was adding some detection code, that KWin doesn’t print out “unknown” in the debug output. Unfortunately using my KWin from Debian did not work, so I could not yet verify the output, but that can be done once it hit project neon.

Once the patches are merged into master KWin will use OpenGL in virtual machines – if supported. But given the experience it looks like at least on Linux hosts OpenGL is basically not supported in default setups. I would like to also support Parallels and Windows Virtual PC, but am of course lacking both required host operating system and licenses.

To llvmpipe or Not?

Some days ago I prepared a patch which moves the decision which compositor to use inside the driver detection code. This allows us to remove hacks we currently have inside the compositor initialization code to move specific drivers (e.g. the software rasterizer) to a better suited compositor. The base idea is that we always give a user the compositor which is best suited for her system. It’s in my opinion not a good idea to push users on the OpenGL 2 compositor, just because the hardware supports the required extensions, if we know that the hardware is actually not capable of performing well. For such users the OpenGL 1 based compositor is better suited or maybe even the XRender based compositor.

As well for rather old hardware it’s a better idea to not provide any OpenGL based compositing even if theoretically the driver supports it.

But automatism is not always the best solution. Yes giving from experience the best suited compositor is a good idea, but still it makes sense to allow the user to overwrite it. In this case the “user” is mostly myself and other developers who have to be able to easily switch the compositor. For me this is most easily done using an environment variable as I am restarting KWin all the time anyway and is faster than going into a UI and changing the settings and afterward changing them back. Of course although this is controlled through an environment variable, there are also settings to control the chosen compositor. E.g. you can select XRender or disable the OpenGL 2 based compositor. But there is no UI option to enforce the OpenGL 2 compositor if our driver recommendation is to use OpenGL 1 or XRender (we don’t want to provide users the option to destroy their system).

One of the side-effects of this driver based recommendation is that the hack to switch to XRender for software based rasterizers has been removed in the patch. Instead these drivers now recommend to use the XRender based compositor. And in combination with the changes in the handling of the KWIN_COMPOSE environment variable this results in the possibility to run OpenGL based compositing with the llvmpipe driver.

But as you can see one has to manually enforce it through an environment variable and I would not recommend anybody (or any distribution) to use it. OpenGL based compositing over llvmpipe is considered as a fallback approach for other OpenGL based desktop environments like GNOME Shell and Unity. But I do not consider it as a solution for the KDE Plasma Workspaces. I think we have better solutions for fallback (XRender or no compositing).

Let’s have a look on the use cases for llvmpipe based compositing:

  • Outdated hardware not providing OpenGL
  • Too new hardware without driver yet
  • Embedded hardware without driver yet
  • Virtual machines
  • RMS not like binary drivers

If your hardware is too old to provide the requirements for OpenGL based compositing, it is quite likely that your CPU is also rather old. We can assume a single-core CPU of either the Pentium era or an early Atom. You don’t want your CPU which is not suited for this task to spend all the time rendering the desktop. If I enable llvmpipe for KWin on my system (quad core) it puts about one core under full steam.

The case that there is new hardware without drivers available on Linux has luckily become a very rare case. Basic modesetting support for NVIDIA’s latest hardware (Kepler) was available on the same day as the announcement of the hardware. And the blobs do support also their latest hardware. So in most cases all that is needed is installing the latest driver or upgrading the distribution. If you spend lots of money for new hardware you probably want to use it and not have the CPU emulate what your expensive hardware is supposed to do.

In the case of embedded hardware it’s unfortunately still rather common that there are no drivers available. But ARM based systems are also not known for having extreme power. If you don’t want your high-end system to use llvmpipe based OpenGL compositing you even less want your low-end system to do it. You bought your Raspberry Pi hopefully for more useful things than running at full-steam to render the desktop.

If you run a virtual machine your resources are rather limited. It needs to share the resources with other virtual machines and your host system. If you run in a cloud you probably don’t want to pay for rendering the desktop through llvmpipe. You have a task to solve and the more CPU that task gets the less you have to pay. In case of a local system you also don’t want to have the virtual system run at full-steam just to render the desktop. There are really better things to do with a VM.

Last but not least there are the users who don’t want to run binary drivers. Sorry I cannot help you there. If you really think you want to spend lots of money for hardware that you don’t use, that’s fine with me. But I would suggest to only buy hardware which has free drivers available and there are nowadays options from all of the three big GPU providers.

Overall we see that llvmpipe as a backend for OpenGL based compositing is actually not needed in the case of the KDE Plasma Workspaces. We have working fallbacks like XRender or no compositing at all and those do not put the system under full-steam.

Given that I am not considering to turn llvmpipe into an option for KWin. I think that most users who would need llvmpipe are better suited with XRender based compositing even if a few effects are missing then. For me as a developer on the other hand llvmpipe is a very interesting target as it can help to optimize our rendering stack.

Of course what I considered in this blog post is only relevant for OpenGL based compositing of the KDE Plasma Workspaces. We do have proper fallbacks like XRender or no compositing which is not available in other environments which decided to use llvmpipe as a fallback. They have to do a different evaluation and I cannot provide it for them as my evaluation result would be: use KWin ;-)

Who needs GLX? KWin does not

I’m very excited about a change I have been working on for KWin since yesterday and which has entered the review process today. The result of it can be seen in this debug output of KWin:

kwin(17876) KWin::Compositor::slotCompositingOptionsInitialized: Initializing OpenGL compositing
kwin(17876) KWin::SceneOpenGL::createScene: Forcing EGL Windowing System through environment variable
kwin(17876) KWin::EglOnXBackend::initRenderingContext: EGL version:  1 . 4
OpenGL vendor string:                   X.Org
OpenGL renderer string:                 Gallium 0.4 on AMD TURKS
OpenGL version string:                  2.1 Mesa 8.0.4
OpenGL shading language version string: 1.20
Driver:                                 R600G
GPU class:                              NI
OpenGL version:                         2.1
GLSL version:                           1.20
Mesa version:                           8.0.4
X server version:                       1.12.3
Linux kernel version:                   3.2
Direct rendering:                       yes
Requires strict binding:                no
GLSL shaders:                           yes
Texture NPOT support:                   yes
kwin(17876) KWin::ShaderManager::initShaders: Ortho Shader is valid
kwin(17876) KWin::ShaderManager::initShaders: Generic Shader is valid
kwin(17876) KWin::ShaderManager::initShaders: Color Shader is valid
kwin(17876) KWin::SceneOpenGL2::SceneOpenGL2: OpenGL 2 compositing successfully initialized

Everything looks quite normal. OpenGL 2 based compositing, everything works fine, all effects load. But still there is something very exciting going on. Instead of GLX, the EGL backend is used, which has been written for the OpenGL ES 2.0 compositor. But this is the normal KWin, not the kwin_gles. We run OpenGL over EGL. To do so the current patch uses an environment variable KWIN_OPENGL_WS which can be set to egl. By default we still use GLX as that’s the safest what we can get at the moment with the available driver collection (and our egl backend needs some more love to be honest). It shows how important the refactoring which I have blogged about last week has been: without it this change would not have been possible.

This is a very exciting change as it means we have one backend to serve both OpenGL and OpenGL ES 2.0, it is also very exciting as it means that KWin is already prepared for the proposed new OpenGL ABI, which will deprecate GLX. The current OpenGL ABI pulls in GLX even if you don’t want GLX at all. I’m very happy that we have this EGL backend as this can turn out to be very important for the adoption of the new OpenGL ABI. It has been noted during the discussion at XDC that there is a chicken and the egg problem by distributions not providing EGL and software therefore not using EGL which means distros do not provide EGL. We can help here as we are a component which most distributions ship and which now requires (currently still optionally) EGL.

As a free software user I’m also excited that this is a change fully built on top of the free OpenGL stack. For a long time quite some functionality of KWin had only been available with proprietary drivers. Now not only the free OpenGL stack provides all we need, it also allows us to move into areas where the proprietary drivers are not yet. With my KWin developer hat on, I of course hope that the proprietary drivers will start to provide EGL, too.

Last but not least I am excited about this change as it is another small step on our long road through the country. It means that changes which will need to happen in the future can also be provided to the OpenGL version of KWin even if we do no longer use GLX as the primary backend.

Workspace Sprint

The last week I was allowed to spend in Pineda de Mar for the Workspace sprint. As others have already pointed out it was quite a different sprint with not much hacking going on and not so much technical discussions into the details. For this we will have a Tokamak sprint in Randa later this year.

Our sprint was mostly focused on community building and getting everyone on the same knowledge level. The weeks leading to the sprint showed that this was needed and I myself – although being involved in Plasma for quite some time – had to catch up on many things I was not aware of. The sprint helped us to identify for example weak spots in our internal documentation and getting this fixed is now one of the highest priorities.

The games Kevin played with us really helped us to identify where we want to go and what that means for our future development. This will also influence KWin’s development as we finally understand how Activities and Desktops go together and that helps us in KWin to properly focus development effort on it. Over the last cycles we did not really work on activities support in KWin as we were very unsure on how the window manager is involved in the process. This is now much clearer to me – I just need to document it and luckily Aurélien already started that process.

Luckily I was able to also get some hacking done. As Mr Gwenview was sitting next to me, I started to add some OpenGL magic to the image viewer. Sometimes I just need to work on a different piece of code and to get away from the daily work on KWin. The result is a much smoother zooming experience. At the moment the feature set is not yet identical to the existing implementation, but it’s on a very promising way. Very pleasing is also that at least on my system it works also well with llvmpipe, which could be a very nice solution for the case that a user does not have a decent OpenGL graphics card. For users who like to experiment, feel free to give a try to graesslin/opengl branch in the gwenview repository.

Overall it was a very nice sprint, but also very exhausting. I’m glad that I have one day to relax before going back to work. And I am really glad to see everyone again at this years Akademy where I will give a talk about KWin scripting.

Please thank the Frankfurt Airport for this blog post. Thanks to needing more than forty minutes to get my luggage from the plane to the belt I missed one train by two minutes (it went > 1h after landing) with the next train only after one hour instead of the normal half hourly service.

Running KWin with OpenGL ES 2.0

Today we expect the release of the KDE Plasma Workspaces in version 4.7 which is the first release including an OpenGL ES 2.0/EGL backend in KWin. This does not only allow us to run KWin on OpenGL ES powered devices (I am particular looking forward to see KWin on Tegra 2 devices), but also gives us a much better compositing experience on the desktop systems. Thanks to the work on OpenGL ES 2.0 our default compositing backend is now OpenGL 2.x based instead of OpenGL 1.x as it was till 4.6.

I want to thank all users who tested our beta version and release candidates and ensured by that a regression free update. I am very glad on how smooth this major transition of the compositor seems to work according to the very low number of bug reports in this cycle.
On Mesa enabled desktop systems there is also the possibility to go a step further and leave GLX completely behind by compiling KWin for OpenGL ES/EGL only. My personal experience is that it is often a much better experience to use the OpenGL ES build. If you compile KWin from sources ensure that the OpenGL ES 2.0 and EGL development libraries are around and enable the CMake option “KWIN_BUILD_WITH_OPENGLES”. If everything is setup correctly, CMake will tell you “Compiling KWin for mobile”.
Of course compiling from sources is not what most of our users do, so it needs distro help. Thanks to the awesome Kubuntu crew there will be a kwin_gles package available in Kubuntu 11.10. So users can switch to use this package.
But that’s still not perfect as users need to know that there is a better GLES variant. So today I pushed a change to master to build KWin twice: once with OpenGL and once with OpenGL ES. Thanks to our CMake Gods for all the help on getting this done :-). Now if OpenGL ES is available at buildtime a “kwin_gles” binary is compiled and can be used just like “kwin”. For all users of masters, do a git pull, recompile and run: kwin_gles –replace & and enjoy.
And that’s also just an intermediate step. Our GSoC student Arthur is currently working in close collaboration with me on decoupling our compositor from the window manager so that we can build just the compositor once with OpenGL and once with OpenGL ES. We will then put the compositor into a plugin and do a runtime check whether OpenGL ES is working and if it is we will load the OpenGL ES compositor. So if everything works fine, in 4.8 we will be able to offer and default to OpenGL ES 2.0 for most of our users.
This is also important for Phase 1 of Plasma on Wayland, which I will highlight in my talk at the Desktop Summit: Compositing after X – KWin on the road to Wayland. Hope to see you there.

On the Road to Modern OpenGL (ES)

With KWin GLES shortly before the merge into workspace master (after git transition), it’s time to look back what I promised to deliver half a year ago, when I first talked about the idea of porting KWin to OpenGL ES at Akademy.

At the time of Akademy I had not yet started to write any code as I considered 4.5 more important and waited for the git transition. Obviously I stopped waiting for git or there would not be any code written ;-) My schedule presented at Akademy looked like that:

  • Remove deprecated OpenGL code
  • Remove unused code in Scene and Effects Lib
  • Drop XGL support
  • Port to OpenGL ES 1.1 till 4.6
  • Port to OpenGL 3 and ES 2.0 till 4.7

Removal of deprecated OpenGL code is mostly done. With GLES KWin has a forward compatible code path, though we still have OpenGL 1.x code for legacy systems and that won’t be dropped in the near future. Considering deprecated OpenGL 1.x code it’s looking quite good, e.g. glBegin/glEnd is removed in the GLES branch. Rendering of geometries is abstracted and the backend performs a runtime check whether legacy or modern rendering is invoked.

For 4.6 I started to port some effects to newer functionality which allowed to drop some then unused code. The remaining functions have been marked as deprecated, but are not yet removed in the GLES branch as they are still used by the Shadow effect (though unavailable in OpenGL ES and broken with OpenGL 2 backend). Given that Shadow is difficult to port, in general broken and unmaintained I consider to just drop it. I am already gathering ideas for a new replacement effect.

The XGL code (XGL was needed to run Compiz before AIGLX was integrated) is still in KWin and should really be dropped. It has been removed from X git tree for quite some time and I doubt anybody has ever run KWin with XGL. There are more things we can now consider to remove: shared memory and "fallback" compositing. Together this will mean a requirement to Texture from Pixmap. That’s pretty well supported nowadays on all drivers and anything else cannot be recommended.

The next bullet point is OpenGL ES 1.1. Well I noticed that it would really clutter the code, so I stopped working on it and switched to going to ES 2.0 directly. My initial plan was to provide 1.1 support at the time of the 4.6 release. Instead we now have ES 2.0 support at the time of 4.6 :-D

In 4.7 we will not only have an OpenGL ES 2.0 backend available, it will be integrated into the release. As a side effect it gives us a complete OpenGL 2.x backend which should benefit all users having decent drivers. I’ll soon integrate an option to the advanced settings tab to choose between OpenGL 1.x, 2.x and 3.x. With the 2.x code there, it’s really easy to go to 3.x. We won’t default to it, we will hardly use it (I want to experiment with Geometry Shaders in MagicLamp effect), but I hope that the drivers supporting OpenGL 3 (that’s for us only the NVIDIA blob at the moment, fglrx is only usable with OpenGL 1.x in the composited case) are more efficient if run in a forward compatible profile.

There is still lots to do and I want to concentrate on improving the rendering stack throughout the 4.7 release cycle. It’s quite fortunate that the new rendering stack can be integrated directly after the git transition. So I hope that lots of developers will be using it and can provide feedback for various hardware.

Powered by Blogilo

Down to Five KWin Effects

Today I ported another bunch of effects to GLES which leaves just five more effects to be ported:

  • Blur
  • Logout
  • Explosion
  • Invert
  • Snow

Blur, Logout and Snow are the more difficult ones as they have quite some custom code. Snow especially is calling for a rewrite (it was my first effect and I can clearly see how much my coding improved). Additionally to those effects Shadow and Sharpen are not yet ported and won’t be ported. I want to replace Shadow with BeShadowed or with something completely new to support Oxygen shadows also for menus. The Sharpen effect will be removed for various reasons.

But there is also some work left in other effects. Flip/Cover Switch are missing the multi-monitor adjustments and in Cube effect Sphere and Cylinder are not yet supported. Those knowing a little bit about KWin internals will notice, that mostly effects using Shaders are left. The shader files need to be adjusted due to the fact that the existing ones were emulating fixed-functionality. This is not available in GLES, so the shaders need to be rewritten to not use the emulation parts. I did that today with the Looking Glass effect.

I did a screencast of KWin GLES with the nouveau driver. It shows that resizing is much faster and effects are more smooth than our existing rendering stack. The quality of the video is not so good (I still have problems getting recordmydesktop to work properly) that’s why I do not include it in this post. If you don’t mind it, you can find it on blip.tv.

Powered by Blogilo