Tag Archives: OpenGL

The return of kwin_gles

Back in 4.x we provided two binaries for KWin: one compiled against OpenGL (kwin) and one compiled against OpenGL ES (kwin_gles). The reason for that is that one can only reasonably link either OpenGL or OpenGL ES and OpenGL ES is only a subset of OpenGL, so one needs to hide the OpenGL calls (especially the OpenGL 1 calls).

With 5.x we were no longer able to provide these two binaries. The reason for that is that OpenGL got “upgraded” in Qt and QtGui itself links either OpenGL or OpenGL ES. To keep the system’s sanity we decided to follow how Qt is compiled. If Qt is compiled with OpenGL support KWin gets compiled with OpenGL support, otherwise with OpenGL ES support.

That’s of course a reasonable design, but it means that it becomes difficult to test the OpenGL ES code paths. One needs a dedicated Qt and all other dependencies compiled against OpenGL ES. Or one needs a nice device like a Nexus 5 with Plasma mobile. This had resulted in breakage already as we were not able to test enough. Such times belong to the past as I have a nice Plasma mobile device to compile and test my KWin on.

But since we introduced OpenGL ES support through a compile time switch, many things have changed. KWin dropped the OpenGL 1 support which means that most of the code which wouldn’t compile with OpenGL ES is just gone. Furthermore we switched to use libepoxy, so we don’t link OpenGL or OpenGL ES at all, but libepoxy which does the right thing for us. With that we are able to remove all the compile time checks. Of course we need runtime checks to ensure that we don’t call functionality which isn’t available on OpenGL ES.

Now with the upcoming 5.5 release we are able to have one binary which serves both OpenGL and OpenGL ES. Note to distributions: the artifact kwinglesutils.so is no longer compiled, please adjust your packaging rules. KWin will use either OpenGL or OpenGL ES depending on what Qt uses.

Given that nowadays it’s also possible to create both an OpenGL and OpenGL ES context at runtime we can also make use of that and introduced a new value for our KWIN_COMPOSE environment variable: O2ES. If that’s specified KWin will use the EGL backend and create an OpenGL ES context. Although it in general is also possible to create an OpenGL ES context through glx we do not support that for simplification. As proof debug output (qdbus org.kde.KWin /KWin supportInformation) from my running KWin instance:

Compositing is active
Compositing Type: OpenGL ES 2.0
OpenGL vendor string: Intel Open Source Technology Center
OpenGL renderer string: Mesa DRI Intel(R) Ivybridge Desktop 
OpenGL version string: OpenGL ES 3.0 Mesa 10.6.8
OpenGL platform interface: EGL
OpenGL shading language version string: OpenGL ES GLSL ES 3.00
Driver: Intel
GPU class: IvyBridge
OpenGL version: 3.0
GLSL version: 3.0
Mesa version: 10.6.8
X server version: 1.17.3
Linux kernel version: 4.2
Direct rendering: Requires strict binding: no
GLSL shaders:  yes
Texture NPOT support:  yes
Virtual Machine:  no
OpenGL 2 Shaders are used
Painting blocks for vertical retrace:  yes

So in a way we have kwin_gles back, it’s different as it’s no longer a dedicated binary, but it’s runtime switchable. For the moment the only way will be the environment variable. I’m reluctant to add a config option as that sounds like quite some chance for breakage.

Thoughts on Vulkan in KWin

Lately I have been asked a lot about using Vulkan in KWin: in fact almost every blog post in the last few months has questions about it and that seems to me there is something to write about it.

So the quick tldr: I don’t have any plans on adding support for Vulkan. Over the last years I ported to so many new technologies, including at least 3 incompatible OpenGL versions. I am not looking forward to another incompatible OpenGL version (whether it’s called Vulkan or OpenGL doesn’t matter to me on that).

Now let’s look at the strength of Vulkan: going closer to the hardware and doing multi-threaded rendering. KWin still performs the rendering in the main GUI thread. Qt tried to do rendering in an off-thread for QtQuick’s scene-graph, in case of KWin we also do that in the main-gui thread. Reworking our compositor to use threading is a lot of work and would also probably improve the performance with OpenGL. Anyway as long as KWin doesn’t support threaded rendering this improvement by Vulkan is rather moot.

Now let’s look on the closer to hardware: yesterday I did a blog post on how I want to use e.g. KMS Planes to get closer to hardware and bypass the OpenGL compositing. Vulkan will allow us to perform rendering closer to hardware, but if we don’t render at all, we don’t benefit from it. Vulkan is probably more power saving, but not using the rendering will save even more.

So Vulkan can only improve when the scene needs to be rendered: e.g. when you wobble a window or spin the desktop cube. I don’t think that those effects are what we should optimize for and spend our time on and even if: there are low-hanging fruits to optimize using OpenGL, we do not yet use OpenGL 3 or 4 features to improve these effects.

Overall Vulkan looks to me like a lot of work as once again we would have to add a new compositing backend, write a new low level interaction, rewrite all shaders, etc. etc. Going to Vulkan early would mean introducing more complexity to KWin, more different code paths our users might use. It sounds like a mood exercise to do so. But I also doubt that there will be useable Vulkan drivers any time soon.

Vulkan is a great technology which will make graphics much better. But I doubt that an application like KWin is the use-case it was developed for.

Now to something else!

KDE is currently running a fundraiser for the Randa Meetings with the topic “Bring Touch to KDE!”. I personally will not participate in the Randa Meetings, but many other KDE developers will go there and we need your support for it. Please help and make the Randa Meetings possible.


Should we target EGL as the default?

When we started the compositing work in KWin the only way to initialize an OpenGL context was by using GLX. In fact GLX is even part of the OpenGL library on Linux. Being an X11 window manager and an X11 compositor it was not a big problem.

Years later we started the effort to get KWin using optionally OpenGL ES in addition to normal OpenGL. One of the differences is that OpenGL ES doesn’t use GLX, but rather EGL. So we gained code to setup compositing using EGL. But this was still a compile time switch and only supported together with using OpenGL ES. If I remember correctly the option to create on OpenGL context on top of EGL was not yet available in our drivers back then.

But there was a time when the FLOSS drivers started to support it and since the 4.10 release in beginning of 2013 we can provide a normal OpenGL context over EGL. This option was not really exposed and only bound to an environment variable. Very few users knew about it and this was also wanted. Using EGL was a good way to shoot yourself in the foot given that not all drivers support it and that we had much more testing on the GLX backend.

Now the situation has changed. With 5.x we expose EGL also through a config interface, but more importantly EGL is the only way to get OpenGL on Wayland. The assumption that every user is using GLX doesn’t hold any more. The assumption that we don’t need to spend so much time (as it’s only relevant in the reduced feature set of mobile devices) on the EGL backend is wrong. It must reach the same quality as the GLX backend, otherwise our Wayland experience suffers.

So this raises an important question: should we try targeting EGL as the default? In the past one of the main reasons to not even think about EGL as default is the reason that the NVIDIA driver didn’t support it. If we would have switched to EGL by default it would have meant dealing with the situation that EGL doesn’t work and we need to fall back to GLX. It would have complicated the startup significantly and would have resulted in our user base using different rendering paths.

Now this one road blocker seems to resolve itself. While I haven’t tested it yet, the latest NVIDIA beta driver supports OpenGL over EGL, so we should be able to have the majority of our user base on the same backend after all. Given that it is only a beta driver and the feature is marked as “experimental” it’s probably still some time till we can default to it.

Nevertheless it’s time to start thinking about it. I encourage NVIDIA users to give EGL now a try and to report issues. Also I encourage all other users to report issues they see with the EGL backend. If we get the EGL backend into a good state maybe 5.6 might be a good point in time to default to it and then start to slowly deprecate the GLX backend with the aim to get it removed in the distant future.

KWin/5, QtQuick 2 and the caused changes for OpenGL

In the KWin/4 days our OpenGL implementation could be based on some assumptions. First of all only KWin creates an OpenGL context, none of the libraries, KWin depends on, uses OpenGL. From this it is possible to derive more assumptions, for example our compositing OpenGL context is always current. Which means that it’s possible to run any OpenGL code in any effect. No matter how the code path got triggered. Also we can link KWin against OpenGL or OpenGL ES. We don’t have to check with other libraries to prevent conflicts.

With KWin/5 these assumptions no longer hold. KWin uses QtQuick 2 and with that we pull in the Qt OpenGL module. One of the direct implications of this change is, that we are no longer able to provide kwin/OpenGL and kwin/OpenGLES at the same time. The compilation of KWin has to follow the compilation of the Qt OpenGL module. So compiling against OpenGLES is only possible if Qt is compiled for OpenGLES, which means that the option to run KWin on OpenGLES is probably non-existing on classic desktop computers and the option to run KWin on OpenGL is probably non-existing on ARM systems such as the Improv. Given that it’s no longer possible to compile both versions at the same time, the binary kwin_gles got removed. A kwin compiled against GLES is just called kwin.

With QtQuick also our assumption that only KWin creates an OpenGL context and makes it current doesn’t hold any more. In fact it could be that at any time Qt creates an OpenGL context and makes it current in the main GUI thread. Now people probably know that QtQuick 2 uses a rendering thread, so that should be fine, right? Well not quite. To decide whether QtQuick uses a rendering thread it creates a dummy context in the main gui thread and makes it current. So our assumption that our context is created once and then kept current, doesn’t hold any more. The first solution to this problem was to make our context current whenever we go into the repaint loop. That worked quite well, till I tested KWin/5 on my SandyBridge notebook.

The problem I stumbled upon is that Qt doesn’t use a rendering thread in all cases. For some hardware it prefers to use the main gui thread. One of them is SandyBridge Mobile, the hardware I am using on my notebook. This showed that the initial solution was not elaborated enough. With another context rendering in the same thread it showed that it’s quite likely that we hit conditions where our compositing context is not current. Resulting in e.g. not rendered window decorations, effects not able to load their textures, etc. etc.

These problems are hopefully solved, now. The effects API is extended by calls to make the context current and I tried to hunt down all effects which do OpenGL calls outside the repaint loop. Unfortunately given the large number of effects it’s still possible that some effects use it incorrectly. It will be difficult to track those down: so please test.

The case when QtQuick uses the main GUI thread for rendering illustrates that we in KWin are not the only ones having incorrect assumptions on OpenGL. QOpenGLContext assumes that every OpenGL context in a Qt application has been created through QOpenGLContext and that an OpenGL context is only made current on the current thread through the QOpenGLContext API. Especially if one uses glx or egl directly to make a context current QOpenGLContext doesn’t notice this and assumes that its own context is still current which can result in nastiness. This is circumvented in KWin now by making sure that QOpenGLContext has correct information once the compositing context is made current. Nevertheless we are still hitting a bug causing a crash. This one is also currently worked around in the development version by enforcing XRender based compositing on the hardware which uses the main GUI thread for rendering. On SandyBridge one can also use the environment variable QT_OPENGL_NO_SANITY_CHECK to force QtQuick to use a dedicated rendering thread as the problem why Qt uses the main gui thread is not relevant to KWin’s usage of QtQuick. KWin also checks for this environment variable and doesn’t force to XRender, if it is set.

Obviously one could question why we are not using QOpenGLContext given that this seems to conflict. We haven’t used Qt’s OpenGL module mostly for historic reasons. Of course I evaluated the option of using QOpenGLContext when investigating this issue and right now in Qt 5.2 it would not be an appropriate solution for the usage in KWin. It’s not possible to create a QOpenGLContext from a native context and even if it were possible our implementation is more flexible. KWin can be compiled against both egl and glx allowing to switch through an environment variable. Qt on the other hand supports either egl or glx, but not both at the same time. If I find time for this, I intend to improve the situation for Qt 5.3, so that we can consider the usage of QOpenGLContext once we depend on Qt 5.3. Switching to Qt’s OpenGL context wrapper would allow us to get rid of a considerable amount of code. I’m especially interested in the QOpenGLFunctions. Obviously that will only be a solution if KWin uses the same windowing system as Qt’s platform plugin. But that’s a problem for another day 😉

Compositing and “lightweight” desktops

In the general discussion about “lightweight” desktop environments I have read a few times that one should disable Compositing in KWin. That’s done in Kubuntu’s low-fat settings package and also something Jos talked about in the context of Klyde.

I have never seen an explanation on why Compositing should matter at all. It mostly boils down to “OpenGL is evil” and “I don’t want 3D”. So let’s leave the “educated guesses” behind us and have a proper look to the question whether Compositing matters for “lightweight”. (Remember: lightweight is a buzz-word without any meaning.)

Let’s start from one of the “lightweight” aspects: Memory. For this it’s important to know that KWin is a Window Manager and Compositor. You cannot just have the Window Manager – even if compositing is disabled. It’s not like xcompmgr which runs in another process. For the binary that gets loaded into memory it doesn’t matter whether compositing is enabled or not. The pure binary containing both window manager and compositor has a size of 1.3 M (on Debian testing, that should be KDE SC 4.8). But when you enable compositing an additional library gets loaded containing the Effects. They are all in one library to not have to open 40 different files. (Again on Debian testing) This library has a size of 783 K. The effects do not link any library which is not already linked by KWin core, so that’s what we are talking about from additional memory costs when using any desktop effect from binary size: 783 K!

Obviously the binary mapped into memory is just one aspect. There are also additional data structures which need to be created on the heap. I have no idea how much is really needed and cannot properly look at my system because I have a KWin with debugging symbols loaded. What’s important to know is that only Effects one uses will get loaded and use memory for their data structures. Also the Effects do not load everything they need but wait till they get activated. E.g. the cube effect will only load a background texture if the cube gets activated. But of course effects do not have anything to do with just using compositing. If you don’t want an effect, just disable it.

The biggest cost from compositing in the memory perspective is probably using the XComposite extension which redirects the rendering into an X pixmap. That’s something which will go away once we are on Wayland as application and compositor share the buffer the application is rendering to.

But even so the usage of the additional memory is not just something which is thrown out. It’s just another case of the time-memory tradeoff. Unrelated windows do not need to repaint if a window get’s unmapped, moved or resized. Just look at this comment from the non-composited switching of virtual desktops:

During virt. desktop switching, desktop areas covered by windows that are going to be hidden are first obscured by new windows with no background ( i.e. transparent ) placed right below the windows. These invisible windows are removed after the switch is complete. Reduces desktop ( wallpaper ) repaints during desktop switching

Without compositing we move the time-memory tradeoff towards using more CPU (create those windows) to prevent even more CPU usage. With compositing we don’t have to do anything of it. So disabling compositing is obviously not the silver bullet for being “lightweight”. It just means moving the time-memory tradeoff slider towards CPU.

What remains is the OpenGL question. I know that many people think it’s about having a “3D desktop”. But the desktop I’m using is quite 2D-ish. There are people saying that 2D is just a special case of 3D. Looking at OpenGL that is certainly the case. As long as you don’t use any effect which needs the “3D aspect” of OpenGL, KWin just does 2D with OpenGL. Using OpenGL means to make use of hardware which is designed to do these operations instead of doing it on the CPU which is not designed for it. But what if one doesn’t have the hardware for it? A valid question, but just not in the context of KWin. If your system only provides software rendering KWin automatically falls back to XRender based compositing.

The last aspect to mention in the context of OpenGL is texture from pixmap. I don’t know how it’s implemented in the drivers so I don’t know whether that needs additional memory or not. But if you are concerned about it: XRender does not need a texture from pixmap.

So as you can see compositing comes with a little bit more memory usage but reduces CPU usage. It’s nothing I could say which one is generally better, but less CPU usage means a sleeping CPU. Granted some effects do animations – that also requires CPU usage. If you are concerned about that it’s better to disable the effects or set the global animation speed to instant than to just disable compositing completely.

And if you try to be “lightweight” and play with the compositing settings: one unrelated change can ruin all your benefit. Just imagine you would disable compositing because you want to be “lightweight” and then you enable a window decoration which uses QML (that is interpreted code) and does animations?

Fallback mode in KDE Plasma Workspaces

Recently there has been a lot of buzz about non-composited fallback modes in various Desktop Shells and of course I have been asked several times about the fallback modes in KDE Plasma workspaces and whether they would be removed, too. Now instead of answering the same question again and again I decided to write a blog post to discuss the situation in more detail.

The first thing to notice is that KDE Plasma workspaces do not have a non-composited fallback mode in the way GNOME Shell or Unity used to have. The main difference is that our window manager (KWin) is able to act as a non-composited, XRender based compositor and OpenGL (ES) based compositor. This means that we do not have to maintain two window managers in order to provide non-composited setups.

The second major difference is that the Desktop Shell (either KDE Plasma Desktop, KDE Plasma Netbook or KDE Plasma Active) is not a plugin to the compositor but a separate application running in an own process. This allows to use a completely different window manager together with the Desktop Shell. Some years ago it was not that uncommon to use Compiz together with Plasma, though we see less and less such usage patterns (mostly caused by Compiz no longer being available in most major distributions except Ubuntu). This is a design decision which served us well and we do not plan to change it. Some time ago I brought up the topic, but more experienced developers (mainly Aaron) illustrated nicely the advantages of our design choices.

Of course we need some code to be able to adapt to non-composited mode inside the Plasma workspaces. But this code would be needed anyway. Why? To answer this question we have to go back in time to when the development of Plasma had started – that is around 2006/2007. Back then OpenGL based compositing had been a rather new topic (I think I first heard about Compiz in 2004 or 2005) and it had only been possible with the binary blobs and hacks such as Xgl. The Intel hardware back then was not powerful enough and drivers were lacking, too. With other words: going an OpenGL-based only path seemed not feasible at that time and consider that the fallback modes in Unity and GNOME Shell got only removed in late 2012/early 2013. This is more than half a decade later – an eternity in IT. Also KWin did not enable compositing support by default till 4.2, so the first releases of KDE Plasma were non-composited by default.

The solution inside Plasma was to make our themes aware of compositing. That is each theme contains an opaque element set which will be used when in non-composited mode. The fact whether compositing is used, is globally available through a standardized X11 manager selection. So whenever the compositing state is changed Plasma is notified and switches the used elements. Simple and elegant. This functionality served us quite well when we introduced the blur effect. Translucency is actually a very difficult topic. If you choose a too strong translucency level the text is no longer readable, if you choose a translucency fitted for text readability the translucency is hardly visible. The solution to this problem is the blur effect. It provides easily readable text even with strong translucency levels. But even today we do not enable the blur effect by default on all hardware as it is an expensive effect and we would not dare to enable it on embedded devices (personally I doubt that Windows 8 removed the blur because the designers disliked it). So we see we need themes with different levels of translucency depending on whether blur is available or not. The design decisions for the Plasma themes provided us the solution directly. As well it means that even if we would drop the non-composited mode we would have to keep our design to support the blur selection.

For most of the applications it hardly matters whether the desktop is composited or not. Only very few applications use the alpha channel in the first place and if they want to use it, everything’s nicely abstracted inside Qt.

Inside the window manager it also doesn’t really matter. Of course our window decorations are aware of whether compositing is used or not and especially Oxygen makes use of it by providing an adjusted look for the non-composited case, so there we have some overhead, but except that there are hardly any adjustments.

This also means that we would not gain anything from requiring compositing. The compositor is a separate module inside the window manager which just gets started by the window manager on start-up. It lives more or less for itself and is just notified when a new window is added to the window manager and so on. Nothing inside the window manager really depends on the compositor. That means it would be more work to get KWin requiring compositing than to keep supporting the non-composited mode. And having the non-composited mode around allows us to do things like turning compositing off when running games or heavy OpenGL based applications such as Blender. So if you want to get some of the now finally available games for Linux, KDE Plasma should be your primary choice to enjoy the game. I have also heard of users switching to KDE Plasma because we still provide non OpenGL based setups.

What remains is the question of what will change with Qt 5? Obviously not much. Plasma will continue to provide a non-composited and a composited theme, but Plasma itself will require OpenGL due to being based on QtQuick 2. I do hope that llvmpipe will be sufficient for Plasma in case of non available drivers. The same is true for KWin: our QtQuick based elements will require OpenGL 2, but that does not mean that we will require OpenGL based compositing. We will still provide non-composited mode and if the XRender based compositor get’s ported to XCB (we have lots of work there to get involved 😉 will continue to provide XRender based compositing as well as OpenGL 1 and OpenGL 2 based. Since the refactoring of the OpenGL based compositor I do not see much difficulties with keeping the OpenGL 1 based compositor around, though if it turns out to not be used anymore, it might be removed.

The last remaining question is what will change with Wayland? Obviously Wayland requires to have a compositor, but it does not require us to stop supporting X11. KWin will continue to be an X11 based window manager allowing you to not use Wayland at all. And even if you choose to use a Wayland based KWin nothing says that it needs to be a compositor on top of OpenGL. Whether we will provide non-OpenGL based compositors for Wayland is a question for the future, though.

A final remark: this blog post describes the situation inside the KDE Plasma workspaces and the design decisions we have done. This is not a critic at the design decisions other projects have done and I am not able to judge whether their decision is right or wrong and it is completely irrelevant what my opinion is on their decisions.

A journey through virtualization

This weekend I had a look on enabling the OpenGL compositor when running in a virtual machine. My assumption was that given that the next version of Ubuntu is going to require OpenGL, there should be no problem with getting OpenGL running in a virtual machine.

I normally use KVM when I have need for a virtual machine running a Linux guest. As far as I know you cannot get OpenGL with this technology, so I hadn’t looked into it for quite some time.

My first attempt was to just use the project neon weekly build. Unfortunately I failed to convert it to a VirtualBox image, nevertheless I decided to run it and verify that KVM does not support OpenGL. Well I was rather surprised once the machine was up to see that OpenGL was functional, but well it was slow. Restarting KWin showed me that there is a missing slot (which I am aware of and is already fixed in one of the pending reviews) in master which makes the fallback to XRender not work, but KWin just uses llvmpipe. OK, result incorrect, but at least validated that KVM is not working.

I then decided to try VirtualBox with an install of Kubuntu Natty. Install went well but no OpenGL available. The Xorg log file told me that VirtualBox does not know the XServer version. Installed the latest version of VirtualBox, installed the latest version of guest additions in the Kubuntu guest, but no success: driver does not work.

So I decided to test with an outdated system, aka Debian Testing as I’m also running this on my workstation. Here I was more successful. After installing the system in the virtual machine OpenGL worked out-of-the-box. And what surprised me even more was that OpenGL in KWin worked when using KWIN_DIRECT_GL=1 and KWIN_COMPOSE=O. This means OpenGL works in general and it’s only blocked by KWin not knowing the driver.

KWin with OpenGL in VirtualBox

The main reason is that KWin does not use direct rendering if it finds an unknown driver and VirtualBox only provides OpenGL in direct rendering mode. If LIBGL_ALWAYS_INDIRECT is set, it falls back to Mesa’s software rasterization and KWin sets this environment variable if it doesn’t know the driver.

This meant I had to perform some coding to recognize the driver and had to test it in the virtual machine. Thanks to host and guest running the same version of Debian I was able to setup a NFS share and just use my normal development version of KWin.

After a little bit of hacking and fighting with NFS I got the driver reported correctly and KWin running with OpenGL without using environment variable hacks. But VirtualBox’s driver is not complete. Some methods we (optionally) use from GLX 1.3 are not implemented and trying to use it resulted in warnings. That unfortunately needed a hack inside KWin (it’s small and well contained, so it’s not much of an issue).

With VirtualBox working I decided to also look into VMware and have installed VMware Player for the first time in years. Converted my Debian image into a VMware compatible format, enabled 3D acceleration and started the system, just to be presented with warnings that 3D is not working.

The Internet told me to add

mks.gl.allowBlacklistedDrivers = TRUE

to the vmx file. But that did not resolve the issue. The two warnings went down to one, but still a warning. Apparently one has to install libtxc-dxtn-s2tc0 in order to get 3D support.

Installed and no warnings: yeah. Started the system, but no OpenGL. Installed the guest additions, but it told me that the system already has X11 integration. Looked into the Xorg log and yes it complained that something is wrong with the module. So no OpenGL inside Debian guest.

KWin with OpenGL in VMWare player

Resolved to my previous plan of trying Kubuntu 12.10. Started it in a live session and KWin is not composited. But glxinfo reported a driver, so I restarted KWin with KWIN_COMPOSE=O and there it was: OpenGL running also in VMware.

Looking at KWin’s output I was quite happy to see that it is a Gallium3D driver, which means we don’t need any adjustments at all. KWin uses OpenGL out-of-the-box, it was only not running, because of the live session of Kubuntu. So all that was needed was adding some detection code, that KWin doesn’t print out “unknown” in the debug output. Unfortunately using my KWin from Debian did not work, so I could not yet verify the output, but that can be done once it hit project neon.

Once the patches are merged into master KWin will use OpenGL in virtual machines – if supported. But given the experience it looks like at least on Linux hosts OpenGL is basically not supported in default setups. I would like to also support Parallels and Windows Virtual PC, but am of course lacking both required host operating system and licenses.

To llvmpipe or Not?

Some days ago I prepared a patch which moves the decision which compositor to use inside the driver detection code. This allows us to remove hacks we currently have inside the compositor initialization code to move specific drivers (e.g. the software rasterizer) to a better suited compositor. The base idea is that we always give a user the compositor which is best suited for her system. It’s in my opinion not a good idea to push users on the OpenGL 2 compositor, just because the hardware supports the required extensions, if we know that the hardware is actually not capable of performing well. For such users the OpenGL 1 based compositor is better suited or maybe even the XRender based compositor.

As well for rather old hardware it’s a better idea to not provide any OpenGL based compositing even if theoretically the driver supports it.

But automatism is not always the best solution. Yes giving from experience the best suited compositor is a good idea, but still it makes sense to allow the user to overwrite it. In this case the “user” is mostly myself and other developers who have to be able to easily switch the compositor. For me this is most easily done using an environment variable as I am restarting KWin all the time anyway and is faster than going into a UI and changing the settings and afterward changing them back. Of course although this is controlled through an environment variable, there are also settings to control the chosen compositor. E.g. you can select XRender or disable the OpenGL 2 based compositor. But there is no UI option to enforce the OpenGL 2 compositor if our driver recommendation is to use OpenGL 1 or XRender (we don’t want to provide users the option to destroy their system).

One of the side-effects of this driver based recommendation is that the hack to switch to XRender for software based rasterizers has been removed in the patch. Instead these drivers now recommend to use the XRender based compositor. And in combination with the changes in the handling of the KWIN_COMPOSE environment variable this results in the possibility to run OpenGL based compositing with the llvmpipe driver.

But as you can see one has to manually enforce it through an environment variable and I would not recommend anybody (or any distribution) to use it. OpenGL based compositing over llvmpipe is considered as a fallback approach for other OpenGL based desktop environments like GNOME Shell and Unity. But I do not consider it as a solution for the KDE Plasma Workspaces. I think we have better solutions for fallback (XRender or no compositing).

Let’s have a look on the use cases for llvmpipe based compositing:

  • Outdated hardware not providing OpenGL
  • Too new hardware without driver yet
  • Embedded hardware without driver yet
  • Virtual machines
  • RMS not like binary drivers

If your hardware is too old to provide the requirements for OpenGL based compositing, it is quite likely that your CPU is also rather old. We can assume a single-core CPU of either the Pentium era or an early Atom. You don’t want your CPU which is not suited for this task to spend all the time rendering the desktop. If I enable llvmpipe for KWin on my system (quad core) it puts about one core under full steam.

The case that there is new hardware without drivers available on Linux has luckily become a very rare case. Basic modesetting support for NVIDIA’s latest hardware (Kepler) was available on the same day as the announcement of the hardware. And the blobs do support also their latest hardware. So in most cases all that is needed is installing the latest driver or upgrading the distribution. If you spend lots of money for new hardware you probably want to use it and not have the CPU emulate what your expensive hardware is supposed to do.

In the case of embedded hardware it’s unfortunately still rather common that there are no drivers available. But ARM based systems are also not known for having extreme power. If you don’t want your high-end system to use llvmpipe based OpenGL compositing you even less want your low-end system to do it. You bought your Raspberry Pi hopefully for more useful things than running at full-steam to render the desktop.

If you run a virtual machine your resources are rather limited. It needs to share the resources with other virtual machines and your host system. If you run in a cloud you probably don’t want to pay for rendering the desktop through llvmpipe. You have a task to solve and the more CPU that task gets the less you have to pay. In case of a local system you also don’t want to have the virtual system run at full-steam just to render the desktop. There are really better things to do with a VM.

Last but not least there are the users who don’t want to run binary drivers. Sorry I cannot help you there. If you really think you want to spend lots of money for hardware that you don’t use, that’s fine with me. But I would suggest to only buy hardware which has free drivers available and there are nowadays options from all of the three big GPU providers.

Overall we see that llvmpipe as a backend for OpenGL based compositing is actually not needed in the case of the KDE Plasma Workspaces. We have working fallbacks like XRender or no compositing at all and those do not put the system under full-steam.

Given that I am not considering to turn llvmpipe into an option for KWin. I think that most users who would need llvmpipe are better suited with XRender based compositing even if a few effects are missing then. For me as a developer on the other hand llvmpipe is a very interesting target as it can help to optimize our rendering stack.

Of course what I considered in this blog post is only relevant for OpenGL based compositing of the KDE Plasma Workspaces. We do have proper fallbacks like XRender or no compositing which is not available in other environments which decided to use llvmpipe as a fallback. They have to do a different evaluation and I cannot provide it for them as my evaluation result would be: use KWin 😉

Who needs GLX? KWin does not

I’m very excited about a change I have been working on for KWin since yesterday and which has entered the review process today. The result of it can be seen in this debug output of KWin:

kwin(17876) KWin::Compositor::slotCompositingOptionsInitialized: Initializing OpenGL compositing
kwin(17876) KWin::SceneOpenGL::createScene: Forcing EGL Windowing System through environment variable
kwin(17876) KWin::EglOnXBackend::initRenderingContext: EGL version:  1 . 4
OpenGL vendor string:                   X.Org
OpenGL renderer string:                 Gallium 0.4 on AMD TURKS
OpenGL version string:                  2.1 Mesa 8.0.4
OpenGL shading language version string: 1.20
Driver:                                 R600G
GPU class:                              NI
OpenGL version:                         2.1
GLSL version:                           1.20
Mesa version:                           8.0.4
X server version:                       1.12.3
Linux kernel version:                   3.2
Direct rendering:                       yes
Requires strict binding:                no
GLSL shaders:                           yes
Texture NPOT support:                   yes
kwin(17876) KWin::ShaderManager::initShaders: Ortho Shader is valid
kwin(17876) KWin::ShaderManager::initShaders: Generic Shader is valid
kwin(17876) KWin::ShaderManager::initShaders: Color Shader is valid
kwin(17876) KWin::SceneOpenGL2::SceneOpenGL2: OpenGL 2 compositing successfully initialized

Everything looks quite normal. OpenGL 2 based compositing, everything works fine, all effects load. But still there is something very exciting going on. Instead of GLX, the EGL backend is used, which has been written for the OpenGL ES 2.0 compositor. But this is the normal KWin, not the kwin_gles. We run OpenGL over EGL. To do so the current patch uses an environment variable KWIN_OPENGL_WS which can be set to egl. By default we still use GLX as that’s the safest what we can get at the moment with the available driver collection (and our egl backend needs some more love to be honest). It shows how important the refactoring which I have blogged about last week has been: without it this change would not have been possible.

This is a very exciting change as it means we have one backend to serve both OpenGL and OpenGL ES 2.0, it is also very exciting as it means that KWin is already prepared for the proposed new OpenGL ABI, which will deprecate GLX. The current OpenGL ABI pulls in GLX even if you don’t want GLX at all. I’m very happy that we have this EGL backend as this can turn out to be very important for the adoption of the new OpenGL ABI. It has been noted during the discussion at XDC that there is a chicken and the egg problem by distributions not providing EGL and software therefore not using EGL which means distros do not provide EGL. We can help here as we are a component which most distributions ship and which now requires (currently still optionally) EGL.

As a free software user I’m also excited that this is a change fully built on top of the free OpenGL stack. For a long time quite some functionality of KWin had only been available with proprietary drivers. Now not only the free OpenGL stack provides all we need, it also allows us to move into areas where the proprietary drivers are not yet. With my KWin developer hat on, I of course hope that the proprietary drivers will start to provide EGL, too.

Last but not least I am excited about this change as it is another small step on our long road through the country. It means that changes which will need to happen in the future can also be provided to the OpenGL version of KWin even if we do no longer use GLX as the primary backend.

Workspace Sprint

The last week I was allowed to spend in Pineda de Mar for the Workspace sprint. As others have already pointed out it was quite a different sprint with not much hacking going on and not so much technical discussions into the details. For this we will have a Tokamak sprint in Randa later this year.

Our sprint was mostly focused on community building and getting everyone on the same knowledge level. The weeks leading to the sprint showed that this was needed and I myself – although being involved in Plasma for quite some time – had to catch up on many things I was not aware of. The sprint helped us to identify for example weak spots in our internal documentation and getting this fixed is now one of the highest priorities.

The games Kevin played with us really helped us to identify where we want to go and what that means for our future development. This will also influence KWin’s development as we finally understand how Activities and Desktops go together and that helps us in KWin to properly focus development effort on it. Over the last cycles we did not really work on activities support in KWin as we were very unsure on how the window manager is involved in the process. This is now much clearer to me – I just need to document it and luckily Aurélien already started that process.

Luckily I was able to also get some hacking done. As Mr Gwenview was sitting next to me, I started to add some OpenGL magic to the image viewer. Sometimes I just need to work on a different piece of code and to get away from the daily work on KWin. The result is a much smoother zooming experience. At the moment the feature set is not yet identical to the existing implementation, but it’s on a very promising way. Very pleasing is also that at least on my system it works also well with llvmpipe, which could be a very nice solution for the case that a user does not have a decent OpenGL graphics card. For users who like to experiment, feel free to give a try to graesslin/opengl branch in the gwenview repository.

Overall it was a very nice sprint, but also very exhausting. I’m glad that I have one day to relax before going back to work. And I am really glad to see everyone again at this years Akademy where I will give a talk about KWin scripting.

Please thank the Frankfurt Airport for this blog post. Thanks to needing more than forty minutes to get my luggage from the plane to the belt I missed one train by two minutes (it went > 1h after landing) with the next train only after one hour instead of the normal half hourly service.