Tag Archives: KWin

October Plasma on Wayland Update: all about geometry

Last month our Wayland efforts made a huge step forward. In KWin we are now at a state where I think the big underlying work is finished, we entered the finishing line of the KWin Wayland porting. The whole system though still needs a little bit more work.

The big remaining task which I worked on last month was geometry handling. That is simplified: moving and resizing windows. Sounds relatively easy, but isn’t. Moving and resizing windows or in general the geometry handling is one of the core aspects of a window manager. It’s where our expertise is, the code which makes KWin such a good window manager. Naturally we don’t want to throw that code out and want to reuse it in a Wayland world.

And that meant: refactoring. The big problem here was that the code in question is also highly X11 specific. Examples are: the sync protocol for synced resizing, window gravity handling, various X11 specific quirks for moving X11 window moving smooth (e.g. only update at end of move if compositing is active, direct updates if not active).

The task was now to separate the X11 specific geometry handling from generic geometry handling so that it can support both Wayland and X11 windows. With this now in place all the geometry handling works also for Wayland windows. This includes move/resize through the alt+f3 menu, but also triggered from the window itself. During move/resize the quick tiling areas are of course triggered correctly, show the outline and snap to the area if released. Resizing is automatically synced to the speed the client supports and that also in a much better way than on X11 (yay for Wayland and double-buffered state!). During move/resize the windows snap to screen borders and other windows.

Unfortunately our checks to ensure that the window titlebar is not moved outside the visible area doesn’t work, because of client side decorations. Yay, everything is awesome! Have I mentioned that client side decorations are a stupid idea because it breaks useful stuff? No, cannot be. Related to that: QtWayland enters move mode if you click the title bar to activate the window. Of course there should be a delay which KWin implements for it’s decorations. After all we have more than 15 years of experience in doing window decorations. Yay, client side decos! Given control to the application! Let’s leave them in charge of window management, what could go wrong? Wohoo! Totally awesome! Broken window management! That’s the way to go! Of course that could be fixed in Qt, but well it doesn’t fix the problem. We have seen this in Chromium years ago, we even had to adjust our quick tiling/maximization behavior because of it. Another fun fact: QtWayland deco has a minimize button which does nothing because it’s not supported in wl_shell protocol. Have I mentioned that client side decos are awesome?

All right, all right, I’ll end the sarcasm mode now. Our geometry handling has a few more very handy features like window packing: you can configure a shortcut to move the window left (or other direction) and it will move to the next window or the screen border. Similar we support growing or shrinking the window to the next window. I was hardly aware of that feature before doing the refactoring, so I thought I should point it out. It can be used from scripting, so should be useful for ideas like poor man tiling or resizing multiple quick tiled windows at the same time. As a nice addition this code is now covered by auto tests.

So with geometry handling in place it’s possible to do real testing and one of my systems (my notebook) migrated from X11 to Wayland. Actually I have been using Wayland on that system already for watching videos since April as it gives tear-free rendering. A clear plus when watching videos.

And this gives us already the outlook for what we will see in the November update: I’ll focus on stabilizing the current state of Plasma/Wayland and fix bugs, fix bugs and fix bugs. My aim is to have a useable early-adopters version ready for the Plasma 5.5 release. It’s looking good, so I’m confident that we will reach that state.

Upgrading libhybris

One of the most important dependencies for our phone project is libhybris. Libhybris is a neat technology to allow interfacing with Android drivers allowing for example to bring Wayland to a device where all we have are Android drivers.

Given that KWin provides a hwcomposer backend which uses libhybris to create an OpenGL context. All other applications need libhybris indirectly to have the Wayland OpenGL buffer exchange work automatically.

When we started the work on the hwcomposer backend we based it on the libhybris version used in Ubuntu (0.1.0+git20131207) as we used Ubuntu as the reference platform. Soon enough we noticed that this version diverged a lot from the upstream version. Lots of recent changes are missing and there are API incompatible changes.

This made working with it difficult. How much time should we invest in investigating issues? Should we write code which we know might break once Ubuntu decides to upgrade libhybris? How well is Wayland integrated in the Ubuntu version given that they don’t need it? If we need help, who to talk to? Ubuntu who will tell us that they don’t know anything about Wayland, or the libhybris devs who might just tell us: use later version?

Furthermore we want other distributions to provide Plasma for the phone. This means they need to provide libhybris. Of course this is difficult if we need to tell them that we need exactly the version used by Ubuntu. And even more it might conflict with other uses. Considering distributions like Mer would have to chose between a libhybris for Plasma and a libhybris for lipstick.

With that in mind we wanted to invest some time on upgrading libhybris in our stack in this release cycle and then fix the issues we were seeing in the stack. Our awesome packagers did the job of creating packages so that I can port KWin against it. And in deed after some hacking I had KWin rendering again. A more difficult task was to get other applications to work as we run into a problem that libwayland-egl does not use the alternatives system. Thus our packagers needed to do some ld tricks to get this worked around. But with that we had a nice rendering system.

A surprise in this exercise was that our input handling code in the hwcomposer backend didn’t compile any more. The code was gone. While that was in the first moment an unpleasant surprise, it soon turned into something wonderful. If that code is not needed at all on an Android powered device it means that we must be able to get libinput to work with it. 400 lines of code deleted and it’s using the shared input stack through libinput. I’m very happy about that!

With that all in place we finally were able to investigate the rendering issues we were seeing. My hope was that just upgrading libhybris would fix the visual tearing, but unfortunately not. While I’m still surprised that it’s possible to get tearing in the first place on Android devices (hey ever heard of things like Atomic Mode Settings, Android?), it at least gives us a vsync event. Unfortunately the only tear-free solution I could find invokes blocking till we get the event. I don’t like that and I think that’s a bad architecture. One can have blocking free and tear free rendering. Our DRM (kernel mode setting) backend can do so with an easy to use API. Really disappointing that the Android stack is in that regard not better than the glx backend. But well at least it’s tear free :-)

As we now use upstream libhybris I hope to see distributions to pick up the work and provide a Plasma phone spin. I’d love to see an openSUSE phone or a Fedora phone (or any other distribution). Distributions: you can of course ask us on how to integrate :-)

The return of kwin_gles

Back in 4.x we provided two binaries for KWin: one compiled against OpenGL (kwin) and one compiled against OpenGL ES (kwin_gles). The reason for that is that one can only reasonably link either OpenGL or OpenGL ES and OpenGL ES is only a subset of OpenGL, so one needs to hide the OpenGL calls (especially the OpenGL 1 calls).

With 5.x we were no longer able to provide these two binaries. The reason for that is that OpenGL got “upgraded” in Qt and QtGui itself links either OpenGL or OpenGL ES. To keep the system’s sanity we decided to follow how Qt is compiled. If Qt is compiled with OpenGL support KWin gets compiled with OpenGL support, otherwise with OpenGL ES support.

That’s of course a reasonable design, but it means that it becomes difficult to test the OpenGL ES code paths. One needs a dedicated Qt and all other dependencies compiled against OpenGL ES. Or one needs a nice device like a Nexus 5 with Plasma mobile. This had resulted in breakage already as we were not able to test enough. Such times belong to the past as I have a nice Plasma mobile device to compile and test my KWin on.

But since we introduced OpenGL ES support through a compile time switch, many things have changed. KWin dropped the OpenGL 1 support which means that most of the code which wouldn’t compile with OpenGL ES is just gone. Furthermore we switched to use libepoxy, so we don’t link OpenGL or OpenGL ES at all, but libepoxy which does the right thing for us. With that we are able to remove all the compile time checks. Of course we need runtime checks to ensure that we don’t call functionality which isn’t available on OpenGL ES.

Now with the upcoming 5.5 release we are able to have one binary which serves both OpenGL and OpenGL ES. Note to distributions: the artifact kwinglesutils.so is no longer compiled, please adjust your packaging rules. KWin will use either OpenGL or OpenGL ES depending on what Qt uses.

Given that nowadays it’s also possible to create both an OpenGL and OpenGL ES context at runtime we can also make use of that and introduced a new value for our KWIN_COMPOSE environment variable: O2ES. If that’s specified KWin will use the EGL backend and create an OpenGL ES context. Although it in general is also possible to create an OpenGL ES context through glx we do not support that for simplification. As proof debug output (qdbus org.kde.KWin /KWin supportInformation) from my running KWin instance:

Compositing is active
Compositing Type: OpenGL ES 2.0
OpenGL vendor string: Intel Open Source Technology Center
OpenGL renderer string: Mesa DRI Intel(R) Ivybridge Desktop 
OpenGL version string: OpenGL ES 3.0 Mesa 10.6.8
OpenGL platform interface: EGL
OpenGL shading language version string: OpenGL ES GLSL ES 3.00
Driver: Intel
GPU class: IvyBridge
OpenGL version: 3.0
GLSL version: 3.0
Mesa version: 10.6.8
X server version: 1.17.3
Linux kernel version: 4.2
Direct rendering: Requires strict binding: no
GLSL shaders:  yes
Texture NPOT support:  yes
Virtual Machine:  no
OpenGL 2 Shaders are used
Painting blocks for vertical retrace:  yes

So in a way we have kwin_gles back, it’s different as it’s no longer a dedicated binary, but it’s runtime switchable. For the moment the only way will be the environment variable. I’m reluctant to add a config option as that sounds like quite some chance for breakage.

September update for Plasma’s Wayland porting

September was a busy months in the KDE Wayland world. We have worked hard to bring Plasma closer to a workable system and could cross off some very important milestones.

Transient window positioning

One of the biggest oddities when trying out Plasma on Wayland in the 5.4 release is the fact that menus open at random positions. The reason for this is that KWin applied it’s placing strategy on it and ignored the hints provided by the window. We have now implemented support for the so-called transient windows in KWayland and use the provided placement hint in KWayland. So now all menus open at the correct position. A useable Wayland session is much closer now.

Plasma/KWin specific extensions

Marco did quite some work for the integration of Plasma. KWin provides some Plasma specific extensions like the sliding popups effect, blur and background contrast effect, etc. We have an abstraction through KWindowSystem so that applications do not have to use low level X11 calls directly. Now we extended this abstraction to also Wayland: if the application uses the API it will work on both X11 and Wayland. Granted it will only work with compositors providing the specific protocols, but that is no difference to X11. Also there the compositor needs to implement the custom protocol.

On the client side the protocol is implemented in KWayland and the integration for KWindowSystem is provided through
a plugin provided by the kwayland-integration repository. On KWin side the protocol is also provided in KWayland allowing a very easy to use API. Of course also KWin needed small adjustments in the effects to announce support for the protocol and read the information provided by the windows. Thanks to nice wrapping in KWayland the code is cleaner and simpler than in the X11 case.

Support for multiple X Servers

A change not directly relevant for KWin went into KWindowSystem and will be released with the upcoming 5.15 KDE Frameworks release. KWindowSystem provides an X11 API which looks like it supports multiple X Servers (e.g. one application talking to multiple servers), but that has never worked as it fetched required atoms only on first connection. We refactored the relevant code to no longer have this limitation.

Current Plasma in a nestedKWin/Wayland session with grabbed input
Current Plasma in a nested KWin/Wayland session with grabbed input

Granted normally applications do not talk to multiple X servers, but there is a mode in KWin which uses just that: a nested kwin_wayland on X11. To explain: it needs to talk to the host X server for rendering two (one server) and it starts it’s own Xwayland server (second server). In case you have ever wondered why the nested kwin_wayland window released with Plasma 5.4 neither has a window icon nor a window title: that’s the reason. We couldn’t use Qt’s abstraction (wrong QPA plugin) and also not KWindowSystem as we needed to make sure the atoms get resolved for the Xwayland server. Now with this restriction removed the window has an icon and a title. Even more I added a “grab input mode” as one might know from virtual machines. While it’s easy to implement I didn’t want to implement it without having a way to tell the user what happened and what the current grab state is.

Preparing KWin for the cloud

The most exiting new feature in my opinion was born last Friday based on frustrations about testing KWin Wayland. Last week Marco and I spent quite some time investigating a few regressions (as it turned out due to adding transient window support). The way to test it was just uncomfortable. We had a test case but it meant starting KWin, waiting till it’s started, start the test application, perform some clicks and interpret whether it worked. Once we fixed that issue I started to look at a crash and the process was similar annoying. What I wanted was a way to automate the test condition, that is an autotest which we could even run in our CI system.

So on Friday I decided to dedicate my development time on a virtual framebuffer backend.This backend (to start use kwin_wayland --xwayland --virtual) doesn’t render to any device, but only “simulates” rendering by using a QImage which then isn’t used at all. Well not completely true: there is an environment variable to force the backend to store each rendered frame into a temporary directory.

Why is such a virtual backend so exiting? Well it means we can run KWin anywhere. We are not bound to any hardware restrictions like screen attached or screen resolution. With other words we can run it on servers – in the cloud. The first such instance runs on our CI servers in the form of an automated integration test. And in future there will be much more such tests.

But that is not only interesting for KWin to have it’s autotests, it’s also interesting for all other projects of the workspace as we have now a virtual Wayland server which is functional identical to the one we use. We also have a better virtual X server now as we have Xwayland instead of Xvfb (e.g. support for XRandR extension).

Once I had it implemented ideas came to me for improving our CI system: we could use it for something like OpenQA (or integrate the existing tool) and start a complete Plasma session and screenshot various points (if that sounds like an exiting project: please contact me, also if you want to do that as a Season of KDE project ).

Or integrate a remote rendering solution (e.g. VNC, rdesktop, spice, html5) and run a complete session through the web. That could be a very interesting feature for designers, translators, supporters and many other non-developers. Get the latest state of the code directly tested. We have things like Kubuntu CI which make it easy to test in a virtual machine, but wouldn’t it be even more awesome to just run the latest build of the CI system in the browser?

KWin Tests

With the help of this virtual backend we are now able to start a “complete” KWin in the autotests. This allows us to very precisely test whether a specif feature works as expected. Unit tests are great, but sometimes one wants to test the complete integration and that’s now possible.

Working auto completion tooltip in Kate on KWin/Wayland
Working auto completion tooltip in Kate on KWin/Wayland

The first problem addressed with this new possibility was a bug noticed while writing this blog post. I used Kate on Wayland and the tooltips got keyboard focus. So now we have an autotest which ensures this case works.

Comparing DPMS on X11 and Wayland

On the Plasma workspaces Display Power Management Signaling (DPMS) is handled by the power management daemon (powerdevil). After a configurable idle time it signals the X-Server through the X11 DPMS Extension. The X-Server handles the timeout, switching to a different DPMS level and restoring to enabled after an input event all by itself.

The X11 extension is a one-way channel. One can tell the X-Server to go to DPMS, but it neither notifies when it has done so nor when it restored. This means powerdevil doesn’t know when the screens are off. Granted, there is a call to query the current state, but we cannot perform active polling to figure out whether the state changed. That would be rather bad from a power management perspective.

So overall it’s kind of a black box. Our power management doesn’t know when the screens turn off, Neither does any other application including KWin. For KWin it would be rather important to know about it, because it’s the compositor and there is no point in running the compositor to render for something which will never be visible. The same is obviously true for any other application: it will happily continue to render updated states although it won’t be visible. For a power management feature this is rather disappointing.

Now let’s switch to Wayland – in this case kwin_wayland. Last week I started to implement support for DPMS in KWin. Given that KWin is responsible for DRM, we need to add support for it in KWin. Of course we want to leverage the existing code in powerdevil, so we don’t add the actual power management functionality, but rather an easy to use interface to tell KWin to enter a DPMS state on a given output.

Of course we tried to not repeat the mistake from X11 and thus the interface is bi-directional: it notifies when a given state is entered. Although in a Wayland world that’s not so important any more: KWin enables DPMS and knows that DPMS is enabled and can make internal use of this. So whenever the outputs are put into standby the compositor loop is stopped and doesn’t try to repaint any more. If the compositor is stopped windows are not being repainted and aren’t signaled that the last frame got painted. So all applications properly supporting frame callbacks stop rendering automatically, too. This means on Wayland we are able to properly put the rendering of all applications into an off state when the connected outputs are in standby mode. A huge improvement for power management.

Thoughts on Vulkan in KWin

Lately I have been asked a lot about using Vulkan in KWin: in fact almost every blog post in the last few months has questions about it and that seems to me there is something to write about it.

So the quick tldr: I don’t have any plans on adding support for Vulkan. Over the last years I ported to so many new technologies, including at least 3 incompatible OpenGL versions. I am not looking forward to another incompatible OpenGL version (whether it’s called Vulkan or OpenGL doesn’t matter to me on that).

Now let’s look at the strength of Vulkan: going closer to the hardware and doing multi-threaded rendering. KWin still performs the rendering in the main GUI thread. Qt tried to do rendering in an off-thread for QtQuick’s scene-graph, in case of KWin we also do that in the main-gui thread. Reworking our compositor to use threading is a lot of work and would also probably improve the performance with OpenGL. Anyway as long as KWin doesn’t support threaded rendering this improvement by Vulkan is rather moot.

Now let’s look on the closer to hardware: yesterday I did a blog post on how I want to use e.g. KMS Planes to get closer to hardware and bypass the OpenGL compositing. Vulkan will allow us to perform rendering closer to hardware, but if we don’t render at all, we don’t benefit from it. Vulkan is probably more power saving, but not using the rendering will save even more.

So Vulkan can only improve when the scene needs to be rendered: e.g. when you wobble a window or spin the desktop cube. I don’t think that those effects are what we should optimize for and spend our time on and even if: there are low-hanging fruits to optimize using OpenGL, we do not yet use OpenGL 3 or 4 features to improve these effects.

Overall Vulkan looks to me like a lot of work as once again we would have to add a new compositing backend, write a new low level interaction, rewrite all shaders, etc. etc. Going to Vulkan early would mean introducing more complexity to KWin, more different code paths our users might use. It sounds like a mood exercise to do so. But I also doubt that there will be useable Vulkan drivers any time soon.

Vulkan is a great technology which will make graphics much better. But I doubt that an application like KWin is the use-case it was developed for.

Now to something else!

KDE is currently running a fundraiser for the Randa Meetings with the topic “Bring Touch to KDE!”. I personally will not participate in the Randa Meetings, but many other KDE developers will go there and we need your support for it. Please help and make the Randa Meetings possible.


Layered compositing

On X11 the (OpenGL) compositor renders into a single buffer through the overlay window. This is needed to get features like translucency, shadows, wobbled windows or a desktop cube as Xorg itself doesn’t have any support for such features. The disadvantage of this approach is that we basically always have to perform a “copy” of what needs to be rendered. Consider VLC is playing a fullscreen video the compositor needs to take VLC’s video pixmap and render it onto the overlay window. The compositor needs to run, evaluate the scene and then render the one window.

Of course the compositors optimize for this case by only repainting what is needed and repainting from top to bottom in the window stack so that only the one fully opaque window gets rendered. Even more they have features like unredirecting fullscreen windows or (in the case of KWin) blocking compositing while specific windows are active. Unredirecting is a nice idea but it doesn’t really cover the use case. If one has just a small window (a tool tip, menu, on-screen volume change display) on top of the VLC video window, we need to start compositing again. Although we still would like to just forward the one VLC window.

On Wayland the situation looks better. Instead of rendering to an overlay window the compositor can directly interact with the hardware through e.g. DRM. This allows to for example take the buffer provided by VLC and pass it directly to DRM and bypass the compositing all together. But modern hardware allows us to do more than Xorg ever allowed us through “layers”. Be it the raspberry pi, Android’s hwcomposer or Linux’s KMS Planes, the idea is always similar: let the hardware compose the final image together.

If we think of Plasma phone we realize that in most cases we just need three layers: top panel, bottom panel and a maximized window in between. There is no need to do complex composition of the scene at all as all that is needed is putting the three visible (and opaque) windows in the three layers.

On a desktop system it’s a little bit more complicated. Our windows are not all maximized, they have translucent window decorations, shadows, there is a panel with blurred background. Especially the latter one needs a composition through OpenGL. Nevertheless there should be situations where we can make use of layers. Especially in the case of a fullscreen or maximized window this should help making the compositor faster.

Using layers will mean a large change on how KWin’s compositor works. The infrastructure for unredirecting of fullscreen windows cannot be reused in a sensible way as it is evaluated only for fullscreen windows and before the compositor runs. That is before the effects are executed. The decision whether to put a window into a layer needs to come at a later point: once we have evaluated whether we need to compose the complete screen (you might be wobbling a window) or can short pass through layers.

What I have in mind is to give this architecture a try with the VC4 hardware of the raspberry pi which supports layers in a decent way. And to first adjust the QPainter based compositor (as it’s the simpler one) to make use of this architecture. The idea is to experiment with it to get a useful generic KWin-internal backend API, so that we can also use it for OpenGL compositor and even more for our DRM and hwcomposer backend.

For the DRM backend I want to use the new atomic modesetting feature which will be introduced in Linux 4.2. Given that we cannot depend on it anytime soon, this will still need some time till it will be implemented. Overall all of this is not going to happen tomorrow – there are still more important things to work on. But if you, dear reader, are interested on working on this, please contact me. I’m happy to walk you through the code and share my ideas.

As this blog post is mostly about optimizing the compositor by making use of new hardware features which we didn’t have on X11, I can already point out that I will do a follow up blog post on Vulkan.

Now to something else!

KDE is currently running a fundraiser for the Randa Meetings with the topic “Bring Touch to KDE!”. I personally will not participate in the Randa Meetings, but many other KDE developers will go there and we need your support for it. Please help and make the Randa Meetings possible.


A Qt Platform Abstraction plugin for KWin

In Qt we have the Platform Abstraction (QPA) which allows to better interact with the used windowing system through a plugin. In case of KWin we use the “xcb” plugin on X11 and on Wayland we used to use the “wayland” plugin provided by QtWayland. For quite some time I had been thinking about migrating away from those and use an own KWin-specific plugin at least for Wayland.

What’s wrong with QtWayland?

QtWayland provides a plugin for Wayland clients, but KWin is not a Wayland client: it is the Wayland server. This created some interesting “problems” which we had to workaround in KWin. KWin just doesn’t fit the usecase of QtWayland QPA plugin and it would not be a good idea to change QtWayland in a way that it supports KWin as a first-class citizen – it’s just a too special use case. So let’s look at some of the issues which motivated me to write an own QPA plugin.

Connecting to the Wayland server

The wayland plugin is meant for Wayland clients and tries to connect to the Wayland server during the creation of the QGuiApplication. As KWin is the Wayland server the plugin wants to connect to, we needed to make sure that the Wayland server and the event loop are up and running before constructing the QGuiApplication. This added a restriction that we have to monitor the server socket from the main thread.

Blocking roundtrips

The wayland plugin performs a blocking roundtrip to the Wayland server during startup to wait for all outputs to be announced. But this happens from the main gui thread, in which also the server lives. You can guess what happens: dead-lock. We fixed this one in QtWayland, but the problem was still there: any blocking call from the main gui thread to the Wayland server would freeze KWin. We experienced this with e.g. eglInitialize, startup of dbus services (e.g. kded, kamd), rendering of Qt internal windows in case the compositor didn’t send the frame rendered yet. Overall I found this aspect way to fragile and consider especially the fact that the server is in the main thread as a problem.


KWin uses the Qt::BypassWindowManagerHint on all it’s windows because this is needed on X11. The hint is X11 specific and doesn’t make much sense on other platforms, so the Wayland plugin doesn’t show such windows at all. That is a valid approach – there is nothing I could say against it. So in KWin we needed to remove the hint for all windows on Wayland.

OpenGL Context for QtQuick

QtQuick requires an OpenGL context and if it cannot create one it will abort. That is normally not a problem but can easily become one in the case of KWin: KWin creates a “low-level” OpenGL context using the specific platform (e.g. DRM/GBM, hwcomposer, whatever). Qt(Wayland) doesn’t know which one KWin uses and might not even have code for it. QtWayland tries to create an OpenGL context using the EGL platform Wayland. In the case of Mesa everything works fine, but we noticed that on libhybris the creation of a Wayland context fails if there is already one created for hwcomposer. One could consider this as a libhybris specific issue, but I wouldn’t say so – interacting with multiple EGL platforms from one process is kind of out of the specification.

Window decorations

This is a small issue: QtWayland creates window decorations around all Qt windows. But in the context of KWin we do not want any decorations. So we had to disable them by setting the QT_WAYLAND_DISABLE_WINDOWDECORATION environment variable.

And here is the plugin

So after Akademy I decided to work on it and see whether we can get a plugin working. Now it’s in a state where it works good enough to blog about it. The plugin does not support everything that might be used by Qt, but rather reduces to the feature set used by KWin. For example we don’t need to forward any input events, because KWin (mostly) handles them internally anyway and sends them directly to the respective internal Qt window.

The plugin brings the advantage that it no longer depends on the Wayland server being started before creating the QApplication. In fact it does not even try to connect to it, but can use KWin’s internal connection. Similar we can use this to shortcut e.g. creation of low level windows, etc.

On the OpenGL side the plugin approach shows it’s true strength. As it can access KWin internals we can use the EGLDisplay used by the compositing scene and create a sharing OpenGL context. So we can bypass the windowing system completely to render QtQuick windows and this will also allow us to make our window textures available to QtQuick. So a nice improvement.

In case our compositor doesn’t use OpenGL the plugin creates an OpenGL context for the platform Wayland. This is done in a way that it doesn’t hit the blocking issue I mentioned above.

So is the plugin something you should use in other applications?

The plugin interacts with KWin internals and thus is not in any way useable for non-KWin applications. In fact it even checks whether the running binary is KWin and does nothing if that’s not the case. It will also never be turned into a generic plugin as there is no need for that (just use QtWayland).

When will this plugin get merged?

The plugin can be found in branch “qpa” on my personal kwin clone on git.kde.org. It’s not yet in a state that I could push it to master. I still need to test more and make sure that everything works as intended.

Now to something else!

KDE is currently running a fundraiser for the Randa Meetings with the topic “Bring Touch to KDE!”. I personally will not participate in the Randa Meetings, but many other KDE developers will go there and we need your support for it. Please help and make the Randa Meetings possible.


Should we target EGL as the default?

When we started the compositing work in KWin the only way to initialize an OpenGL context was by using GLX. In fact GLX is even part of the OpenGL library on Linux. Being an X11 window manager and an X11 compositor it was not a big problem.

Years later we started the effort to get KWin using optionally OpenGL ES in addition to normal OpenGL. One of the differences is that OpenGL ES doesn’t use GLX, but rather EGL. So we gained code to setup compositing using EGL. But this was still a compile time switch and only supported together with using OpenGL ES. If I remember correctly the option to create on OpenGL context on top of EGL was not yet available in our drivers back then.

But there was a time when the FLOSS drivers started to support it and since the 4.10 release in beginning of 2013 we can provide a normal OpenGL context over EGL. This option was not really exposed and only bound to an environment variable. Very few users knew about it and this was also wanted. Using EGL was a good way to shoot yourself in the foot given that not all drivers support it and that we had much more testing on the GLX backend.

Now the situation has changed. With 5.x we expose EGL also through a config interface, but more importantly EGL is the only way to get OpenGL on Wayland. The assumption that every user is using GLX doesn’t hold any more. The assumption that we don’t need to spend so much time (as it’s only relevant in the reduced feature set of mobile devices) on the EGL backend is wrong. It must reach the same quality as the GLX backend, otherwise our Wayland experience suffers.

So this raises an important question: should we try targeting EGL as the default? In the past one of the main reasons to not even think about EGL as default is the reason that the NVIDIA driver didn’t support it. If we would have switched to EGL by default it would have meant dealing with the situation that EGL doesn’t work and we need to fall back to GLX. It would have complicated the startup significantly and would have resulted in our user base using different rendering paths.

Now this one road blocker seems to resolve itself. While I haven’t tested it yet, the latest NVIDIA beta driver supports OpenGL over EGL, so we should be able to have the majority of our user base on the same backend after all. Given that it is only a beta driver and the feature is marked as “experimental” it’s probably still some time till we can default to it.

Nevertheless it’s time to start thinking about it. I encourage NVIDIA users to give EGL now a try and to report issues. Also I encourage all other users to report issues they see with the EGL backend. If we get the EGL backend into a good state maybe 5.6 might be a good point in time to default to it and then start to slowly deprecate the GLX backend with the aim to get it removed in the distant future.

Plasma Phone and KWin

As you are probably aware by now we announced the Plasma Phone project during Akademy this weekend. In this blog post I want to discuss the role of KWin in Plasma Phone.

Plasma Phone uses Wayland as the windowing system with KWin being the Wayland compositor. This is our first product which uses Wayland by default and also the first product which uses KWin as the Wayland compositor. The phone project pushed the Wayland efforts in Plasma a lot and is the only reason why we are able to make Wayland a technological preview with the upcoming Plasma 5.4 release.

The phone project gave KWin/Wayland into the hands of many developers who started to actively use it and to develop with it. This obviously helped to find lots of small and larger issues which then could be fixed. It triggered a stabilization process which reached a stage that I can use a Plasma Wayland session on my notebook with enough confidence that I won’t lose data due to crashes. So thanks to the whole team for pushing the system to the limits.

An area which saw lots of work thanks to the Phone is the interaction between Plasma as the desktop shell and KWin as the Wayland compositor. With Wayland we keep the architecture of having a dedicated shell process which is not inside the compositor. This architecture has served us well in the past and we don’t see a reason why we should change this. It means that KWin can serve as a compositor for other desktop projects, but it also means that Plasma can be used with other compositors. Now unlike X11, Wayland’s protocols are secure by default. This means that Plasma as the desktop shell does not know anything about windows from other processes. To solve this problem we designed a special window management protocol which exports the required information. The protocols are still under development, but we hope that they can be also useful for other projects with a similar architecture (LXQt being an obvious candidate). Of course such protocols should only be used by the “blessed” desktop shell process – this is something we still need to implement and one of the reasons why at the moment Plasma/Wayland is only a technological preview.

Window management is not the only area where the shell process needs to be “blessed”. Also for announcing information about its own windows, we need some more information. We need to know whether a window is a “panel” or the “desktop” view. So on the phone the shell background is just a desktop window, the panel on the bottom is just a normal dock. This allows us to share the general layering code with the X11 implementation of KWin. Even more having the panels marked as panels allows us to properly define the maximized area. And the windows on the phone open maximized by using the “Maximizing” placement strategy in KWin.

Working on the phone project also had some surprises. For example the virtual keyboard just didn’t want to show. It turned out that the reason for this was that Qt(Wayland) only requests the virtual keyboard if it has keyboard focus. Keyboard focus inside KWin is bound to the “active” window. So we had to implement activating Wayland clients. But the initial implementation didn’t really solve it, we still had areas where the keyboard didn’t come up. E.g. on the desktop in the KRunner search field we couldn’t get it to show. The reason was related to the described problem: we did not yet support activating Wayland clients when clicking on them. This got solved by implementing mouse action (and touch action) support inside KWin. So thanks to this change (done for the phone) we can properly switch between Wayland windows on the desktop with a mouse driven setup.

Another nice touch is that KWin requires a running Xwayland server. This gives us support for all “legacy” X11 applications such as Qt 4 or GTK2 on the phone. Just we hit a problem with them: they did not react on touch events. The reason for this is quite simple: touch support is not yet implemented in Xwayland. As a solution we implemented simulating pointer events in case a connected Wayland window does not bind the touch interface. Thus all Wayland and X11 applications can now be used with a touch screen – be it on the phone or on a notebook with a touch screen.

So far I have only spoken about progress made in KWin which is relevant for the desktop. So what about the phone specific adjustments? Well there are hardly any. In the core of KWin there is no phone specific code. The only phone specific code is the hwcomposer backend. This is a plugin used for creating an OpenGL context using libhybris and for reading input events (unfortunately libinput cannot read events on a libhybris enabled system). According to cloc this plugin is just about 800 lines of code and about 200 are just very straight forward mapping of Android key codes to “normal” Linux key codes. For comparison: KWin + kwayland/server currently have about 120,000 lines of code. And even this hwcomposer backend is not phone specific. It could also be used to run a normal KWin/Plasma session on any libhybris enabled device. There is one important difference of the plugin to the rest of KWin which is worth to mention: it is GPLv3+ licensed, while everything else is GPLv2+. The reason for this change is the fact that libhybris is Apache 2 licensed and this license requires a change to GPLv3.