Wayland improvements since Plasma 5.8 release

Two weeks have passed since the Plasma 5.8 release and our Wayland efforts have seen quite some improvements. Some changes went into Plasma 5.8 as bug fixes, some changes are only available in master for the next release. With this blog post I want to highlight what we have improved since Plasma 5.8.

Resize only borders

KWin’s server side decorations have a feature that one can resize the window in the shadow area. With the Breeze window decoration this is available if one uses the border size “No Side Borders” or “No Borders”. For Wayland we just had to adjust the input area of a window slightly and honor it when evaluating the mouse pointer movements.

Global Shortcut handling

We found a few bugs related to global shortcut triggering. There is some unexpected behavior for shortcut triggering in xkbcommon, which will be addressed in the next release by adding new API. For now we had to workaround it to support some shortcuts which no longer triggered. Of course for every kind of shortcut which did not trigger we added a test case so we can also in future ensure that this works once the new xkbcommon release is available. At the moment we are not aware of any not working global shortcuts on Wayland. If you hit one, please report a bug.

Support for Keyboard LEDs through libinput

KWin did not enable the LEDs for num lock, caps lock, etc. This was mostly because I don’t have any keyboard which has such LEDs – neither my desktop keyboard nor my two notebooks have any LEDs. So I just didn’t notice that this was missing. Once we got the bug report we looked into adding this. I want to take this as an example of the “obvious bug” one doesn’t report because it’s so obvious. But if one doesn’t have such hardware it’s not so obvious any more.

Relative pointer support

A feature we added for Plasma 5.9 is support for the relative pointer protocol.

Relative Pointer Events

The protocol is implemented in KWayland 5.28 and KWin is adjusted to support the relative pointer events as can be seen in the screenshot of the input debug console. This is a rather important protocol to support games on Wayland. We also plan to add pointer confinement for Plasma 5.9.

Move windows through the widget style

Our widget styles Breeze and Oxygen have a feature to move the window when clicking in empty areas. This is a feature which needs to interact with the windowing system directly as Qt doesn’t provide an abstraction for it. On X11 it uses the NETRootInfo::moveResizeRequest, on Wayland support for triggering a window move is built into the core protocol. But so far we were not able to provide the feature on Wayland as we just didn’t have enough information from QtWayland. For example we lacked access to the wl_shell_surface on which we have to trigger the move. So some time ago I added support to QtWayland that we can access the wl_shell_surface through the native interface. Now about a year later we can start to use it. To support this feature we need to create an own wl_seat and wl_pointer object and track the serial of pointer button press. This we can then pass to the move request on the ShellSurface. The change is not KWin specific at all and will work on all Wayland compositors.

Color scheme sync to decoration

A new feature we added in KWin 5.0 is the possibility to synchronize the color scheme from the window into the window decoration and the context menu on the decoration. On X11 this works through a property which our KStyle library sets. This was the best we had back in the early days of the 5.x series as Qt didn’t expose enough information. It has the disadvantage that the sync only works with QWidget based applications and only with widget styles inheriting KStyle. For Plasma 5.9 we improved that and brought the relevant code into plasma-integration. The restriction to QWidget is gone and it works now with all kind of windows by listening to the QPlatformSurfaceEvent. This very useful event which got added in Qt 5.5. It informs us when a native window is created for a QWindow. Thus we can add our own X11 properties on the native window directly after creation and before the window is mapped.

Custom color scheme support

While adjusting this code for X11 we also added the relevant bits for Wayland. We use the Qt Surface Extension protocol to pass a property to the server. That’s a small and neat addition the Qt devs did to allow communication between a Qt based client and a Qt based Wayland compositor. As one can see in the screenshot the color scheme now updates also for Wayland applications.

Window icons

Window icon handling in Wayland is different to X11. On X11 the icons are passed as pixmaps. That has a few disadvantages nowadays because the icons provided on the window might not have a high enough resolution to work well on high-dpi systems. The icon from the icon-theme though provides higher resolution. On Wayland there is no way to pass window icons around and the compositor takes the icon from the desktop file of the application. This works well unless we don’t have a desktop file. For such windows we now started to use a generic Wayland icon as the fallback, just like we use a generic X icon as fallback for X11 windows which don’t have an icon.

Proper icons for X windows in task manager

That’s an icon which one might have noticed when using a Plasma Wayland session as every Xwayland window only had the generic X icon in the task manager. The communication between KWin and the task manager also passes the icon name around and not pixmap data. This works well for everything which isn’t Xwayland where we normally just don’t have the name. For Plasma 5.9 we addressed this problem and extended our protocol to request pixmap data for a window icon which doesn’t have a name. Thus we are now able to also support Xwayland windows, which increases the useability of the system quite a lot.

Multi screen effect improvements

On Wayland several of our effects broke in a multi-screen setup. This is because rendering is different. On X11 all screens are rendered together in one rendering pass and we have one OpenGL window to render to. On Wayland we have one OpenGL window per screen and have one rendering pass per screen. That’s something our effects didn’t handle well and resulted in rendering issues. For Plasma 5.9 these issues are finally resolved.

Wobbly windows

One of the affected effects is Wobbly windows. A rather important effect given that this blog is subtitled “From the land of wobbly windows”. We experienced that in a multi-screen setup the effect was only active on one screen. If the window got moved to the other screen it completely vanished.

I was quite certain that this is not a problem with the effect itself, but rather with the way how we render. As we also saw other effects having rendering issues in multi-screen setups I was quite optimistic that fixing wobbly would fix many effects.

The investigation showed that the problem in fact was an incorrect area passed to glScissor due to the general changes in rendering explained above. Rendering on other screens got clipped away. With the proper change we got wobbly working and several other effects (Present Windows, Desktop Grid, Alt+Tab for example) without having to touch the effects at all.


With that knowledge in place we looked into fixing other effects. E.g. the screenshot effect which allows to save a screenshot in the tmp directory. A few example of screenshots taken with this effect can be seen in this blog post. The problem with this effect was that when taking a fullscreen shot over all screens only one got captured. The assumption here was that our glBlitFramebuffer code needs adjustment to be per output and with that we can now screenshot every screen individually or all screens combined.

Multi-screen shot with blur

Blur and Background Contrast

Related to that are the blur and background contrast effect as they also interact with the frame buffer, though don’t use the glBlitFramebuffer extension. With those effects one of the biggest problems was that the viewport got restored to a wrong value after unbinding the frame buffer object. Due to that the rendering got screwed up and we had severe rendering issues with blur on multi screen. These issues are now fixed as can be seen in the screenshot above: both screens are rendered correctly even with blur enable.

Panel improvements

Plasma’s panel got some improvements for Plasma 5.9. This started from bug reports about windows can cover not working and also auto-hide not working. Another example that it is important to report bugs.

Auto hiding panel

On X11 auto hiding panels use a custom protocol with KWin to indicate that they want to be restored if the mouse cursor touches the screen edge. It uses low level X11 code thus we also need a low level Wayland protocol for it. We extended our plasma shell protocol to expose auto hiding state and implemented it in both KWin and Plasma.

Search in widget explorer

We had a bug report that search in the widget explorer doesn’t work. The investigation showed that the reason for that is that the widget explorer is a panel window and we designed panels on Wayland so that they don’t take any keyboard focus. This is correct for the normal panel, but not for this special panel. We adjusted our protocol to provide an additional hint that the panel takes focus and implemented this in kwayland-integration in a way that the widget explorer gains focus without any adjustments to it.

KRunner as a panel

Of course there are more potential users for this new feature. One being KRunner. Once we had the code in place we decided to make KRunner a Panel on Wayland which brings us quite some improvements like it will be above other windows and on all desktops.

Animating auto-hiding panels

With Plasma 5 a change regarding auto-hiding panels was introduced. The complete interaction was moved from Plasma to KWin. This was an idea which we had in mind for a long time. The main idea is to have only one process to reserve the interaction with the screen edges (which is needed to show the panel when hidden) and to prevent conflicts there. Also it allowed to have only one place for providing the hint that there is the panel.

On an implementation level that uses an x11-property protocol between KWin and Plasma to indicate when the panel should be hidden. KWin then does the interaction to hide it and to show on screen edge activation.

Unfortunately from a visual perspective this created a regression. In Plasma 4 the auto-hiding panel was animated with our Sliding-Popups effect, but in Plasma 5 this doesn’t work any more. The reason for that is that our effect system can only animate when a window gets mapped and unmapped. With the new protocol the window doesn’t get unmapped when hidden and not mapped when shown, so our effects are not able to react on it. Technically our Client object is kept instead of being released on unmapped.

Thanks to Wayland this will be working again starting with Plasma 5.8. For Wayland windows KWin keeps the object around when a window gets unmapped and uses the same object when it gets shown again. KWin keeps more state for Wayland windows than for X11 windows. But this also means that our animations are not working for Wayland windows. Last week I addressed this and extended the internal effects API to also support hiding/showing again of Wayland windows. As the effects API does not differentiate between Wayland and X11 windows this change also enables the auto-hiding panel animation. All that was needed was emitting two signals at the right place.

Here Wayland shows another strength: not only does it help to bring features to X11 it also allows us to automate the testing. With a pure X11 system we would have had a hard time to properly auto-test this. But for Wayland we have a nice isolated integration test-framework which allowed to add a test-case for auto-hiding panels.

Wayland in Plasma 5.7

Last week we released the beta version of Plasma 5.7 which means we know what this release will have for better Wayland support. First of all I need to mention what didn’t make it: unfortunately I missed the freeze of Frameworks 5.23 to land support for xdg-shell. I have a working implementation, but was not yet satisfied with the API. This is a difficult interface to provide an API for due to the unstable nature of the interface. Due to lack of xdg-shell support GTK applications are still going to use X11 on Wayland (like the Firefox window I’m just typing this blog post in).

In the past I already blogged about a few new features in 5.7 like the improved task manager for Wayland, the virtual keyboard support, sub-surface support and improved input device support. So in this blog post I want to focus on a different topic: quality.

For Plasma 5.7 my aim was to get the Plasma session into a state that I can use it as my primary system. And since last week I have not started into an X11 session any more. This means that we needed to get the whole system stable enough to have neither KWin nor applications crash due to Wayland. Given that our Wayland code is quite a fair amount of new code, changing lots of assumptions there are of course bugs to be expected. We still have code which calls into X11 unconditionally, we have things which are not implemented correctly and of course we do stupid mistakes. So for Plasma 5.7 the task was to find these issues and fix one for one.

For Wayland it’s much easier for us to test. KWayland – our framework for Wayland support – is developed in a test driven approach making it possible to create test cases for every problem. They expose the problems and verify that they are fixed and as regression tests ensure that they won’t hit us again. Over the last release cycle we added several thousand lines of test code in KWayland alone.

Finding those issues is not always easy. If KWin crashes we don’t have DrKonqi like normally, it doesn’t work for Wayland (tries to connect to a display server, but that just crashed). What I saw on my Wayland test device was that KWin sometimes randomly crashed – more often when I interacted with X11 windows. But when attaching gdb to KWin it didn’t crash. But once I caught it: it turned out to be an error in KWin in the handling of Xwayland windows. There are two possible code paths it can take and one was with a mistake. Due to running through a debugger it was more likely to take the correct one. So yeah it’s not always easy.

With that problem gone we were able to find a few more and fix also some bugs which caused windows to quit. Unfortunately some of this fixes had to go into KWayland after the 5.23 release. This means the frameworks version used with Plasma 5.7 is not going to have all fixes. If you want to give Plasma/Wayland a try I recommend to not just wait for Plasma 5.7 but also for frameworks 5.24.

This week I will be at the openSUSE conference in Nuremberg, where I will also give a talk about how Wayland helps us to improve our quality and our workflows. I’ll do another blog post about the content of that presentation – don’t want to spoil 😉 Though if you follow our development you are already aware of it. Thanks to openSUSE to give me the possibility to present at the conference and thanks to KDE e.V. for the support to go there.

I can hear you asking now the question of all questions: “When will it be ready?” I think that I am not objective enough to answer the question or to say that it is ready. I’m too close to the code and might just omit important problems because I don’t see them. Thus I cannot say that it is ready. It depends on your workflow and whether that workflow is already fully implemented. This is something only you can know.

Last week KDE had a very important developer sprint (where I did not participate) and is currently running a fundraising campaign for this sprint. We need the money to send our developers to such meetings or to a conference like openSUSE conf where I will be this week. At the moment just 107 people have participated and donated. This is something which makes me sad. I see the statistics for my blog posts and know that this one will have at least 1000 direct hits. In addition there are people reading planetkde and not directly my blog. We are trying to raise 24000 EUR and please do the math yourself to see how close we would be to it if everybody would donate just 10 EUR.

A task manager for the Plasma Wayland session

Plasma 5.7 will ship with a new taskmanager library. One of the reasons to implement a new taskmanager library is the port to Wayland. Our old code base was heavily relying on X11 and didn’t support the concept of multiple windowing systems. You can read more on that in Eike’s blog post about the new task manager. In this blog post I want to focus a little bit on the Wayland side of a task manager.

Architectural difference to X11

On X11 every application is able to read every aspect of every other window. This allows to implement a task manager. The task manager can get notified when a window is added, when a window is removed and install event filters for the changes of the window properties. That’s the basics for a task manager.

On Wayland the world is different. An application is not able to see the windows of other applications. This means by default Plasma is not able to see what windows exists, the taskmanager would be empty. The only application which knows about the existing windows is the Wayland compositor.

Merging Desktop Shell and Wayland compositor?

A possible solution to this problem could be to merge the desktop shell and the compositor into one process. This is the architecture used in e.g. Unity and GNOME Shell on X11.

But such an architecture has also disadvantages. It violates the concept of separating concerns and of “doing one thing and do it right”. We have an excellent task manager, an excellent window manager and an excellent desktop shell.

Also it would make code sharing between platforms more difficult. A task manager for Wayland would be very different from a task manager on X11. All the business logic and presentation should be the same. No matter which windowing system is used.

By merging the code bases together we would also make the code less reusable. One couldn’t use KWin/Wayland in another desktop environment (e.g. LXQt) nor could one use Plasma/Wayland with another compositor (e.g. sway). While we of course recommend to use KWin as the Wayland (and X11) compositor we don’t enforce it and also don’t want to enforce it.

Protocols, protocols

So a different solution is needed and that is done through a custom Wayland protocol which is called org_kde_plasma_windowmanagement. We have a few interfaces prefixed with Plasma. This does not mean that it can only be used by Plasma. It just means that the interface was first designed and developed for Plasma’s needs. We are happy to share the interface with other desktop environments.

The protocol is provided by KWin and it announces whenever a new window gets created. Once a new window gets created a bound client can bind an org_kde_plasma_window interface and get notified about all the states of the window.

The protocol exposes pretty much all the information KWin has about the window and it gets updated automatically whenever the state in KWin changes. In addition the protocol has requests the task manager needs, like “close the window”, “minimize it”, “maximize it”, etc. etc. KWin also listens to these requests and honors them.

Although the protocol is added to Wayland, it is windowing system agnostic. The created Plasma Window does not expose the actual windowing system resource (after all on Wayland another client cannot get access to it). KWin exposes a Plasma Window for both X11 managed windows and Wayland windows. By that the task manager is able to manage tasks from multiple windowing systems without knowing that it does so.

Evolution of the protocol

We added the Plasma window protocol initially for the work on the Plasma phone about a year ago. Also on the Plasma phone we had the use case of switching windows and using the X11 based task manager was just no option on a Wayland only device 😉

While the protocol worked fine for our initial needs, it was not yet sufficient for the use in the desktop task manager. So over the last weeks we introduced further support and improved the API to make it work more reliable. Also quite some changes went into KWin to ensure that only “interesting” windows are exposed. E.g. a context menu should not be shown in the task manager.

Our KWayland library of course has support for this protocol through the classes PlasmaWindowManagement and PlasmaWindow on client side. In addition there is a model exposed as PlasmaWindowModel.

Future work

The work on the interface is not done yet. There are still changes to the API needed and the upcoming KDE Frameworks 5.23 release will include new API calls in PlasmaWindow. As well we have already the first change scheduled for KDE Frameworks 5.24. And we also know that we need to expose a few more data elements in the protocol to get the task manager to the same level as on X11.

There is also an interesting idea floating around to use the – windowing system agnostic – Wayland implementation on both X11 and Wayland. For this KWin (on X11) would need to create a dummy Wayland server for the task manager to connect to. It’s an idea which we might or might not implement.

Security considerations

Currently every Wayland client is able to bind this interface, which means that some of the security improvements of Wayland are not available in Plasma/Wayland. We are fully aware of that and were also fully aware of the consequence when we added the interface. I do have ideas on how to address this and this will be implemented before we will recommend the Plasma/Wayland session for daily usage. The design allows to add security checks in these areas. Unfortunately my priority list did not allow me to implement this for Plasma 5.7, next chance 5.8.

Looking at the security of Plasma/Wayland

Our beta release of Plasma 5.5 includes a Wayland session file which allows to start in a Plasma on Wayland session directly from a login manager supporting Wayland session. This is a good point in time to look at the security of our Plasma on Wayland session.

Situation on X11

X11 is not secure and has severe conceptual issues like

  • any client connected to the X server (either remote or local) can read all input events
  • any client can get information about when another window rendered and get the content of the window
  • any client can change any X attribute of any other window
  • any window can position itself
  • many more issues

This can be used to create very interesting attacks. It’s one of the reasons why I for example think it’s a very bad idea to start the file manager as root on the same X server. I’m quite certain that if I wanted to I could exploit this relatively easily just through what X provides.

The insecurity of X11 also influenced the security design of applications running on X11. It’s pointless to think about preventing potential attacks if you could get the same by just using core X11 functionality. For example KWin’s scripting functionality allows to interact with the X11 windows. In general one could say that’s dangerous as it allows untrusted code to change aspects of the managed windows, but it’s nothing you could not get with plain X11.

Improvements on Wayland

Wayland removes the threats from the X11 world. The protocols are designed in a secure way. A client cannot in any way interact with windows from other clients. This implies:

  • reading of key events for other windows is not possible
  • getting window content of other windows is not possible

In addition the protocols do not allow to e.g. position windows themselves, or raise themselves, change stacking order and so on.

Security removed in Plasma

But lots of these security restrictions do not work for us in Plasma. Our desktop shell need to be able to get information of other windows (e.g. Tasks applet), be able to mark a panel as a panel (above other windows) and need to be able to position windows itself.

Given that we removed some of the security aspects again and introduced a few custom protocols to provide window management facilities and also to allow Plasma windows to position themselves. At the moment we have no security restrictions on that yet which gives this functionality to all clients.

We will address this in a future release. There are multiple ways how this could be addressed, e.g. using the Wayland security module library or use systemd in some way to restrict access. Overall I think it will require rethinking security on a Linux user session in general, more on that later on.

Security added in Plasma compared to X11

The most important change on the security front of the desktop session is a secure screen locker. With Plasma 5.5 on Wayland we are able to provide that and address some long standing issues from X11. The screen locks even if a context menu is open or anything else grabbing input. The compositor knows that the screen is locked and knows which window is the screen locker. This is a huge change compared to X11: the XServer has no concept of a screen locker. Our compositor can now do the right thing when the screen is locked:

  • don’t render other windows
  • ensure input events are only handled in the lock screen
  • prevent access to screen grabbing functionality while screen is locked

As a matter of fact the Wayland protocol itself doesn’t know anything about screen locking either. This is now something we added directly to KWin and doesn’t need any additional custom Wayland interfaces.

How to break the security?

Now imagine you want to write a key logger in a Plasma/Wayland world. How would you do it? I asked myself this question recently, thought about it, found a possible solution and had a key logger in less than 10 minutes: ouch.

Of course there is no way to get a client to act as a key logger. The Wayland protocol is designed in a secure way and also our protocol additions do not weaken that. So the key to get a key logger is to attack KWin.

So what can an attacker do with KWin if he owns it? Well pretty much anything. KWin internally has a very straight forward trust model: everything is trusted and everything can access anything. There is not much to do about that, this is kind of how binaries work.

For example as a Qt application each loaded plugin has access to the QCoreApplication::instance. From there one could just use Qt’s meta object inspection to e.g. get to the InputRedirection model and connect to the internal signal on each key press:

<code>void ExamplePlugin::installKeyLogger()
    const auto children = QCoreApplication::instance()-&gt;children();
    for (auto it = children.begin(); it != children.end(); ++it) {
        const QMetaObject *meta = (*it)-&gt;metaObject();
        if (qstrcmp(meta-&gt;className(), "KWin::InputRedirection") != 0) {
        connect(*it, SIGNAL(keyStateChanged(quint32,InputRedirection::KeyboardKeyState)), this, SLOT(keyPressed(quint32)), Qt::DirectConnection);

void ExamplePlugin::keyPressed(quint32 key)
    qDebug() &lt;&lt; "!!!! Key: " &lt;&lt; key;

But Martin, why don’t you just remove the signal, why should any other aspect of KWin see the key events? Because this is just the example of the most trivial exploit. Of course it’s not the only one. If you have enough time and money you could write more sophisticated ones. For example look at this scenario:

KWin uses logind to open restricted files like the input event files or the DRM node. For this KWin registers as the session controller in logind. Now a binary plugin could just send a DBus call to logind to also open the input event files and read all events. Or open the DRM node and take over rendering from KWin. There is nothing logind could do about it: how should it be able to distinguish a valid from an invalid request coming from KWin?

How to secure again?

As we can see the threat is in loading plugins. So all we need to do is ensure that KWin doesn’t load any plugins from not trusted locations (that is not from any user owned locations). This is easy enough for QML plugins where we have the complete control. In fact it’s easy to ensure for any of KWin’s own plugins. We can restrict the location of all of them.

And even more: by default a system is setup in a way that no binary plugins are loaded from user’s home. So yeah, no problem after all? Well, unfortunately not. During session startup various scripts are sourced which can override the environment variables to influence the loading of plugins. And this allows to also use the well known LD_PRELOAD hack. My naive approach to circumvent this issue didn’t work out at all as I had to learn that already the session startup and the PAM interaction source scripts. So your session might be owned very early.

An approach to black list (unset) env variables is futile. There are too many libraries KWin relies on which in turn load plugins through custom env variables. Most obvious examples are Qt and Mesa. But there are probably many more. If we forget to unset one variable the protection is broken.

A different approach would be to white list some known secure env variables to be passed to KWin. But this also requires that at the point where we want to do the restriction the session is not already completely broken. This in turn means that neither PAM nor the session manager may load any variables into the session before starting the session startup. And that’s unfortunately outside what we can do in our session startup.

So for Plasma 5.5 I think there is nothing we can do to get this secure, which is fine given that the Wayland session is still under development. For Plasma 5.6 we need to rethink the approach completely and that might involve changing the startup overall. We need to have a secure and controlled startup process. Only once KWin is started we can think about sourcing env variables from user locations.

So how big is the threat? By default it’s of course secure. Only if there is already a malicious program running in the system there is a chance of installing a key logger in this way. If one is able to exploit e.g. a browser in a way that it can store an env variable script in one of the locations, you are owned. Or if someone is able to get physical access to your unencrypted hard disk, there is a threat. There are easy workarounds for a user: make all locations from where scripts are sourced during session startup non-writable and non-executable, best change ownership to root and encrypt your home location.

Overall it means that Plasma 5.5 on Wayland is not yet able to provide the security I would have liked to have, but it’s still a huge improvement over X11. And I’m quite certain that we will be able to solve this.

Plasma Phone and KWin

As you are probably aware by now we announced the Plasma Phone project during Akademy this weekend. In this blog post I want to discuss the role of KWin in Plasma Phone.

Plasma Phone uses Wayland as the windowing system with KWin being the Wayland compositor. This is our first product which uses Wayland by default and also the first product which uses KWin as the Wayland compositor. The phone project pushed the Wayland efforts in Plasma a lot and is the only reason why we are able to make Wayland a technological preview with the upcoming Plasma 5.4 release.

The phone project gave KWin/Wayland into the hands of many developers who started to actively use it and to develop with it. This obviously helped to find lots of small and larger issues which then could be fixed. It triggered a stabilization process which reached a stage that I can use a Plasma Wayland session on my notebook with enough confidence that I won’t lose data due to crashes. So thanks to the whole team for pushing the system to the limits.

An area which saw lots of work thanks to the Phone is the interaction between Plasma as the desktop shell and KWin as the Wayland compositor. With Wayland we keep the architecture of having a dedicated shell process which is not inside the compositor. This architecture has served us well in the past and we don’t see a reason why we should change this. It means that KWin can serve as a compositor for other desktop projects, but it also means that Plasma can be used with other compositors. Now unlike X11, Wayland’s protocols are secure by default. This means that Plasma as the desktop shell does not know anything about windows from other processes. To solve this problem we designed a special window management protocol which exports the required information. The protocols are still under development, but we hope that they can be also useful for other projects with a similar architecture (LXQt being an obvious candidate). Of course such protocols should only be used by the “blessed” desktop shell process – this is something we still need to implement and one of the reasons why at the moment Plasma/Wayland is only a technological preview.

Window management is not the only area where the shell process needs to be “blessed”. Also for announcing information about its own windows, we need some more information. We need to know whether a window is a “panel” or the “desktop” view. So on the phone the shell background is just a desktop window, the panel on the bottom is just a normal dock. This allows us to share the general layering code with the X11 implementation of KWin. Even more having the panels marked as panels allows us to properly define the maximized area. And the windows on the phone open maximized by using the “Maximizing” placement strategy in KWin.

Working on the phone project also had some surprises. For example the virtual keyboard just didn’t want to show. It turned out that the reason for this was that Qt(Wayland) only requests the virtual keyboard if it has keyboard focus. Keyboard focus inside KWin is bound to the “active” window. So we had to implement activating Wayland clients. But the initial implementation didn’t really solve it, we still had areas where the keyboard didn’t come up. E.g. on the desktop in the KRunner search field we couldn’t get it to show. The reason was related to the described problem: we did not yet support activating Wayland clients when clicking on them. This got solved by implementing mouse action (and touch action) support inside KWin. So thanks to this change (done for the phone) we can properly switch between Wayland windows on the desktop with a mouse driven setup.

Another nice touch is that KWin requires a running Xwayland server. This gives us support for all “legacy” X11 applications such as Qt 4 or GTK2 on the phone. Just we hit a problem with them: they did not react on touch events. The reason for this is quite simple: touch support is not yet implemented in Xwayland. As a solution we implemented simulating pointer events in case a connected Wayland window does not bind the touch interface. Thus all Wayland and X11 applications can now be used with a touch screen – be it on the phone or on a notebook with a touch screen.

So far I have only spoken about progress made in KWin which is relevant for the desktop. So what about the phone specific adjustments? Well there are hardly any. In the core of KWin there is no phone specific code. The only phone specific code is the hwcomposer backend. This is a plugin used for creating an OpenGL context using libhybris and for reading input events (unfortunately libinput cannot read events on a libhybris enabled system). According to cloc this plugin is just about 800 lines of code and about 200 are just very straight forward mapping of Android key codes to “normal” Linux key codes. For comparison: KWin + kwayland/server currently have about 120,000 lines of code. And even this hwcomposer backend is not phone specific. It could also be used to run a normal KWin/Plasma session on any libhybris enabled device. There is one important difference of the plugin to the rest of KWin which is worth to mention: it is GPLv3+ licensed, while everything else is GPLv2+. The reason for this change is the fact that libhybris is Apache 2 licensed and this license requires a change to GPLv3.

Global shortcut handling in a Plasma Wayland session

KDE Frameworks contain a framework called KGlobalAccel. This framework allows applications to register key bindings (e.g. Alt+Tab) for actions. When the key binding is triggered the action gets invoked. Internally this framework uses a DBus interface to communicate with a daemon (kglobalaccel5) to register the key bindings and for getting notified when the action triggered.

On X11 the daemon uses the X11 core functionality to get notified whenever key events it is interested in happen. Basically it is a global key logger. Such an architecture has the disadvantage that any process could have this infrastructure and it would be possible for multiple processes grabbing the same global shortcut. In such a case undefined behavior is triggered as either multiple actions are triggered at the same time or only one action is triggered while the others do not get informed at all.

In addition the X11 protocol and the X server do not know that kglobalaccel5 is a shortcut daemon. It doesn’t know that for example the shortcut to lock the screen must be forwarded even if there is an open context menu which grabbed the keyboard.

In Wayland the security around input handling got fixed. A global key logger is no longer possible. So our kglobalaccel5 just doesn’t get any input events (sad, sad kglobalaccel5 cannot do anything) and even when started on Xwayland with the xcb plugin it’s pretty much broken. Only if key events are sent to another Xwayland client it will be able to intercept the events.

This means a global shortcut handling needs support from the compositor. Now it doesn’t make much sense to keep the architecture with a separate daemon process as that would introduce a possible security vulnerability: it would mean that there is a way how to log the keys. One only needs to become the global shortcuts daemon and there you go. Also we don’t want to introduce a round trip to another application to decide where to deliver the key event to.

Therefore the only logical place is to integrate global shortcut handling directly into KWin. Now this is a little bit tricky. First of all kglobalaccel5 gets DBus activated when the first application tries to access the DBus interface. And of course KWin itself is using the DBus interface. So KWin starts up and has launched the useless kglobalaccel5. Which means one of our tasks is to prevent kglobalaccel5 from starting.

Of course we do not want to duplicate all the work which was done in kglobalaccel. We want to make use of as much work as possible. Because of that kglobalaccel5 got a little surgery and the platform specific parts got split out into loadable runtime plugins depending on the QGuiApplication::platformName(). This allows KWin to provide a plugin to perform the “platform specific” parts. But the plugin would still be loaded as part of kglobalaccel5 and not as part of KWin. So another change was to turn the functionality of kglobalaccel into a library and make the binary just a small wrapper around the library. This allows KWin to link the library and start kglobalaccel from within the KWin process and feed in its own plugin.

Starting the linked KGlobalAccel is one of the first things KWin needs to do during startup. It’s essential that KWin takes over the DBus interface before any process tries to access it (as a good part it’s done so early that the Wayland sockets do not accept connections yet and Xwayland is not even started). We will also try to make kglobalaccel5 a little bit more robust about it to not launch at all in a Plasma/Wayland session.

Now the reader might think: wait, that still gives me the possibility to install a stealth key logger, I just need to create shortcuts for all keys. Nope, doesn’t work. As key events get filtered out a user would pretty quickly notice that something is broken.

Integrating KGlobalAccel into KWin on Wayland brings an obvious disadvantage: it’s linked to KWin. If one wants to use applications using KGlobalAccel on other compositors some additional work might be needed to use their local global shortcut system – if there is some. For most applications this is no problem, though, as they are part of the Plasma workspace. Also for other global shortcut systems to work with KWin it’s needed to port them to use KGlobalAccel internally when running in a Plasma/Wayland session (that’s also a good idea for X11 sessions as KGlobalAccel can provide additional features like checking whether the key is already taken by another process).

Four years later

At beginning of June 2011 I made my first blog post about KWin support Wayland clients featuring a screenshot of Desktop Grid effect with a Wayland window shown on each desktop.


Now it’s almost four years later and I show once again such a screenshot:

Desktop Grid effect in a Plasma on Wayland session
Desktop Grid effect in a Plasma on Wayland session

A few things have changed, for example the screenshot shows a KWin running on DRM. And there’s one huge difference: the KWin instance is using Wayland internally and no longer X11 (it still uses it for X11 applications, or course). Also KWin is able to properly integrate the Wayland windows. They are not rendered a top of the scene but take up normal spots like all other windows, too. Otherwise they are also more like “normal” windows. The plasmashell in this screenshot is also a Wayland client including all the panels and windows it creates (not visible as desktop grid effect hides panels).

A good question to ask is how much of the initial code written in 2011 ended in todays KWin and the honest answer is none. The branch got never merged and pretty quickly bitrotted. I got the rendering done pretty quickly, but at that time Wayland hadn’t had a stable release yet, so we simply couldn’t merge the code into master as that would have been rather inconvenient for development. With two moving targets (Wayland and KWin) the code diverged too quickly, broke too often and made development difficult. Thus when I heard a stable release of Wayland was planned it didn’t make much sense to continue the work on the branch.

But there was of course one thing which the branch provided: experience. The branch showed that our compositor is up to the task of integrating non-X11 windows, but the rest of KWin wasn’t. Which triggered a refactoring to make KWin more useable with multiple platforms. There were side-effects from that development which went already in some releases: e.g. the reworked scripting helped to identify the interface of a managed Client. Very recently this got split out into a new abstract base class which is now inherited by the good old X11 Client and the new Wayland ShellClient. A different area is screen edge handling which got reworked to not only allow multiple backends but also to have more fine grained per screen edges.

The work to get Wayland integrated into KWin took much longer than expected, part of it was the problem described above of the missing stable Wayland releases. Of course there was also a Qt5 port which I didn’t expect. The Qt5 port required a port to xcb – again something I didn’t expect. But most of all I underestimated the problem scope of how much work would be needed to get the window manager ported to Wayland. Still it will be quite some time till all features which are provided by KWin will be available for Wayland. Too much code is too X11 specific to be directly reusable.

And to get a complete session up and running there is much more to be ported than just KWin. We have Plasma which needs to access window management information provided by KWin (a taskbar should be able to show tasks after all). Some parts need to be moved into KWin for better security: screen locker and global shortcut handling (almost finished) are obvious candidates. We have functionality provided in our libraries which need adjustments: examples are KWindowSystem and KIdleTime. Our power management code need to learn how to talk with KWin to turn the screen off. KScreen needs to learn to interact with KWin to configure screens. And much, much more.

Also many of our applications need to be adjusted to work properly on Wayland. Obviously if one uses low-level X11 calls those won’t work any more. But also when you use higher level abstractions it might be that your feature doesn’t work any more, there are so many things which are X11 specific and you might not even know off. I plan to host a session on “Applications on Wayland” on this years Akademy. Make sure to go there. I’ll show all the “don’t” (and most are don’t on X11, too 😉 ) and show how one can easily setup a Wayland session to test the application.

Now I know that this blog post sounds a little bit like one shouldn’t expect Wayland on Plasma anytime soon. That’s not the case. I’m quite optimistic that I will shift my systems to Wayland as the primary work system before Akademy to properly dog food. And I’m sure that many Plasma developers will soon follow. After all I’m just editing this blog post in a full Plasma on Wayland session (though my Firefox is using XWayland).

And of course there are many, many tasks to work on. Most are really easy and easy to get into. Every contribution matters and helps us to embrace Wayland faster. Setting up a KWin development environment to test the Wayland progress is easy. All you need is to build kwayland and kwin through kdesrc-build (all other dependencies can be provided by your distribution). Once it’s installed just run:

kwin_wayland --windowed --xwayland

and a nice nested Wayland server will be started in your X session.

And if you want to get all down to running KWin directly on DRM like the screenshot above:

kwin_wayland --drm --libinput --xwayland

In all cases you can specify applications to be launched once KWin has fully started. Those can be either X11 or Wayland applications, e.g.:

kwin_wayland --drm --libinput --xwayland "konsole --platform wayland"

Will start a konsole as a Wayland client once can has started.

A recent addition as of today is that you don’t have to specify the commands like windowed or drm any more. KWin will automatically pick the correct backend depending on the environment variables which are exported. But be careful: if you start a kwin_wayland on an X session without DISPLAY exported, it will start the drm backend and that might break your X session!

The relationship between Plasma and KWin in Workspaces 2

Yesterday during the Tokamak 6 sprint in Nuremberg we discussed the role of KWin in Plasma Workspaces 2. At the moment in Plasma Workspaces 1 KWin is of course the recommended window manager and compositor, but it’s also possible to use a different window manager. Back in the days there were quite a lot of users who run Plasma with Compiz. In theory that shouldn’t matter because everything is standardized with EWMH and ICCCM. Over the years we added more and more extensions to EWMH. It’s all open source so anyone can implement these extensions (Compiz used to do so), nevertheless right now there is probably no other window manager available to offer the full experience except KWin.

Plasma Workspaces 2 will be released at an interesting point of time. We don’t want to do the transition to Qt 5 and Wayland at the same time, so it will still be X based. But we all agree that our future will be on Wayland and even if we use X as the windowing system our primary focus is on Wayland. With Wayland quite a few things will change. KWin will play a more important role as it will be the Wayland compositor – we do not plan to use Weston.

Given that we know that the Wayland shell interface only covers part of what Plasma needs and some of our needs are extremely Plasma specific (for example Activities) it would be tempting to say that we tie KWin to Plasma. Let’s face it: which other compositor will be there to replace KWin? The reference compositor will probably never accept Plasma specific patches for things like Activities, Compiz won’t be ported to Wayland and GNOME Shell will probably never be a solution for Plasma. For the small window managers we do not know whether they will go to Wayland at all, but I expect rather not, though I expect that we will see Weston forks/extensions for substitutions of tiling window managers.

We decided to resist the temptation to go the easy road, but instead will develop all our integration bits in a way that one could replace KWin by a different Wayland compositor, even if that is just a theoretical option. Of course we will not do any fallback modes for the case that one is using e.g. Weston without Plasma integration bits. So the features which we need might then just be disabled. Adding fallback modes would most likely just result in bit rotting code as nobody would use it.

Of course to make it possible that others can provide compatibility features we need to properly document our extensions and additional interfaces. Luckily Wayland implicitly forces us to do so. The general plan is to publish our extensions and also try to standardize what makes sense to be standardized and we hope that this would also benefit other projects. What we especially had in mind is of course Razor-Qt which already supports using KWin. By properly documenting all our Plasma-KWin communication channel, they can also use what is useful to them and it ensures that we don’t break KWin in a way that it gets unusable for Razor-Qt.

Fallback mode in KDE Plasma Workspaces

Recently there has been a lot of buzz about non-composited fallback modes in various Desktop Shells and of course I have been asked several times about the fallback modes in KDE Plasma workspaces and whether they would be removed, too. Now instead of answering the same question again and again I decided to write a blog post to discuss the situation in more detail.

The first thing to notice is that KDE Plasma workspaces do not have a non-composited fallback mode in the way GNOME Shell or Unity used to have. The main difference is that our window manager (KWin) is able to act as a non-composited, XRender based compositor and OpenGL (ES) based compositor. This means that we do not have to maintain two window managers in order to provide non-composited setups.

The second major difference is that the Desktop Shell (either KDE Plasma Desktop, KDE Plasma Netbook or KDE Plasma Active) is not a plugin to the compositor but a separate application running in an own process. This allows to use a completely different window manager together with the Desktop Shell. Some years ago it was not that uncommon to use Compiz together with Plasma, though we see less and less such usage patterns (mostly caused by Compiz no longer being available in most major distributions except Ubuntu). This is a design decision which served us well and we do not plan to change it. Some time ago I brought up the topic, but more experienced developers (mainly Aaron) illustrated nicely the advantages of our design choices.

Of course we need some code to be able to adapt to non-composited mode inside the Plasma workspaces. But this code would be needed anyway. Why? To answer this question we have to go back in time to when the development of Plasma had started – that is around 2006/2007. Back then OpenGL based compositing had been a rather new topic (I think I first heard about Compiz in 2004 or 2005) and it had only been possible with the binary blobs and hacks such as Xgl. The Intel hardware back then was not powerful enough and drivers were lacking, too. With other words: going an OpenGL-based only path seemed not feasible at that time and consider that the fallback modes in Unity and GNOME Shell got only removed in late 2012/early 2013. This is more than half a decade later – an eternity in IT. Also KWin did not enable compositing support by default till 4.2, so the first releases of KDE Plasma were non-composited by default.

The solution inside Plasma was to make our themes aware of compositing. That is each theme contains an opaque element set which will be used when in non-composited mode. The fact whether compositing is used, is globally available through a standardized X11 manager selection. So whenever the compositing state is changed Plasma is notified and switches the used elements. Simple and elegant. This functionality served us quite well when we introduced the blur effect. Translucency is actually a very difficult topic. If you choose a too strong translucency level the text is no longer readable, if you choose a translucency fitted for text readability the translucency is hardly visible. The solution to this problem is the blur effect. It provides easily readable text even with strong translucency levels. But even today we do not enable the blur effect by default on all hardware as it is an expensive effect and we would not dare to enable it on embedded devices (personally I doubt that Windows 8 removed the blur because the designers disliked it). So we see we need themes with different levels of translucency depending on whether blur is available or not. The design decisions for the Plasma themes provided us the solution directly. As well it means that even if we would drop the non-composited mode we would have to keep our design to support the blur selection.

For most of the applications it hardly matters whether the desktop is composited or not. Only very few applications use the alpha channel in the first place and if they want to use it, everything’s nicely abstracted inside Qt.

Inside the window manager it also doesn’t really matter. Of course our window decorations are aware of whether compositing is used or not and especially Oxygen makes use of it by providing an adjusted look for the non-composited case, so there we have some overhead, but except that there are hardly any adjustments.

This also means that we would not gain anything from requiring compositing. The compositor is a separate module inside the window manager which just gets started by the window manager on start-up. It lives more or less for itself and is just notified when a new window is added to the window manager and so on. Nothing inside the window manager really depends on the compositor. That means it would be more work to get KWin requiring compositing than to keep supporting the non-composited mode. And having the non-composited mode around allows us to do things like turning compositing off when running games or heavy OpenGL based applications such as Blender. So if you want to get some of the now finally available games for Linux, KDE Plasma should be your primary choice to enjoy the game. I have also heard of users switching to KDE Plasma because we still provide non OpenGL based setups.

What remains is the question of what will change with Qt 5? Obviously not much. Plasma will continue to provide a non-composited and a composited theme, but Plasma itself will require OpenGL due to being based on QtQuick 2. I do hope that llvmpipe will be sufficient for Plasma in case of non available drivers. The same is true for KWin: our QtQuick based elements will require OpenGL 2, but that does not mean that we will require OpenGL based compositing. We will still provide non-composited mode and if the XRender based compositor get’s ported to XCB (we have lots of work there to get involved 😉 will continue to provide XRender based compositing as well as OpenGL 1 and OpenGL 2 based. Since the refactoring of the OpenGL based compositor I do not see much difficulties with keeping the OpenGL 1 based compositor around, though if it turns out to not be used anymore, it might be removed.

The last remaining question is what will change with Wayland? Obviously Wayland requires to have a compositor, but it does not require us to stop supporting X11. KWin will continue to be an X11 based window manager allowing you to not use Wayland at all. And even if you choose to use a Wayland based KWin nothing says that it needs to be a compositor on top of OpenGL. Whether we will provide non-OpenGL based compositors for Wayland is a question for the future, though.

A final remark: this blog post describes the situation inside the KDE Plasma workspaces and the design decisions we have done. This is not a critic at the design decisions other projects have done and I am not able to judge whether their decision is right or wrong and it is completely irrelevant what my opinion is on their decisions.