Posted on

Global shortcut handling in a Plasma Wayland session

KDE Frameworks contain a framework called KGlobalAccel. This framework allows applications to register key bindings (e.g. Alt+Tab) for actions. When the key binding is triggered the action gets invoked. Internally this framework uses a DBus interface to communicate with a daemon (kglobalaccel5) to register the key bindings and for getting notified when the action triggered.

On X11 the daemon uses the X11 core functionality to get notified whenever key events it is interested in happen. Basically it is a global key logger. Such an architecture has the disadvantage that any process could have this infrastructure and it would be possible for multiple processes grabbing the same global shortcut. In such a case undefined behavior is triggered as either multiple actions are triggered at the same time or only one action is triggered while the others do not get informed at all.

In addition the X11 protocol and the X server do not know that kglobalaccel5 is a shortcut daemon. It doesn’t know that for example the shortcut to lock the screen must be forwarded even if there is an open context menu which grabbed the keyboard.

In Wayland the security around input handling got fixed. A global key logger is no longer possible. So our kglobalaccel5 just doesn’t get any input events (sad, sad kglobalaccel5 cannot do anything) and even when started on Xwayland with the xcb plugin it’s pretty much broken. Only if key events are sent to another Xwayland client it will be able to intercept the events.

This means a global shortcut handling needs support from the compositor. Now it doesn’t make much sense to keep the architecture with a separate daemon process as that would introduce a possible security vulnerability: it would mean that there is a way how to log the keys. One only needs to become the global shortcuts daemon and there you go. Also we don’t want to introduce a round trip to another application to decide where to deliver the key event to.

Therefore the only logical place is to integrate global shortcut handling directly into KWin. Now this is a little bit tricky. First of all kglobalaccel5 gets DBus activated when the first application tries to access the DBus interface. And of course KWin itself is using the DBus interface. So KWin starts up and has launched the useless kglobalaccel5. Which means one of our tasks is to prevent kglobalaccel5 from starting.

Of course we do not want to duplicate all the work which was done in kglobalaccel. We want to make use of as much work as possible. Because of that kglobalaccel5 got a little surgery and the platform specific parts got split out into loadable runtime plugins depending on the QGuiApplication::platformName(). This allows KWin to provide a plugin to perform the “platform specific” parts. But the plugin would still be loaded as part of kglobalaccel5 and not as part of KWin. So another change was to turn the functionality of kglobalaccel into a library and make the binary just a small wrapper around the library. This allows KWin to link the library and start kglobalaccel from within the KWin process and feed in its own plugin.

Starting the linked KGlobalAccel is one of the first things KWin needs to do during startup. It’s essential that KWin takes over the DBus interface before any process tries to access it (as a good part it’s done so early that the Wayland sockets do not accept connections yet and Xwayland is not even started). We will also try to make kglobalaccel5 a little bit more robust about it to not launch at all in a Plasma/Wayland session.

Now the reader might think: wait, that still gives me the possibility to install a stealth key logger, I just need to create shortcuts for all keys. Nope, doesn’t work. As key events get filtered out a user would pretty quickly notice that something is broken.

Integrating KGlobalAccel into KWin on Wayland brings an obvious disadvantage: it’s linked to KWin. If one wants to use applications using KGlobalAccel on other compositors some additional work might be needed to use their local global shortcut system – if there is some. For most applications this is no problem, though, as they are part of the Plasma workspace. Also for other global shortcut systems to work with KWin it’s needed to port them to use KGlobalAccel internally when running in a Plasma/Wayland session (that’s also a good idea for X11 sessions as KGlobalAccel can provide additional features like checking whether the key is already taken by another process).

Posted on

Four years later

At beginning of June 2011 I made my first blog post about KWin support Wayland clients featuring a screenshot of Desktop Grid effect with a Wayland window shown on each desktop.

desktopgrid-with-wayland

Now it’s almost four years later and I show once again such a screenshot:

Desktop Grid effect in a Plasma on Wayland session
Desktop Grid effect in a Plasma on Wayland session

A few things have changed, for example the screenshot shows a KWin running on DRM. And there’s one huge difference: the KWin instance is using Wayland internally and no longer X11 (it still uses it for X11 applications, or course). Also KWin is able to properly integrate the Wayland windows. They are not rendered a top of the scene but take up normal spots like all other windows, too. Otherwise they are also more like “normal” windows. The plasmashell in this screenshot is also a Wayland client including all the panels and windows it creates (not visible as desktop grid effect hides panels).

A good question to ask is how much of the initial code written in 2011 ended in todays KWin and the honest answer is none. The branch got never merged and pretty quickly bitrotted. I got the rendering done pretty quickly, but at that time Wayland hadn’t had a stable release yet, so we simply couldn’t merge the code into master as that would have been rather inconvenient for development. With two moving targets (Wayland and KWin) the code diverged too quickly, broke too often and made development difficult. Thus when I heard a stable release of Wayland was planned it didn’t make much sense to continue the work on the branch.

But there was of course one thing which the branch provided: experience. The branch showed that our compositor is up to the task of integrating non-X11 windows, but the rest of KWin wasn’t. Which triggered a refactoring to make KWin more useable with multiple platforms. There were side-effects from that development which went already in some releases: e.g. the reworked scripting helped to identify the interface of a managed Client. Very recently this got split out into a new abstract base class which is now inherited by the good old X11 Client and the new Wayland ShellClient. A different area is screen edge handling which got reworked to not only allow multiple backends but also to have more fine grained per screen edges.

The work to get Wayland integrated into KWin took much longer than expected, part of it was the problem described above of the missing stable Wayland releases. Of course there was also a Qt5 port which I didn’t expect. The Qt5 port required a port to xcb – again something I didn’t expect. But most of all I underestimated the problem scope of how much work would be needed to get the window manager ported to Wayland. Still it will be quite some time till all features which are provided by KWin will be available for Wayland. Too much code is too X11 specific to be directly reusable.

And to get a complete session up and running there is much more to be ported than just KWin. We have Plasma which needs to access window management information provided by KWin (a taskbar should be able to show tasks after all). Some parts need to be moved into KWin for better security: screen locker and global shortcut handling (almost finished) are obvious candidates. We have functionality provided in our libraries which need adjustments: examples are KWindowSystem and KIdleTime. Our power management code need to learn how to talk with KWin to turn the screen off. KScreen needs to learn to interact with KWin to configure screens. And much, much more.

Also many of our applications need to be adjusted to work properly on Wayland. Obviously if one uses low-level X11 calls those won’t work any more. But also when you use higher level abstractions it might be that your feature doesn’t work any more, there are so many things which are X11 specific and you might not even know off. I plan to host a session on “Applications on Wayland” on this years Akademy. Make sure to go there. I’ll show all the “don’t” (and most are don’t on X11, too 😉 ) and show how one can easily setup a Wayland session to test the application.

Now I know that this blog post sounds a little bit like one shouldn’t expect Wayland on Plasma anytime soon. That’s not the case. I’m quite optimistic that I will shift my systems to Wayland as the primary work system before Akademy to properly dog food. And I’m sure that many Plasma developers will soon follow. After all I’m just editing this blog post in a full Plasma on Wayland session (though my Firefox is using XWayland).

And of course there are many, many tasks to work on. Most are really easy and easy to get into. Every contribution matters and helps us to embrace Wayland faster. Setting up a KWin development environment to test the Wayland progress is easy. All you need is to build kwayland and kwin through kdesrc-build (all other dependencies can be provided by your distribution). Once it’s installed just run:

kwin_wayland --windowed --xwayland

and a nice nested Wayland server will be started in your X session.

And if you want to get all down to running KWin directly on DRM like the screenshot above:

kwin_wayland --drm --libinput --xwayland

In all cases you can specify applications to be launched once KWin has fully started. Those can be either X11 or Wayland applications, e.g.:

kwin_wayland --drm --libinput --xwayland "konsole --platform wayland"

Will start a konsole as a Wayland client once can has started.

A recent addition as of today is that you don’t have to specify the commands like windowed or drm any more. KWin will automatically pick the correct backend depending on the environment variables which are exported. But be careful: if you start a kwin_wayland on an X session without DISPLAY exported, it will start the drm backend and that might break your X session!

Posted on

The relationship between Plasma and KWin in Workspaces 2

Yesterday during the Tokamak 6 sprint in Nuremberg we discussed the role of KWin in Plasma Workspaces 2. At the moment in Plasma Workspaces 1 KWin is of course the recommended window manager and compositor, but it’s also possible to use a different window manager. Back in the days there were quite a lot of users who run Plasma with Compiz. In theory that shouldn’t matter because everything is standardized with EWMH and ICCCM. Over the years we added more and more extensions to EWMH. It’s all open source so anyone can implement these extensions (Compiz used to do so), nevertheless right now there is probably no other window manager available to offer the full experience except KWin.

Plasma Workspaces 2 will be released at an interesting point of time. We don’t want to do the transition to Qt 5 and Wayland at the same time, so it will still be X based. But we all agree that our future will be on Wayland and even if we use X as the windowing system our primary focus is on Wayland. With Wayland quite a few things will change. KWin will play a more important role as it will be the Wayland compositor – we do not plan to use Weston.

Given that we know that the Wayland shell interface only covers part of what Plasma needs and some of our needs are extremely Plasma specific (for example Activities) it would be tempting to say that we tie KWin to Plasma. Let’s face it: which other compositor will be there to replace KWin? The reference compositor will probably never accept Plasma specific patches for things like Activities, Compiz won’t be ported to Wayland and GNOME Shell will probably never be a solution for Plasma. For the small window managers we do not know whether they will go to Wayland at all, but I expect rather not, though I expect that we will see Weston forks/extensions for substitutions of tiling window managers.

We decided to resist the temptation to go the easy road, but instead will develop all our integration bits in a way that one could replace KWin by a different Wayland compositor, even if that is just a theoretical option. Of course we will not do any fallback modes for the case that one is using e.g. Weston without Plasma integration bits. So the features which we need might then just be disabled. Adding fallback modes would most likely just result in bit rotting code as nobody would use it.

Of course to make it possible that others can provide compatibility features we need to properly document our extensions and additional interfaces. Luckily Wayland implicitly forces us to do so. The general plan is to publish our extensions and also try to standardize what makes sense to be standardized and we hope that this would also benefit other projects. What we especially had in mind is of course Razor-Qt which already supports using KWin. By properly documenting all our Plasma-KWin communication channel, they can also use what is useful to them and it ensures that we don’t break KWin in a way that it gets unusable for Razor-Qt.

Posted on

Fallback mode in KDE Plasma Workspaces

Recently there has been a lot of buzz about non-composited fallback modes in various Desktop Shells and of course I have been asked several times about the fallback modes in KDE Plasma workspaces and whether they would be removed, too. Now instead of answering the same question again and again I decided to write a blog post to discuss the situation in more detail.

The first thing to notice is that KDE Plasma workspaces do not have a non-composited fallback mode in the way GNOME Shell or Unity used to have. The main difference is that our window manager (KWin) is able to act as a non-composited, XRender based compositor and OpenGL (ES) based compositor. This means that we do not have to maintain two window managers in order to provide non-composited setups.

The second major difference is that the Desktop Shell (either KDE Plasma Desktop, KDE Plasma Netbook or KDE Plasma Active) is not a plugin to the compositor but a separate application running in an own process. This allows to use a completely different window manager together with the Desktop Shell. Some years ago it was not that uncommon to use Compiz together with Plasma, though we see less and less such usage patterns (mostly caused by Compiz no longer being available in most major distributions except Ubuntu). This is a design decision which served us well and we do not plan to change it. Some time ago I brought up the topic, but more experienced developers (mainly Aaron) illustrated nicely the advantages of our design choices.

Of course we need some code to be able to adapt to non-composited mode inside the Plasma workspaces. But this code would be needed anyway. Why? To answer this question we have to go back in time to when the development of Plasma had started – that is around 2006/2007. Back then OpenGL based compositing had been a rather new topic (I think I first heard about Compiz in 2004 or 2005) and it had only been possible with the binary blobs and hacks such as Xgl. The Intel hardware back then was not powerful enough and drivers were lacking, too. With other words: going an OpenGL-based only path seemed not feasible at that time and consider that the fallback modes in Unity and GNOME Shell got only removed in late 2012/early 2013. This is more than half a decade later – an eternity in IT. Also KWin did not enable compositing support by default till 4.2, so the first releases of KDE Plasma were non-composited by default.

The solution inside Plasma was to make our themes aware of compositing. That is each theme contains an opaque element set which will be used when in non-composited mode. The fact whether compositing is used, is globally available through a standardized X11 manager selection. So whenever the compositing state is changed Plasma is notified and switches the used elements. Simple and elegant. This functionality served us quite well when we introduced the blur effect. Translucency is actually a very difficult topic. If you choose a too strong translucency level the text is no longer readable, if you choose a translucency fitted for text readability the translucency is hardly visible. The solution to this problem is the blur effect. It provides easily readable text even with strong translucency levels. But even today we do not enable the blur effect by default on all hardware as it is an expensive effect and we would not dare to enable it on embedded devices (personally I doubt that Windows 8 removed the blur because the designers disliked it). So we see we need themes with different levels of translucency depending on whether blur is available or not. The design decisions for the Plasma themes provided us the solution directly. As well it means that even if we would drop the non-composited mode we would have to keep our design to support the blur selection.

For most of the applications it hardly matters whether the desktop is composited or not. Only very few applications use the alpha channel in the first place and if they want to use it, everything’s nicely abstracted inside Qt.

Inside the window manager it also doesn’t really matter. Of course our window decorations are aware of whether compositing is used or not and especially Oxygen makes use of it by providing an adjusted look for the non-composited case, so there we have some overhead, but except that there are hardly any adjustments.

This also means that we would not gain anything from requiring compositing. The compositor is a separate module inside the window manager which just gets started by the window manager on start-up. It lives more or less for itself and is just notified when a new window is added to the window manager and so on. Nothing inside the window manager really depends on the compositor. That means it would be more work to get KWin requiring compositing than to keep supporting the non-composited mode. And having the non-composited mode around allows us to do things like turning compositing off when running games or heavy OpenGL based applications such as Blender. So if you want to get some of the now finally available games for Linux, KDE Plasma should be your primary choice to enjoy the game. I have also heard of users switching to KDE Plasma because we still provide non OpenGL based setups.

What remains is the question of what will change with Qt 5? Obviously not much. Plasma will continue to provide a non-composited and a composited theme, but Plasma itself will require OpenGL due to being based on QtQuick 2. I do hope that llvmpipe will be sufficient for Plasma in case of non available drivers. The same is true for KWin: our QtQuick based elements will require OpenGL 2, but that does not mean that we will require OpenGL based compositing. We will still provide non-composited mode and if the XRender based compositor get’s ported to XCB (we have lots of work there to get involved 😉 will continue to provide XRender based compositing as well as OpenGL 1 and OpenGL 2 based. Since the refactoring of the OpenGL based compositor I do not see much difficulties with keeping the OpenGL 1 based compositor around, though if it turns out to not be used anymore, it might be removed.

The last remaining question is what will change with Wayland? Obviously Wayland requires to have a compositor, but it does not require us to stop supporting X11. KWin will continue to be an X11 based window manager allowing you to not use Wayland at all. And even if you choose to use a Wayland based KWin nothing says that it needs to be a compositor on top of OpenGL. Whether we will provide non-OpenGL based compositors for Wayland is a question for the future, though.

A final remark: this blog post describes the situation inside the KDE Plasma workspaces and the design decisions we have done. This is not a critic at the design decisions other projects have done and I am not able to judge whether their decision is right or wrong and it is completely irrelevant what my opinion is on their decisions.

Posted on

Workspace Sprint

The last week I was allowed to spend in Pineda de Mar for the Workspace sprint. As others have already pointed out it was quite a different sprint with not much hacking going on and not so much technical discussions into the details. For this we will have a Tokamak sprint in Randa later this year.

Our sprint was mostly focused on community building and getting everyone on the same knowledge level. The weeks leading to the sprint showed that this was needed and I myself – although being involved in Plasma for quite some time – had to catch up on many things I was not aware of. The sprint helped us to identify for example weak spots in our internal documentation and getting this fixed is now one of the highest priorities.

The games Kevin played with us really helped us to identify where we want to go and what that means for our future development. This will also influence KWin’s development as we finally understand how Activities and Desktops go together and that helps us in KWin to properly focus development effort on it. Over the last cycles we did not really work on activities support in KWin as we were very unsure on how the window manager is involved in the process. This is now much clearer to me – I just need to document it and luckily Aurélien already started that process.

Luckily I was able to also get some hacking done. As Mr Gwenview was sitting next to me, I started to add some OpenGL magic to the image viewer. Sometimes I just need to work on a different piece of code and to get away from the daily work on KWin. The result is a much smoother zooming experience. At the moment the feature set is not yet identical to the existing implementation, but it’s on a very promising way. Very pleasing is also that at least on my system it works also well with llvmpipe, which could be a very nice solution for the case that a user does not have a decent OpenGL graphics card. For users who like to experiment, feel free to give a try to graesslin/opengl branch in the gwenview repository.

Overall it was a very nice sprint, but also very exhausting. I’m glad that I have one day to relax before going back to work. And I am really glad to see everyone again at this years Akademy where I will give a talk about KWin scripting.

Please thank the Frankfurt Airport for this blog post. Thanks to needing more than forty minutes to get my luggage from the plane to the belt I missed one train by two minutes (it went > 1h after landing) with the next train only after one hour instead of the normal half hourly service.