Lock screen security of phones

These days we can read a lot about a lock screen vulnerability in the Android system. Given that I have spent quite some thought on how we can use Plasma’s lock screen on our phone system I take the incident as an opportunity to share some thoughts about the topic. The tldr is “much ado about nothing”.

In Plasma we have an in my opinion rather secure infrastructure for the lock screen. Of course it suffers from the general problems of X11, but once it’s ported to Wayland it will be truly secure (till the first exploit is found). Given that I would like to use our lock screen architecture also on the phone. It’s secure by not letting anyone in even if the lock screen crashes (one of the problems hit in the Android exploit), by ensuring nothing else is rendered and no input is passed to any other application. So awesome! It will be secure!

But on second look we notice that the requirements on phone and desktop are different. On a phone we need to allow a few exceptions:

  • Accept phone calls even if screen is locked
  • Interact with notifications (e.g. alarm clock)
  • Allow emergency phone calls

The last item is also an important part of the puzzle for the Android exploit. These exceptions directly conflict with the requirements for our lock screen on the desktop. To quote:

Blocking input devices, so that an attacker cannot interact with the running session

It allows interacting with the running session (even more with the hardware) and it doesn’t block input devices any more.

I have over the last months spent quite some time thinking about how we can combine these requirements without compromising the security and so far I haven’t come to a sufficient solution. All I see is that if we allow applications (e.g. phone app) to bypass the lock screen, we in truth add a hole into the architecture and if there is a hole you can get through it. There will be ways to bypass the security then. No point in fooling ourselves. A phone app is not designed for the secure requirements of a lock screen.

Now phone calls are not all we need to care about on a lock screen – this could be solved by e.g. integrating the functionality into the greeter app. Users might want to take photos without having to unlock the screen (another piece of the Android exploit). It’s from a security perspective a questionable feature, but I can understand why it got added. Now this feature directly adds a huge hole into it: it writes to the file system. I can easily imagine ways to bypass the lock screen from a camera app, get to the file system, etc.

At this point we need to take a step back and think about what we want to achieve with a lock screen. On the desktop it’s clear: if there is a keyboard somewhere you should not be able to penetrate the session even if you have hours to try. But on a phone? Does this requirement hold? If I have the chance to unattended attack the lock screen of a phone, it means I own it. For desktop hardware we can say that the lock screen doesn’t protect against screw drivers. This also holds for phones. If one has enough time, it’s unlikely that one can keep the attacker out and the lock screen is most likely not the weakest link in the chain. Phones have things like finger print readers (easy to break), various easy to reverse construct passphrase systems, simple passwords, etc. So the lock screen itself is relatively easy to bypass and then we have not even looked at all the things one can do when attaching a usb cable…

Given that the requirements for phone security might be different? Maybe it’s not about blocking input devices, preventing anyone to get in? Maybe the aim is only to hold of people having access to it when unattended for a few moments?

If that’s the aim our lock screen architecture of the desktop might even be over done, adding holes to it would be wrong and we shouldn’t share the code? It also means that the Android vulnerability doesn’t matter. The exploit is a complicated process needing quite some time. The lock screen prevents access for uneducated people and also for those having just a few moments of unattended access. It only breaks in situation where it might not matter: when you already physically own it.

Comparing DPMS on X11 and Wayland

On the Plasma workspaces Display Power Management Signaling (DPMS) is handled by the power management daemon (powerdevil). After a configurable idle time it signals the X-Server through the X11 DPMS Extension. The X-Server handles the timeout, switching to a different DPMS level and restoring to enabled after an input event all by itself.

The X11 extension is a one-way channel. One can tell the X-Server to go to DPMS, but it neither notifies when it has done so nor when it restored. This means powerdevil doesn’t know when the screens are off. Granted, there is a call to query the current state, but we cannot perform active polling to figure out whether the state changed. That would be rather bad from a power management perspective.

So overall it’s kind of a black box. Our power management doesn’t know when the screens turn off, Neither does any other application including KWin. For KWin it would be rather important to know about it, because it’s the compositor and there is no point in running the compositor to render for something which will never be visible. The same is obviously true for any other application: it will happily continue to render updated states although it won’t be visible. For a power management feature this is rather disappointing.

Now let’s switch to Wayland – in this case kwin_wayland. Last week I started to implement support for DPMS in KWin. Given that KWin is responsible for DRM, we need to add support for it in KWin. Of course we want to leverage the existing code in powerdevil, so we don’t add the actual power management functionality, but rather an easy to use interface to tell KWin to enter a DPMS state on a given output.

Of course we tried to not repeat the mistake from X11 and thus the interface is bi-directional: it notifies when a given state is entered. Although in a Wayland world that’s not so important any more: KWin enables DPMS and knows that DPMS is enabled and can make internal use of this. So whenever the outputs are put into standby the compositor loop is stopped and doesn’t try to repaint any more. If the compositor is stopped windows are not being repainted and aren’t signaled that the last frame got painted. So all applications properly supporting frame callbacks stop rendering automatically, too. This means on Wayland we are able to properly put the rendering of all applications into an off state when the connected outputs are in standby mode. A huge improvement for power management.

Thoughts on Vulkan in KWin

Lately I have been asked a lot about using Vulkan in KWin: in fact almost every blog post in the last few months has questions about it and that seems to me there is something to write about it.

So the quick tldr: I don’t have any plans on adding support for Vulkan. Over the last years I ported to so many new technologies, including at least 3 incompatible OpenGL versions. I am not looking forward to another incompatible OpenGL version (whether it’s called Vulkan or OpenGL doesn’t matter to me on that).

Now let’s look at the strength of Vulkan: going closer to the hardware and doing multi-threaded rendering. KWin still performs the rendering in the main GUI thread. Qt tried to do rendering in an off-thread for QtQuick’s scene-graph, in case of KWin we also do that in the main-gui thread. Reworking our compositor to use threading is a lot of work and would also probably improve the performance with OpenGL. Anyway as long as KWin doesn’t support threaded rendering this improvement by Vulkan is rather moot.

Now let’s look on the closer to hardware: yesterday I did a blog post on how I want to use e.g. KMS Planes to get closer to hardware and bypass the OpenGL compositing. Vulkan will allow us to perform rendering closer to hardware, but if we don’t render at all, we don’t benefit from it. Vulkan is probably more power saving, but not using the rendering will save even more.

So Vulkan can only improve when the scene needs to be rendered: e.g. when you wobble a window or spin the desktop cube. I don’t think that those effects are what we should optimize for and spend our time on and even if: there are low-hanging fruits to optimize using OpenGL, we do not yet use OpenGL 3 or 4 features to improve these effects.

Overall Vulkan looks to me like a lot of work as once again we would have to add a new compositing backend, write a new low level interaction, rewrite all shaders, etc. etc. Going to Vulkan early would mean introducing more complexity to KWin, more different code paths our users might use. It sounds like a mood exercise to do so. But I also doubt that there will be useable Vulkan drivers any time soon.

Vulkan is a great technology which will make graphics much better. But I doubt that an application like KWin is the use-case it was developed for.

Now to something else!

KDE is currently running a fundraiser for the Randa Meetings with the topic “Bring Touch to KDE!”. I personally will not participate in the Randa Meetings, but many other KDE developers will go there and we need your support for it. Please help and make the Randa Meetings possible.


Layered compositing

On X11 the (OpenGL) compositor renders into a single buffer through the overlay window. This is needed to get features like translucency, shadows, wobbled windows or a desktop cube as Xorg itself doesn’t have any support for such features. The disadvantage of this approach is that we basically always have to perform a “copy” of what needs to be rendered. Consider VLC is playing a fullscreen video the compositor needs to take VLC’s video pixmap and render it onto the overlay window. The compositor needs to run, evaluate the scene and then render the one window.

Of course the compositors optimize for this case by only repainting what is needed and repainting from top to bottom in the window stack so that only the one fully opaque window gets rendered. Even more they have features like unredirecting fullscreen windows or (in the case of KWin) blocking compositing while specific windows are active. Unredirecting is a nice idea but it doesn’t really cover the use case. If one has just a small window (a tool tip, menu, on-screen volume change display) on top of the VLC video window, we need to start compositing again. Although we still would like to just forward the one VLC window.

On Wayland the situation looks better. Instead of rendering to an overlay window the compositor can directly interact with the hardware through e.g. DRM. This allows to for example take the buffer provided by VLC and pass it directly to DRM and bypass the compositing all together. But modern hardware allows us to do more than Xorg ever allowed us through “layers”. Be it the raspberry pi, Android’s hwcomposer or Linux’s KMS Planes, the idea is always similar: let the hardware compose the final image together.

If we think of Plasma phone we realize that in most cases we just need three layers: top panel, bottom panel and a maximized window in between. There is no need to do complex composition of the scene at all as all that is needed is putting the three visible (and opaque) windows in the three layers.

On a desktop system it’s a little bit more complicated. Our windows are not all maximized, they have translucent window decorations, shadows, there is a panel with blurred background. Especially the latter one needs a composition through OpenGL. Nevertheless there should be situations where we can make use of layers. Especially in the case of a fullscreen or maximized window this should help making the compositor faster.

Using layers will mean a large change on how KWin’s compositor works. The infrastructure for unredirecting of fullscreen windows cannot be reused in a sensible way as it is evaluated only for fullscreen windows and before the compositor runs. That is before the effects are executed. The decision whether to put a window into a layer needs to come at a later point: once we have evaluated whether we need to compose the complete screen (you might be wobbling a window) or can short pass through layers.

What I have in mind is to give this architecture a try with the VC4 hardware of the raspberry pi which supports layers in a decent way. And to first adjust the QPainter based compositor (as it’s the simpler one) to make use of this architecture. The idea is to experiment with it to get a useful generic KWin-internal backend API, so that we can also use it for OpenGL compositor and even more for our DRM and hwcomposer backend.

For the DRM backend I want to use the new atomic modesetting feature which will be introduced in Linux 4.2. Given that we cannot depend on it anytime soon, this will still need some time till it will be implemented. Overall all of this is not going to happen tomorrow – there are still more important things to work on. But if you, dear reader, are interested on working on this, please contact me. I’m happy to walk you through the code and share my ideas.

As this blog post is mostly about optimizing the compositor by making use of new hardware features which we didn’t have on X11, I can already point out that I will do a follow up blog post on Vulkan.

Now to something else!

KDE is currently running a fundraiser for the Randa Meetings with the topic “Bring Touch to KDE!”. I personally will not participate in the Randa Meetings, but many other KDE developers will go there and we need your support for it. Please help and make the Randa Meetings possible.


A Qt Platform Abstraction plugin for KWin

In Qt we have the Platform Abstraction (QPA) which allows to better interact with the used windowing system through a plugin. In case of KWin we use the “xcb” plugin on X11 and on Wayland we used to use the “wayland” plugin provided by QtWayland. For quite some time I had been thinking about migrating away from those and use an own KWin-specific plugin at least for Wayland.

What’s wrong with QtWayland?

QtWayland provides a plugin for Wayland clients, but KWin is not a Wayland client: it is the Wayland server. This created some interesting “problems” which we had to workaround in KWin. KWin just doesn’t fit the usecase of QtWayland QPA plugin and it would not be a good idea to change QtWayland in a way that it supports KWin as a first-class citizen – it’s just a too special use case. So let’s look at some of the issues which motivated me to write an own QPA plugin.

Connecting to the Wayland server

The wayland plugin is meant for Wayland clients and tries to connect to the Wayland server during the creation of the QGuiApplication. As KWin is the Wayland server the plugin wants to connect to, we needed to make sure that the Wayland server and the event loop are up and running before constructing the QGuiApplication. This added a restriction that we have to monitor the server socket from the main thread.

Blocking roundtrips

The wayland plugin performs a blocking roundtrip to the Wayland server during startup to wait for all outputs to be announced. But this happens from the main gui thread, in which also the server lives. You can guess what happens: dead-lock. We fixed this one in QtWayland, but the problem was still there: any blocking call from the main gui thread to the Wayland server would freeze KWin. We experienced this with e.g. eglInitialize, startup of dbus services (e.g. kded, kamd), rendering of Qt internal windows in case the compositor didn’t send the frame rendered yet. Overall I found this aspect way to fragile and consider especially the fact that the server is in the main thread as a problem.


KWin uses the Qt::BypassWindowManagerHint on all it’s windows because this is needed on X11. The hint is X11 specific and doesn’t make much sense on other platforms, so the Wayland plugin doesn’t show such windows at all. That is a valid approach – there is nothing I could say against it. So in KWin we needed to remove the hint for all windows on Wayland.

OpenGL Context for QtQuick

QtQuick requires an OpenGL context and if it cannot create one it will abort. That is normally not a problem but can easily become one in the case of KWin: KWin creates a “low-level” OpenGL context using the specific platform (e.g. DRM/GBM, hwcomposer, whatever). Qt(Wayland) doesn’t know which one KWin uses and might not even have code for it. QtWayland tries to create an OpenGL context using the EGL platform Wayland. In the case of Mesa everything works fine, but we noticed that on libhybris the creation of a Wayland context fails if there is already one created for hwcomposer. One could consider this as a libhybris specific issue, but I wouldn’t say so – interacting with multiple EGL platforms from one process is kind of out of the specification.

Window decorations

This is a small issue: QtWayland creates window decorations around all Qt windows. But in the context of KWin we do not want any decorations. So we had to disable them by setting the QT_WAYLAND_DISABLE_WINDOWDECORATION environment variable.

And here is the plugin

So after Akademy I decided to work on it and see whether we can get a plugin working. Now it’s in a state where it works good enough to blog about it. The plugin does not support everything that might be used by Qt, but rather reduces to the feature set used by KWin. For example we don’t need to forward any input events, because KWin (mostly) handles them internally anyway and sends them directly to the respective internal Qt window.

The plugin brings the advantage that it no longer depends on the Wayland server being started before creating the QApplication. In fact it does not even try to connect to it, but can use KWin’s internal connection. Similar we can use this to shortcut e.g. creation of low level windows, etc.

On the OpenGL side the plugin approach shows it’s true strength. As it can access KWin internals we can use the EGLDisplay used by the compositing scene and create a sharing OpenGL context. So we can bypass the windowing system completely to render QtQuick windows and this will also allow us to make our window textures available to QtQuick. So a nice improvement.

In case our compositor doesn’t use OpenGL the plugin creates an OpenGL context for the platform Wayland. This is done in a way that it doesn’t hit the blocking issue I mentioned above.

So is the plugin something you should use in other applications?

The plugin interacts with KWin internals and thus is not in any way useable for non-KWin applications. In fact it even checks whether the running binary is KWin and does nothing if that’s not the case. It will also never be turned into a generic plugin as there is no need for that (just use QtWayland).

When will this plugin get merged?

The plugin can be found in branch “qpa” on my personal kwin clone on git.kde.org. It’s not yet in a state that I could push it to master. I still need to test more and make sure that everything works as intended.

Now to something else!

KDE is currently running a fundraiser for the Randa Meetings with the topic “Bring Touch to KDE!”. I personally will not participate in the Randa Meetings, but many other KDE developers will go there and we need your support for it. Please help and make the Randa Meetings possible.


Should we target EGL as the default?

When we started the compositing work in KWin the only way to initialize an OpenGL context was by using GLX. In fact GLX is even part of the OpenGL library on Linux. Being an X11 window manager and an X11 compositor it was not a big problem.

Years later we started the effort to get KWin using optionally OpenGL ES in addition to normal OpenGL. One of the differences is that OpenGL ES doesn’t use GLX, but rather EGL. So we gained code to setup compositing using EGL. But this was still a compile time switch and only supported together with using OpenGL ES. If I remember correctly the option to create on OpenGL context on top of EGL was not yet available in our drivers back then.

But there was a time when the FLOSS drivers started to support it and since the 4.10 release in beginning of 2013 we can provide a normal OpenGL context over EGL. This option was not really exposed and only bound to an environment variable. Very few users knew about it and this was also wanted. Using EGL was a good way to shoot yourself in the foot given that not all drivers support it and that we had much more testing on the GLX backend.

Now the situation has changed. With 5.x we expose EGL also through a config interface, but more importantly EGL is the only way to get OpenGL on Wayland. The assumption that every user is using GLX doesn’t hold any more. The assumption that we don’t need to spend so much time (as it’s only relevant in the reduced feature set of mobile devices) on the EGL backend is wrong. It must reach the same quality as the GLX backend, otherwise our Wayland experience suffers.

So this raises an important question: should we try targeting EGL as the default? In the past one of the main reasons to not even think about EGL as default is the reason that the NVIDIA driver didn’t support it. If we would have switched to EGL by default it would have meant dealing with the situation that EGL doesn’t work and we need to fall back to GLX. It would have complicated the startup significantly and would have resulted in our user base using different rendering paths.

Now this one road blocker seems to resolve itself. While I haven’t tested it yet, the latest NVIDIA beta driver supports OpenGL over EGL, so we should be able to have the majority of our user base on the same backend after all. Given that it is only a beta driver and the feature is marked as “experimental” it’s probably still some time till we can default to it.

Nevertheless it’s time to start thinking about it. I encourage NVIDIA users to give EGL now a try and to report issues. Also I encourage all other users to report issues they see with the EGL backend. If we get the EGL backend into a good state maybe 5.6 might be a good point in time to default to it and then start to slowly deprecate the GLX backend with the aim to get it removed in the distant future.

Porting Qt applications to Wayland

During Akademy I hold a session about porting applications to Wayland. I collected some of the general problems I saw in various KDE projects and want to highlight them in this blog post, too.

Starting Wayland

First the basics: how to start a Wayland compositor? Assuming one is using Plasma 5.4 (aka master) the way to start a nested KWin is:

kwin_wayland --xwayland

Important: ensure that you have Xwayland installed!

If you run nested this will open a window, if it turns black, everything works. If it doesn’t turn black: contact me (please check whether you use free drivers first).

As an alternative you can also start Weston:


If you run into issues with your application running under Wayland, please always test whether the problem is also visible on Weston before blaming me 😛

Starting applications on Wayland

Now we have our black KWin window and we want to get a window to it. There are several ways to start it. As an example I’ll use kate.

Environment variable

Maybe the easiest way is to just use an environment variable to switch the QPA platform to Wayland:

export QT_QPA_PLATFORM=wayland

And a Kate window will open on the nested KWin.

The environment variable is set to wayland in our startplasmacompositor script to start a full Plasma on Wayland session on tty.

Command line switch

Every Qt application also has a command line switch to select the windowing system:

kate --platform wayland

Starting applications together with KWin

KWin supports starting applications once the Wayland and Xwayland servers are started. You can add the programs to start to the command which starts KWin:

kwin_wayland --xwayland "kate --platform wayland"

Or to simplify with environment variables:

export QT_QPA_PLATFORM=wayland
kwin_wayland --xwayland kate

Of course it’s also possible to just start a Konsole and then launch applications from inside the Konsole. Please note that any key combination with ctrl-key is unfortunately broken when running Konsole as a Wayland application.

X11 Specific code

Many applications have X11 specific code. Being it implicitly or explicitly. So we need to move away from it and in most cases it means also improving the situation on X11 because the solution is wrong.

Opening X connection

A common mistake I saw in Qt applications is to open a connection to the X server. E.g. like that:

Display *dpy = XOpenDisplay(NULL);

Now this code is already wrong on X11 as it opens another connection which is not the Qt display connection. It might be on the wrong head, it might be on the wrong X server. From an integration point of view it’s broken.

So the solution is to always use QX11Info from QtX11Extras to integrate with the Display:

Display *dpy = QX11Info::display();

Runtime checks in addition to compile time checks

Before Qt5 the way to support multiple windowing systems was to use compile time switches, like:

#if HAVE_X11
#include <QX11Info>

void doSomethingWithX11()

Now this will crash on Wayland. It will be compiled in, but crashes when running on Wayland. So you need to improve. Let’s try:

#if HAVE_X11
#include <QX11Info>

void doSomethingWithX11()
    Display *dpy = QX11Info::display();
    if (!dpy) {

Now this code will also crash! The problem is that QX11Info::display() goes through Qt’s native interface and uses the key “display”. QtWayland also provides the key “display” and returns a wl_display for it, which gets casted into a void in the native interface and then to Display* in QX11Info and boom at interesting places.

The proper way is to check the platform name:

#if HAVE_X11
#include <QX11Info>

void doSomethingWithX11()
    if (!QX11Info::isPlatformX11()) {

My recommendation is to keep the check in a local variable if the check is needed multiple times as in the end it’s a string comparison each time. Ideally you should restructure your code to have platform specific subclasses or similar refactored solutions.

General Changes with Wayland

Now here it becomes interesting: runtime changes. The most important information for application developers is probably that there are no more global window ids.

No global window ids

This has quite some implications for your application. E.g. QWindow::winId() returns an internal incremented number. Neither the compositor nor any other application knows anything about your internal window id counter. Similarly QWindow::fromWinId is not able to map a window id to a QWindow any more. This of course affects use cases like sending a window id through DBus to another process to use it for setting the transient hint. Also it means that KWindowSystem API has a hard time to support the calls on Wayland and in most cases it will just print a warning (this might improve, though).

Setting window icon

On X11 the applications exported the window icon as pixmap data. This is no longer possible on Wayland. Instead the compositor maps the window icon from the desktop file and the application announces the desktop file for a window. In Qt this is done by using the reverse domain name combined with the binary name. The domain name is set with KAboutData and defaults to “kde.org”. So for example in the case of kate the desktop file should be called “org.kde.kate.desktop”. So please fix the naming of your desktop files. If you want to test please be aware that this requires Qt 5.5 to work properly.

No longer working API

Quite some of the API calls of QWindow cannot be supported by Wayland on default (some of them might work on KWin, but please don’t implement against KWin!).
A short list of those API calls I know that they cannot work:

  • QWindow::setPosition
  • QWindow::setMouseGrabEnabled
  • QWindow::setKeyboardGrabEnabled
  • QWindow::alert
  • QWindow::lower
  • QWindow::raise
  • QWindow::requestActivate

In addition it’s no longer possible to grab the image of the screen using e.g. QScreen::grabWindow (most obvious for the case of “root” window). Also warping the pointer using QCursor::setPos is no longer supported. Please note that warping the pointer is also a bad idea on X11 and you shouldn’t do that.

In case your application set the Qt::BypassWindowManagerHint on your QWindow/QWidget you need to do some porting as QtWayland doesn’t show such windows. Unfortunately that needs a case by case evaluation of the use case and the solution I presented during my Akademy talk should not be applied for applications.

Porting to Wayland

Don’t port

The solution to port to Wayland is: DON’T PORT! Mostly your X11 specific code or behavior is wrong and could be solved in better ways. Instead of porting you might want to rethink your problem or change to existing abstracted API in e.g. Qt or KWindowSystem. I have seen many cases where X11 API was used as a workaround by e.g. raising windows, force activating them and so on. If one things about it one realizes that there seems to be an obvious problem. So don’t try to port that but rethink your problem and improve not just on Wayland, but also on X11.

But I really need platform specific code

Ok, ok, I get it! Now let’s start with the obvious: your code needs to be compile time and runtime checked. Let’s start with how to not do it:

#if HAVE_X11
    if (!QX11Info::isPlatformX11()) {
    // here goes the X11 specific code
    // here goes the Wayland specific code

The problem with this code snippet is that X11 support is probably also available when you use Wayland. Also the else part is not just Wayland, but also any other platform.

As I said the proper solution is to make it compile and runtime checked:

#if HAVE_X11
    if (QX11Info::isPlatformX11()) {
    if (QGuiApplication::platformName().startsWith(
            QLatin1String("wayland"), Qt::CaseInsensitive)) {

Be aware that your code might also run on other platforms on Wayland, e.g. eglfs or other custom windowing systems. So don’t crash if you don’t hit any of the code paths 😉

Plasma Phone and KWin

As you are probably aware by now we announced the Plasma Phone project during Akademy this weekend. In this blog post I want to discuss the role of KWin in Plasma Phone.

Plasma Phone uses Wayland as the windowing system with KWin being the Wayland compositor. This is our first product which uses Wayland by default and also the first product which uses KWin as the Wayland compositor. The phone project pushed the Wayland efforts in Plasma a lot and is the only reason why we are able to make Wayland a technological preview with the upcoming Plasma 5.4 release.

The phone project gave KWin/Wayland into the hands of many developers who started to actively use it and to develop with it. This obviously helped to find lots of small and larger issues which then could be fixed. It triggered a stabilization process which reached a stage that I can use a Plasma Wayland session on my notebook with enough confidence that I won’t lose data due to crashes. So thanks to the whole team for pushing the system to the limits.

An area which saw lots of work thanks to the Phone is the interaction between Plasma as the desktop shell and KWin as the Wayland compositor. With Wayland we keep the architecture of having a dedicated shell process which is not inside the compositor. This architecture has served us well in the past and we don’t see a reason why we should change this. It means that KWin can serve as a compositor for other desktop projects, but it also means that Plasma can be used with other compositors. Now unlike X11, Wayland’s protocols are secure by default. This means that Plasma as the desktop shell does not know anything about windows from other processes. To solve this problem we designed a special window management protocol which exports the required information. The protocols are still under development, but we hope that they can be also useful for other projects with a similar architecture (LXQt being an obvious candidate). Of course such protocols should only be used by the “blessed” desktop shell process – this is something we still need to implement and one of the reasons why at the moment Plasma/Wayland is only a technological preview.

Window management is not the only area where the shell process needs to be “blessed”. Also for announcing information about its own windows, we need some more information. We need to know whether a window is a “panel” or the “desktop” view. So on the phone the shell background is just a desktop window, the panel on the bottom is just a normal dock. This allows us to share the general layering code with the X11 implementation of KWin. Even more having the panels marked as panels allows us to properly define the maximized area. And the windows on the phone open maximized by using the “Maximizing” placement strategy in KWin.

Working on the phone project also had some surprises. For example the virtual keyboard just didn’t want to show. It turned out that the reason for this was that Qt(Wayland) only requests the virtual keyboard if it has keyboard focus. Keyboard focus inside KWin is bound to the “active” window. So we had to implement activating Wayland clients. But the initial implementation didn’t really solve it, we still had areas where the keyboard didn’t come up. E.g. on the desktop in the KRunner search field we couldn’t get it to show. The reason was related to the described problem: we did not yet support activating Wayland clients when clicking on them. This got solved by implementing mouse action (and touch action) support inside KWin. So thanks to this change (done for the phone) we can properly switch between Wayland windows on the desktop with a mouse driven setup.

Another nice touch is that KWin requires a running Xwayland server. This gives us support for all “legacy” X11 applications such as Qt 4 or GTK2 on the phone. Just we hit a problem with them: they did not react on touch events. The reason for this is quite simple: touch support is not yet implemented in Xwayland. As a solution we implemented simulating pointer events in case a connected Wayland window does not bind the touch interface. Thus all Wayland and X11 applications can now be used with a touch screen – be it on the phone or on a notebook with a touch screen.

So far I have only spoken about progress made in KWin which is relevant for the desktop. So what about the phone specific adjustments? Well there are hardly any. In the core of KWin there is no phone specific code. The only phone specific code is the hwcomposer backend. This is a plugin used for creating an OpenGL context using libhybris and for reading input events (unfortunately libinput cannot read events on a libhybris enabled system). According to cloc this plugin is just about 800 lines of code and about 200 are just very straight forward mapping of Android key codes to “normal” Linux key codes. For comparison: KWin + kwayland/server currently have about 120,000 lines of code. And even this hwcomposer backend is not phone specific. It could also be used to run a normal KWin/Plasma session on any libhybris enabled device. There is one important difference of the plugin to the rest of KWin which is worth to mention: it is GPLv3+ licensed, while everything else is GPLv2+. The reason for this change is the fact that libhybris is Apache 2 licensed and this license requires a change to GPLv3.

Plugins, plugins, plugins – or how KDE Frameworks learned to Wayland

Several of KDE’s frameworks have windowing system specific code. This means that they need adjustments if they should be run on more than just X11. The frameworks were already adjusted to only call into X11 specific code if they are used on the X11 platform, but a proper Wayland port was still missing.

Over the last weeks I prepared several frameworks to support Wayland in a proper way. Especially I looked at KGlobalAccel, KIdleTime and KWindowSystem. For all three frameworks there is no “generic” Wayland solution available. That is there is no official interface available which could be used. So in order to add support we need to restrict the usage to KWin for the moment. This is especially the case for KGlobalAccel where the aim is to integrate directly into KWin as described in a previous blog post.

For KWindowSystem I wanted to reuse some Plasma-specific interfaces which we had been developing for the Plasma 5.4 release anyway. As those interfaces are part of the KWayland repository it was an obvious idea to reuse KWayland instead of duplicating the code. But using KWayland from a framework – especially a tier 1 framework is not trivial. KWayland is part of Workspace, thus depending on KWayland would create circular project dependencies. Turning KWayland into a framework would adjust the tier for pretty much all frameworks. Clearly neither a good idea.

Thinking about the problem one realizes there is another project with similar problems and they solved it: Qt. QtWayland provides plugins for Qt and by that qtbase itself does not need to depend on Wayland. So for our frameworks plugins can also be a good solution. We can provide plugins which depend on the respective frameworks and KWayland. As KWin would (for the time being) the only compositor to provide the interfaces anyway, it’s not a problem to depend on the KWayland libraries which are part of Workspace.

So as a first step the frameworks got restructured to have their platform dependent code provided by plugins which get loaded at runtime depending on the platform they are run in. In a second step the header files for the plugin interface needed to be installed. This is a requirement for the integration plugins to provide a plugin.

Now as a third step the development of the interfaces was required. This happened in a new repository called kwayland-integration which will see the initial release with Plasma 5.4. So far this repository provides two plugins: one for KIdleTime, one for KWindowSystem.

For KIdleTime I designed a specific Wayland protocol which allows us to get notified when a seat went idle for a specified interval. This is an important feature for many applications, like screen brightness, setting chat to away, etc. Given that I think it’s an interface which might be in general useful and I think it could be a candidate for inclusion in Wayland. This is something I will look into after Akademy when I have more time for it.

KWindowSystem is a little bit more Plasma/KWin specific, so I doubt that we can have anything which is in general useful for all Wayland compositors. KWindowSystem is also on X11 a little bit KWin/Plasma specific, so there is no general difference.

Nevertheless platforms should provide plugins for these frameworks if they want to support frameworks based applications properly. That’s especially a hint to all those distributions who decided to create their own windowing system.

For the long term I think the way to go is to make KWayland part of frameworks (targeting tier 1) and then KWayland-Integration also a framework (current dependencies would aim for tier 2, but also tier 4 could be a reasonable target). This would be especially important if the idle time interface becomes a proper Wayland interface and would be provided by more compositors than just KWin.

More information about the bigger picture of KWayland-Integration and Plasma on Wayland at Akademy next weekend :-) We have great Wayland things to show.

Global shortcut handling in a Plasma Wayland session

KDE Frameworks contain a framework called KGlobalAccel. This framework allows applications to register key bindings (e.g. Alt+Tab) for actions. When the key binding is triggered the action gets invoked. Internally this framework uses a DBus interface to communicate with a daemon (kglobalaccel5) to register the key bindings and for getting notified when the action triggered.

On X11 the daemon uses the X11 core functionality to get notified whenever key events it is interested in happen. Basically it is a global key logger. Such an architecture has the disadvantage that any process could have this infrastructure and it would be possible for multiple processes grabbing the same global shortcut. In such a case undefined behavior is triggered as either multiple actions are triggered at the same time or only one action is triggered while the others do not get informed at all.

In addition the X11 protocol and the X server do not know that kglobalaccel5 is a shortcut daemon. It doesn’t know that for example the shortcut to lock the screen must be forwarded even if there is an open context menu which grabbed the keyboard.

In Wayland the security around input handling got fixed. A global key logger is no longer possible. So our kglobalaccel5 just doesn’t get any input events (sad, sad kglobalaccel5 cannot do anything) and even when started on Xwayland with the xcb plugin it’s pretty much broken. Only if key events are sent to another Xwayland client it will be able to intercept the events.

This means a global shortcut handling needs support from the compositor. Now it doesn’t make much sense to keep the architecture with a separate daemon process as that would introduce a possible security vulnerability: it would mean that there is a way how to log the keys. One only needs to become the global shortcuts daemon and there you go. Also we don’t want to introduce a round trip to another application to decide where to deliver the key event to.

Therefore the only logical place is to integrate global shortcut handling directly into KWin. Now this is a little bit tricky. First of all kglobalaccel5 gets DBus activated when the first application tries to access the DBus interface. And of course KWin itself is using the DBus interface. So KWin starts up and has launched the useless kglobalaccel5. Which means one of our tasks is to prevent kglobalaccel5 from starting.

Of course we do not want to duplicate all the work which was done in kglobalaccel. We want to make use of as much work as possible. Because of that kglobalaccel5 got a little surgery and the platform specific parts got split out into loadable runtime plugins depending on the QGuiApplication::platformName(). This allows KWin to provide a plugin to perform the “platform specific” parts. But the plugin would still be loaded as part of kglobalaccel5 and not as part of KWin. So another change was to turn the functionality of kglobalaccel into a library and make the binary just a small wrapper around the library. This allows KWin to link the library and start kglobalaccel from within the KWin process and feed in its own plugin.

Starting the linked KGlobalAccel is one of the first things KWin needs to do during startup. It’s essential that KWin takes over the DBus interface before any process tries to access it (as a good part it’s done so early that the Wayland sockets do not accept connections yet and Xwayland is not even started). We will also try to make kglobalaccel5 a little bit more robust about it to not launch at all in a Plasma/Wayland session.

Now the reader might think: wait, that still gives me the possibility to install a stealth key logger, I just need to create shortcuts for all keys. Nope, doesn’t work. As key events get filtered out a user would pretty quickly notice that something is broken.

Integrating KGlobalAccel into KWin on Wayland brings an obvious disadvantage: it’s linked to KWin. If one wants to use applications using KGlobalAccel on other compositors some additional work might be needed to use their local global shortcut system – if there is some. For most applications this is no problem, though, as they are part of the Plasma workspace. Also for other global shortcut systems to work with KWin it’s needed to port them to use KGlobalAccel internally when running in a Plasma/Wayland session (that’s also a good idea for X11 sessions as KGlobalAccel can provide additional features like checking whether the key is already taken by another process).

From the land of wobbly windows

Creative Commons Attribution-ShareAlike 3.0 Deutschland
This work by Martin Gräßlin is licensed under a Creative Commons Attribution-ShareAlike 3.0 Deutschland.