Posted on

Rootless Xwayland server integration into KWin

Over the last weeks I concentrated my KWin related work on trying to integrate the Xwayland server properly. Xwayland is an interesting step on the way to Wayland as it maps X11 windows to Wayland surfaces. But it also needs an X11 window manager to manage the X11 windows. Overall it allows us to start integrating Wayland into the compositor without too much breakage. It’s still X11 after all, so our existing code base continues to work. And gruadually functionality can be replaced with the Wayland equivalent, so that we can afterwards start integrating proper Wayland clients.

Integrating Xwayland showed interesting challenges. KWin as an X11 window manager requires a running X server prior to start. It also can only communicate with one X server, the code base has many hard constraints on being only one X server. This means if we want to use Xwayland the Xwayland server must be the one KWin uses. Which means Xwayland must be started prior to KWin’s X11 usage. So far KWin also enforced the usage of the “xcb” QPA plugin, the windowing system abstraction plugin for the X world in Qt. This plugin gets loaded directly at application startup and aborts if there is no X server it can connect to.

As you can see KWin needs to ensure that Xwayland is running prior to the application startup. But Xwayland requires a running Wayland server, which is supposed to be provided by KWin and for processing Wayland events we need a running event loop, which is only present after the application started. Looks like we are caught in a dependency loop:
circular-dependencies

After some hacking and experimenting I found a solution which can start the Wayland server and Xwayland prior to the application startup, but it’s considerable fragile and it can only be a temporary solution. In the long run it would of course be better if KWin could use the Wayland QPA plugin provided by QtWayland to connect to it’s own Wayland server and start Xwayland later-on.

Nevertheless I succeeded in getting KWin connect to the Xwayland server and to start transitioning X clients to be rendered using Wayland buffers instead of performing texture from X11 pixmap:

Aus 2015-02-11

But it just hit another problem: it didn’t support accelerated OpenGL rendering on the X server. That’s quite a problem if no X11 client connected to the Xwayland server can use proper OpenGL and it’s even a problem for KWin as KWin uses QtQuick scenes which use the X server. So KWin itself fails to render accelerated UI. Compositing is not affected as we don’t use Qt for that.

The reason for the problem is that Xwayland expects the Wayland “wl_drm” interface to be present. This interface gets created when binding an EGLDisplay to a Wayland display. In case of KWin the EGLDisplay exists after the Compositor is fully initialized. And again we are in a dependency chain: the Compositor gets created and uses the Workspace class. This class controls the complete startup of the X11 window manager which means it’s highly X11 dependent and requires Xwayland to be present. Again we are in a dependency loop.

Breaking up this dependency loop is quite tricky. The Compositor is too deeply nested into the application to be considered started before creating the QApplication. This means we must be able to create the QApplication before we have an X Server running. This means no usage of xcb QPA plugin. As KWin is going to start a Wayland server anyway, it would be good to get KWin to use the wayland QPA plugin.

This was again quite a challenge. The QtWayland QPA plugin performs blocking roundtrips to the Wayland server in the main thread during startup. But the Wayland server is running in the same thread. So a blocking call to the Wayland server dead locks the server. It’s not possible to move the Wayland server at that point into a thread as one cannot start a QThread prior to having the QCoreApplication created.

The solution I developed for this problem involves creating an own event dispatcher prior to creating the QApplication. So we have the event dispatching for the Wayland server ready to use. Just the event loop is not yet running. This allowed to provide a small patch for QtWayland to run any event dispatcher set before creating the QApplication. A test application in the kwayland repository is also adjusted to make use of it (the test application is also able to start an Xwayland server prior to creating the QApplication).

Unfortunately there is still another issue: QtWayland might call eglInitialize in the main thread which again performs a blocking wayland call. This is a problem I haven’t solved yet and currently just hacked around by disabling OpenGL in Qt (which breaks the QtQuick views).

Being able to use the Wayland QPA just creates a new bunch of problems for KWin. KWin still needs to use X11 and thus needs to create an xcb connection. Just that this wouldn’t help much. KWin on X11 doesn’t init the xcb connection, it’s the xcb plugin in Qt which does it and we only access it via QX11Info. Thus KWin needs to be moved away from the usage. Luckily in many cases we already wrapped the functionality as going through the QPA interface (which QX11Info does internally) is too expensive for our use cases. So the changes are not that invasive. But KWin also uses frameworks like KF5WindowSystem which use QX11Info. Even more those frameworks were properly fixed to perform platform checks and don’t do the X11 specific code if it’s not on platform xcb. But it’s providing important X11 window manager functionality for KWin. For some classes like KWindowInfo and KWindowSystem the fix was trivial: don’t use in KWin. There’s also the more low-level NETRootInfo and NETWinInfo and that’s what KWin should use – usage of the KWindowInfo or KWindowSystem inside KWin can be considered a bug. For some other classes it was already partially possible to be used without the xcb plugin on X11. The classes are only used if we compile with xcb present, so it was possible to add more xcb specific methods which can then be used by KWin even if we do not use platform xcb. The required changes will be part of frameworks 5.8 release.

To summary where we are now: we can start kwin_wayland on platform wayland connecting to a Wayland server started by kwin_wayland, we do not require QX11Info in (most of) KF5WindowSystem and KWin. We are a good step closer to the aim, but still not there. The dependency loop is still in place: Workspace starts the Compositor, the Compositor creates the EGLDisplay which is needed to start Xwayland, which provides X11 which is needed for starting the Workspace.

This means: reorder the startup. We need to be able to start the Compositor prior to the Workspace (which could be interesting for kwin_x11, too, as it could improve persumed startup time). This task was easier than expected. Workspace got split into many modules over the last years and most modules which need to be created prior to creating the Compositor do not depend on Workspace and do not depend on X11. In the few cases where it actually does depend on X11 it was not difficult to delay the X11 specific code till after the X11 connection is created.

With all that in place I was able to delay starting Xwayland to after the Compositor is created and Xwayland can provide OpenGL to the connected clients:

This screenshot shows kwin_wayland running on top of Weston connected to an Xwayland server supporting proper OpenGL as can be seen by the output of glxinfo (in the konsole) and glxgears. Also plasmoidviewer just works on top of this X stack.

Of course there is still some work to be done till this is production ready code, but it looks really promising and I hope to have this ready for the KWin 5.3 release. The aim is also to get more and more features changed to use the Wayland functionality instead of the X11 functionality. E.g. for damage event handling it already uses the damage event of a wl_surface instead of creating an X11 damage handle. This makes supporting Wayland clients easier afterwards.

Posted on

Global Shortcuts and the Lock Screen

With Plasma 5 our lock screen architecture changed significantly. For example we do no longer support screen saver hacks or widgets on top of the locked screen. Both are very unlikely to make a return in future releases. This means that bug reports against the old infrastructure might no longer apply to our current code base. Two weeks ago I went through all bug reports and feature requests to evaluate whether they still apply to our new infrastructure or should be closed.

This is something I do not like to do. I find it extremely sad to close bug reports because they are outdated. Especially if the bugs have been open for several years without any activity. After going through all those reports it is obvious that we offered too many possibilities to configure the screen locker with too few people caring about it. The number of available screen savers was just immense – especially if one considers that there are also 3rd party savers. While it’s easy to install them, there is basically nobody who cared about them. Some are decades old with the devs having moved on years ago.

Apart from that one could also notice that there were important features missing in our lock screen: audio and multimedia control. The problem is obvious: you suspend your notebook while audio is playing, resume in a place where it should not play audio (e.g. classes) and first need to unlock the screen before being able to mute the audio. An unpleasant experience.

The problem here is that media keys are not supposed to work. The lock screen grabs all keyboard input and prevents other applications to get the keyboard input. This includes our global keyboard infrastructure. We cannot just forward all keys to the global keyboard infrastructure as that could be used to create a key logger when the screen is locked. Even more: most short cuts shouldn’t be invokable when the screen is locked, e.g. you don’t want the desktop to switch.

After brooding over it for a few days I had an idea on how to resolve the problem: the lock screen needs to integrate with our globalshortcut handling. When the screen gets locked we get the available shortcuts from the daemon and map them against a white list of allowed shortcuts. Whenever a key is now pressed while the screen is locked, it’s verified against the fetched list. If we have a match the shortcut is invoked. Not all shortcuts are supported, though. The architecture ensures that one cannot abuse the infrastructure to turn it into a key logger. All alphanumeric keys are excluded. In addition it uses as mentioned a white list, which is not configurable, but hard coded on purpose. At the moment we support only a very limited set of global shortcuts: volume keys, brightness keys and media control keys.

Media control keys were also an interesting topic to work on. Our Plasma session didn’t have any global shortcuts for media handling, so there was nothing which our lock screen could do about it. It cannot figure out whether there is a media application running and then invoke an action on it.

So I stepped back and thought about whether there is a better way to solve it in a general way. Plasma supports the mpris2 interface allowing to control any mpris2-enabled media application. What if the mpris2 engine inside Plasma binds the multi media keys? They could forward to the currently running media player, give us a consistent way to interact with media players and allow us to expose it in the lock screen. So now we have by default mapped the media player controls as global shortcut delegating to any media application. If you press the “Play” button it will Play/Pause VLC, no matter whether it’s the active application or not. And in addition also in the lock screen.

Now there was just one problem to solve: pressing shortcuts and not having visual feedback is not that good. On the desktop we show an on-screen-display whenever the volume changes, but as the screen is locked, we cannot see it. So this architecture also needed enhancement. With a few more changes our lock screen is now able to listen to the requests for on-screen-display information and integrate them:
New lock screen with audio info

All just for supporting shortcuts in the lock screen.