Wayland in Plasma 5.7

Last week we released the beta version of Plasma 5.7 which means we know what this release will have for better Wayland support. First of all I need to mention what didn’t make it: unfortunately I missed the freeze of Frameworks 5.23 to land support for xdg-shell. I have a working implementation, but was not yet satisfied with the API. This is a difficult interface to provide an API for due to the unstable nature of the interface. Due to lack of xdg-shell support GTK applications are still going to use X11 on Wayland (like the Firefox window I’m just typing this blog post in).

In the past I already blogged about a few new features in 5.7 like the improved task manager for Wayland, the virtual keyboard support, sub-surface support and improved input device support. So in this blog post I want to focus on a different topic: quality.

For Plasma 5.7 my aim was to get the Plasma session into a state that I can use it as my primary system. And since last week I have not started into an X11 session any more. This means that we needed to get the whole system stable enough to have neither KWin nor applications crash due to Wayland. Given that our Wayland code is quite a fair amount of new code, changing lots of assumptions there are of course bugs to be expected. We still have code which calls into X11 unconditionally, we have things which are not implemented correctly and of course we do stupid mistakes. So for Plasma 5.7 the task was to find these issues and fix one for one.

For Wayland it’s much easier for us to test. KWayland – our framework for Wayland support – is developed in a test driven approach making it possible to create test cases for every problem. They expose the problems and verify that they are fixed and as regression tests ensure that they won’t hit us again. Over the last release cycle we added several thousand lines of test code in KWayland alone.

Finding those issues is not always easy. If KWin crashes we don’t have DrKonqi like normally, it doesn’t work for Wayland (tries to connect to a display server, but that just crashed). What I saw on my Wayland test device was that KWin sometimes randomly crashed – more often when I interacted with X11 windows. But when attaching gdb to KWin it didn’t crash. But once I caught it: it turned out to be an error in KWin in the handling of Xwayland windows. There are two possible code paths it can take and one was with a mistake. Due to running through a debugger it was more likely to take the correct one. So yeah it’s not always easy.

With that problem gone we were able to find a few more and fix also some bugs which caused windows to quit. Unfortunately some of this fixes had to go into KWayland after the 5.23 release. This means the frameworks version used with Plasma 5.7 is not going to have all fixes. If you want to give Plasma/Wayland a try I recommend to not just wait for Plasma 5.7 but also for frameworks 5.24.

This week I will be at the openSUSE conference in Nuremberg, where I will also give a talk about how Wayland helps us to improve our quality and our workflows. I’ll do another blog post about the content of that presentation – don’t want to spoil ­čśë Though if you follow our development you are already aware of it. Thanks to openSUSE to give me the possibility to present at the conference and thanks to KDE e.V. for the support to go there.

I can hear you asking now the question of all questions: “When will it be ready?” I think that I am not objective enough to answer the question or to say that it is ready. I’m too close to the code and might just omit important problems because I don’t see them. Thus I cannot say that it is ready. It depends on your workflow and whether that workflow is already fully implemented. This is something only you can know.

Last week KDE had a very important developer sprint (where I did not participate) and is currently running a fundraising campaign for this sprint. We need the money to send our developers to such meetings or to a conference like openSUSE conf where I will be this week. At the moment just 107 people have participated and donated. This is something which makes me sad. I see the statistics for my blog posts and know that this one will have at least 1000 direct hits. In addition there are people reading planetkde and not directly my blog. We are trying to raise 24000 EUR and please do the math yourself to see how close we would be to it if everybody would donate just 10 EUR.

Why planetkde needs to have political posts

Johnathan’s blog post about the EU triggered quite some noise about that there should not be political posts on planetkde. I think it’s important that we have political posts on planetkde. KDE is a political movement, let’s look at our vision:

A world in which everyone has control over their digital life and enjoys freedom and privacy.

This is a political statement. We want to change the world! We are political! We need to be able to discuss political topics like the Snowden-documents, we need to be able to lobby on planetkde for Free Software in our governments, we need to be able to report about such efforts.

This means politics belong to KDE and belong to planetkde.

This means we need to also tolerate political opinion posts on topics which are not directly KDE related, like Johnathan’s. If we start to remove such posts or start to censor us, we run the risk of not having the relevant posts on planetkde anymore because people don’t post political articles at all anymore.

Please remember that planetkde is for the KDE community by the KDE community. Any political opinion expressed on planetkde does not represent the opinion of KDE or KDE e.V.

A task manager for the Plasma Wayland session

Plasma 5.7 will ship with a new taskmanager library. One of the reasons to implement a new taskmanager library is the port to Wayland. Our old code base was heavily relying on X11 and didn’t support the concept of multiple windowing systems. You can read more on that in Eike’s blog post about the new task manager. In this blog post I want to focus a little bit on the Wayland side of a task manager.

Architectural difference to X11

On X11 every application is able to read every aspect of every other window. This allows to implement a task manager. The task manager can get notified when a window is added, when a window is removed and install event filters for the changes of the window properties. That’s the basics for a task manager.

On Wayland the world is different. An application is not able to see the windows of other applications. This means by default Plasma is not able to see what windows exists, the taskmanager would be empty. The only application which knows about the existing windows is the Wayland compositor.

Merging Desktop Shell and Wayland compositor?

A possible solution to this problem could be to merge the desktop shell and the compositor into one process. This is the architecture used in e.g. Unity and GNOME Shell on X11.

But such an architecture has also disadvantages. It violates the concept of separating concerns and of “doing one thing and do it right”. We have an excellent task manager, an excellent window manager and an excellent desktop shell.

Also it would make code sharing between platforms more difficult. A task manager for Wayland would be very different from a task manager on X11. All the business logic and presentation should be the same. No matter which windowing system is used.

By merging the code bases together we would also make the code less reusable. One couldn’t use KWin/Wayland in another desktop environment (e.g. LXQt) nor could one use Plasma/Wayland with another compositor (e.g. sway). While we of course recommend to use KWin as the Wayland (and X11) compositor we don’t enforce it and also don’t want to enforce it.

Protocols, protocols

So a different solution is needed and that is done through a custom Wayland protocol which is called org_kde_plasma_windowmanagement. We have a few interfaces prefixed with Plasma. This does not mean that it can only be used by Plasma. It just means that the interface was first designed and developed for Plasma’s needs. We are happy to share the interface with other desktop environments.

The protocol is provided by KWin and it announces whenever a new window gets created. Once a new window gets created a bound client can bind an org_kde_plasma_window interface and get notified about all the states of the window.

The protocol exposes pretty much all the information KWin has about the window and it gets updated automatically whenever the state in KWin changes. In addition the protocol has requests the task manager needs, like “close the window”, “minimize it”, “maximize it”, etc. etc. KWin also listens to these requests and honors them.

Although the protocol is added to Wayland, it is windowing system agnostic. The created Plasma Window does not expose the actual windowing system resource (after all on Wayland another client cannot get access to it). KWin exposes a Plasma Window for both X11 managed windows and Wayland windows. By that the task manager is able to manage tasks from multiple windowing systems without knowing that it does so.

Evolution of the protocol

We added the Plasma window protocol initially for the work on the Plasma phone about a year ago. Also on the Plasma phone we had the use case of switching windows and using the X11 based task manager was just no option on a Wayland only device ­čśë

While the protocol worked fine for our initial needs, it was not yet sufficient for the use in the desktop task manager. So over the last weeks we introduced further support and improved the API to make it work more reliable. Also quite some changes went into KWin to ensure that only “interesting” windows are exposed. E.g. a context menu should not be shown in the task manager.

Our KWayland library of course has support for this protocol through the classes PlasmaWindowManagement and PlasmaWindow on client side. In addition there is a model exposed as PlasmaWindowModel.

Future work

The work on the interface is not done yet. There are still changes to the API needed and the upcoming KDE Frameworks 5.23 release will include new API calls in PlasmaWindow. As well we have already the first change scheduled for KDE Frameworks 5.24. And we also know that we need to expose a few more data elements in the protocol to get the task manager to the same level as on X11.

There is also an interesting idea floating around to use the – windowing system agnostic – Wayland implementation on both X11 and Wayland. For this KWin (on X11) would need to create a dummy Wayland server for the task manager to connect to. It’s an idea which we might or might not implement.

Security considerations

Currently every Wayland client is able to bind this interface, which means that some of the security improvements of Wayland are not available in Plasma/Wayland. We are fully aware of that and were also fully aware of the consequence when we added the interface. I do have ideas on how to address this and this will be implemented before we will recommend the Plasma/Wayland session for daily usage. The design allows to add security checks in these areas. Unfortunately my priority list did not allow me to implement this for Plasma 5.7, next chance 5.8.

Introducing a virtual framebuffer Wayland server in KWayland

The next KDE framework release will include a small addition in KWayland which will make it much easier to run auto tests for Wayland based applications. KWayland now installs a small virtual framebuffer test server into libexec directory.

This means it’s an application inspired by Xvfb which allows to run a graphical application with a virtual framebuffer. The new tool is the same for Wayland. KWayland makes testing already quite easy, but we noticed that we don’t have a good setup for the case that we want to test an application which uses QtWayland QPA plugin as that is not possible with the test support so far in KWayland.

The idea behind the server is that every test gets it’s own dedicated test Wayland server. The test Wayland server will start the test and terminate itself once the test finished (either successful, failure or crash). All output from the test – including exit code – is forwarded, which makes integration into a ctest based system really easy.

In fact if you use CMake as a build system to define your tests, this becomes super easy:

find_package(KF5Wayland CONFIG)
add_executable(myTest myTest.cpp)
target_link_libraries(myTest Qt5::Gui Qt5::Test)
kwaylandtest(myTest)

This also allows to run the same test on X11 and on Wayland. Assuming the test is meant for build.kde.org which has an Xvfb running during the test phase, one can add the test the normal way (X11) and just add the kwaylandtest in addition. So the test will run on both windowing systems.

Now this is a really new project and the test server is still rather limited. All it does is faking rendering at 60 Hz. Otherwise it’s still limited and does not provide the functionality of e.g. a window manager on X11. In that regard it’s also inspired by Xvfb which doesn’t support that. I plan to extend this as I have need in the tests I’m going to add for it.

Speaking of that: the first test case making use of this new infrastructure is already pushed.

Virtual keyboard support in KWin/Wayland 5.7

Over the last weeks I worked on improved input device support in KWin/Wayland and support for virtual keyboard. KWin 5.7 will integrate the new QtVirtualKeyboard module which is now available under GPLv3. For us this means that we have access to a high quality QML based keyboard. For Qt it means that the virtual keyboard is exposed to more users and thanks to the open source nature it means that we can upstream fixes.

The virtual keyboard is integrated into KWin/Wayland making it part of our platform. To support it we implemented the text input interface in KWayland. If an application supports it, then the virtual keyboard can input text from the compositor side.

Video on youtube:

As you can see in the video the virtual keyboard is only enabled by default if no hardware keyboard is available. If a hardware keyboard gets plugged in the virtual keyboard gets disabled automatically. In addition there is a Status Notifier Item registered which allows to enable/disable the virtual keyboard independently of the hardware keyboard state. So on a notebook with a touch screen it’s possible to get the virtual keyboard even if there is (of course) a hardware keyboard.

Implementing support for the virtual keyboard was an interesting journey which required many unexpected changes and improvements all over the place. First of all wl_text_input interface needed to be implemented as that’s what is used in QtWayland 5.6. At one point I hit a wall and saw that the implementation in QtWayland is incomplete. So I checked how it looks like in QtWayland 5.7 and alas the complete implementation changed, it’s using zwp_text_input_unstable_v2, which is not even in wayland-protocols. I’m not happy about QtWayland using not standardized interfaces as I think that’s very bad for compatibility. Nevertheless I also added support for it in KWayland as otherwise we would not be able to communicate with applications using Qt 5.7. As the QtVirtualKeyboard will only be available with Qt 5.7, we need to support Qt 5.7’s interface as well. In KWayland this is implemented in a transparent way and KWin announces support for both interfaces.

Another area which needed lots of work is the support for input devices in KWin. We need to properly detect whether a real keyboard is available. We needed to add further support for touch events in KWin. So thanks to the integration of virtual keyboard KWin now supports touch much better on Wayland.

The improvements for input devices are also useful in other areas. E.g. today I landed a change to not show the cursor image if no pointer device (mouse or touchpad) is connected.

KWayland joined KDE Frameworks

The current release 5.22 of KDE Frameworks gained a new framework: KWayland. So far KWayland got released together with Plasma. KWayland entered as tier 1/integration and is only available on Linux (and other Linux-like systems).

For us working on the Wayland stack in Plasma and KDE this is a very important step. Now we can use KWayland also in other frameworks. Also with KWayland in frameworks we expose it to a larger audience. We hope that it is a useful framework for anyone using Wayland with Qt. It’s not a replacement for QtWayland, rather an addition and way more flexible by being closer to the Wayland API.

This change is also important for distributions. Please note that this is only a logical move in the project structure. KWayland 5.22 is fully ABI compatible to the last release 5.6. You can just exchange the KWayland from Plasma with the one in frameworks. Plasma 5.6 of course works perfectly fine with KWayland 5.22.

Improved input device support in KWin/Wayland

One of the areas I’m currently working on is improving our support for input devices in KWin. While general usage is already quite good in Plasma 5.6, we are not yet able to configure the input devices. This is something I’m working on to improve for Plasma 5.7.

Input devices are provided by libinput and KWin integrates with that for quite some time. What it didn’t do yet is to configure them and keep track of them.

So as a first step we created a Device wrapper class which encapsulates a libinput device in a way that we can easily use it from inside KWin. The Device allows us to gain knowledge about what it supports and also to configure it.

The used Devices and their states can be queried through KWin’s new Debug Console:
Input Devices in Debug console

The Debug Console is also new in Plasma 5.7 and allows us to have a deeper introspection into KWin. It can be opened through KRunner (Alt+Space) and then with the command “KWin”. The functionality set is inspired by tools like xprop (one tab allows to read all window properties), xwininfo -tree (one tab shows a (sub)surface tree), xev (input events).

Back to input devices. Once we had the Device wrapper class in place we could start making use of it to configure devices. And today I landed a series of commits in KWin master which bring us back many features from X11. From the configuration point of view it’s really easy for us as we can just reuse our existing configuration tools. For example the mouse configuration module writes to a config file which KWin can read. When the settings change the configuration module sends out a DBus signal which KWin uses to trigger a reconfigure.

So KWin now supports setting pointer devices (mouse) to left-handed usage and modifying the pointer acceleration.
Toggle Touchpad

In addition KWin started to integrate changes for touchpads. KWin listens to the global shortcut for touchpad toggle and uses this to enable/disable the touchpad. Of course this comes with triggering the new on-screen-display notification for enabling the touchpad. Next step will be to integrate more options for touchpads, but this will require writing a kwin-backend for our touchpad control module.

Running Plasma-Wayland on KDE Neon

This morning I installed KDE Neon on my good old ExoPC as I need to test some touch functionality on Wayland. So here are the steps on how to switch Neon to use Wayland:

Install package plasma-workspace-wayland:

sudo apt install plasma-workspace-wayland

Log out and select in session drop down Plasma (Wayland), log in and enjoy your Wayland session.

If you want to verify that it truly is a Wayland session, just open any KDE application, open the “Help” Menu, go to “About <Application Name>”, click “Version” tab, it should say “The Wayland windowing system” is being used.

Starting KWin/Wayland on another virtual terminal

So far when one started KWin/Wayland on a virtual terminal it took over this virtual terminal. This made it difficult to read the debug output and even more difficult to run the complete session through gdb.

The reason for this behavior is that KWin interacts with logind and needs to take session control on the current logind session. This is needed to have logind open the restricted device files like /dev/dri/card0 or the /dev/input/event* files.

Latest KWin master changes the interaction with logind to have it based on XDG_SESSION_ID. In addition changing virtual terminals is now based on logind instead of custom code for switching through ioctl calls.

A side-effect of these changes is that one can start KWin for another logind session. E.g. it’s now possible to create a logind session on virtual terminal 1 and start KWin on virtual terminal 2 for the logind session on virtual terminal 1. KWin automatically switches the virtual terminal when starting up, so one gets the graphical output. This makes it much easier to get to the debug output.

But this also makes it possible to run KWin through gdb by connecting through ssh. Log in to your system through ssh and start kwin_wayland through gdb with XDG_SESSION_ID pointing to the session on your virtual terminal 1.

March Wayland update: ASCII Art rendering

Another busy month, another round of great features being worked on. March featured our Plasma 5.6 release which had lots of last minute (aka bug fixing) focus. We had our sprint at CERN. But of course also new code was added.

One of the features on my TODO list for Plasma 5.7 is the ASCII art output rendering. Mplayer supports that and while that is nice, it’s not really a good solution. It would be way more convenient to have the complete workspace rendered through ASCII art to not have to implement this feature in every application.

So without much further ado I’m happy to announce the caca platform for KWin:

Glxgears running in nested KWin on caca platform
Glxgears running in nested KWin on caca platform

Unfortunately it’s only supported by our QPainter based compositor. Thus to run it, you need to make sure to use KWIN_COMPOSE=Q to pick the QPainter backend (yeah we should improve that to have a better automatic selection). This is the first platform which support both nested and full mode. Thus you can run with –ascii in your normal X session which will open a window, just like our nested X11 platform plugin. The rendering is done through the great caca library.

Lock screen in KWin's caca platform
Lock screen in KWin’s caca platform

What I’m personally impressed about is how small this new platform plugin is. Less than 400 lines of code, great job by libcaca. But it also shows how easy it is to get KWin on a new platform.