Wayland in Plasma 5.7

Last week we released the beta version of Plasma 5.7 which means we know what this release will have for better Wayland support. First of all I need to mention what didn’t make it: unfortunately I missed the freeze of Frameworks 5.23 to land support for xdg-shell. I have a working implementation, but was not yet satisfied with the API. This is a difficult interface to provide an API for due to the unstable nature of the interface. Due to lack of xdg-shell support GTK applications are still going to use X11 on Wayland (like the Firefox window I’m just typing this blog post in).

In the past I already blogged about a few new features in 5.7 like the improved task manager for Wayland, the virtual keyboard support, sub-surface support and improved input device support. So in this blog post I want to focus on a different topic: quality.

For Plasma 5.7 my aim was to get the Plasma session into a state that I can use it as my primary system. And since last week I have not started into an X11 session any more. This means that we needed to get the whole system stable enough to have neither KWin nor applications crash due to Wayland. Given that our Wayland code is quite a fair amount of new code, changing lots of assumptions there are of course bugs to be expected. We still have code which calls into X11 unconditionally, we have things which are not implemented correctly and of course we do stupid mistakes. So for Plasma 5.7 the task was to find these issues and fix one for one.

For Wayland it’s much easier for us to test. KWayland – our framework for Wayland support – is developed in a test driven approach making it possible to create test cases for every problem. They expose the problems and verify that they are fixed and as regression tests ensure that they won’t hit us again. Over the last release cycle we added several thousand lines of test code in KWayland alone.

Finding those issues is not always easy. If KWin crashes we don’t have DrKonqi like normally, it doesn’t work for Wayland (tries to connect to a display server, but that just crashed). What I saw on my Wayland test device was that KWin sometimes randomly crashed – more often when I interacted with X11 windows. But when attaching gdb to KWin it didn’t crash. But once I caught it: it turned out to be an error in KWin in the handling of Xwayland windows. There are two possible code paths it can take and one was with a mistake. Due to running through a debugger it was more likely to take the correct one. So yeah it’s not always easy.

With that problem gone we were able to find a few more and fix also some bugs which caused windows to quit. Unfortunately some of this fixes had to go into KWayland after the 5.23 release. This means the frameworks version used with Plasma 5.7 is not going to have all fixes. If you want to give Plasma/Wayland a try I recommend to not just wait for Plasma 5.7 but also for frameworks 5.24.

This week I will be at the openSUSE conference in Nuremberg, where I will also give a talk about how Wayland helps us to improve our quality and our workflows. I’ll do another blog post about the content of that presentation – don’t want to spoil 😉 Though if you follow our development you are already aware of it. Thanks to openSUSE to give me the possibility to present at the conference and thanks to KDE e.V. for the support to go there.

I can hear you asking now the question of all questions: “When will it be ready?” I think that I am not objective enough to answer the question or to say that it is ready. I’m too close to the code and might just omit important problems because I don’t see them. Thus I cannot say that it is ready. It depends on your workflow and whether that workflow is already fully implemented. This is something only you can know.

Last week KDE had a very important developer sprint (where I did not participate) and is currently running a fundraising campaign for this sprint. We need the money to send our developers to such meetings or to a conference like openSUSE conf where I will be this week. At the moment just 107 people have participated and donated. This is something which makes me sad. I see the statistics for my blog posts and know that this one will have at least 1000 direct hits. In addition there are people reading planetkde and not directly my blog. We are trying to raise 24000 EUR and please do the math yourself to see how close we would be to it if everybody would donate just 10 EUR.

Virtual keyboard support in KWin/Wayland 5.7

Over the last weeks I worked on improved input device support in KWin/Wayland and support for virtual keyboard. KWin 5.7 will integrate the new QtVirtualKeyboard module which is now available under GPLv3. For us this means that we have access to a high quality QML based keyboard. For Qt it means that the virtual keyboard is exposed to more users and thanks to the open source nature it means that we can upstream fixes.

The virtual keyboard is integrated into KWin/Wayland making it part of our platform. To support it we implemented the text input interface in KWayland. If an application supports it, then the virtual keyboard can input text from the compositor side.

Video on youtube:

As you can see in the video the virtual keyboard is only enabled by default if no hardware keyboard is available. If a hardware keyboard gets plugged in the virtual keyboard gets disabled automatically. In addition there is a Status Notifier Item registered which allows to enable/disable the virtual keyboard independently of the hardware keyboard state. So on a notebook with a touch screen it’s possible to get the virtual keyboard even if there is (of course) a hardware keyboard.

Implementing support for the virtual keyboard was an interesting journey which required many unexpected changes and improvements all over the place. First of all wl_text_input interface needed to be implemented as that’s what is used in QtWayland 5.6. At one point I hit a wall and saw that the implementation in QtWayland is incomplete. So I checked how it looks like in QtWayland 5.7 and alas the complete implementation changed, it’s using zwp_text_input_unstable_v2, which is not even in wayland-protocols. I’m not happy about QtWayland using not standardized interfaces as I think that’s very bad for compatibility. Nevertheless I also added support for it in KWayland as otherwise we would not be able to communicate with applications using Qt 5.7. As the QtVirtualKeyboard will only be available with Qt 5.7, we need to support Qt 5.7’s interface as well. In KWayland this is implemented in a transparent way and KWin announces support for both interfaces.

Another area which needed lots of work is the support for input devices in KWin. We need to properly detect whether a real keyboard is available. We needed to add further support for touch events in KWin. So thanks to the integration of virtual keyboard KWin now supports touch much better on Wayland.

The improvements for input devices are also useful in other areas. E.g. today I landed a change to not show the cursor image if no pointer device (mouse or touchpad) is connected.

Improved input device support in KWin/Wayland

One of the areas I’m currently working on is improving our support for input devices in KWin. While general usage is already quite good in Plasma 5.6, we are not yet able to configure the input devices. This is something I’m working on to improve for Plasma 5.7.

Input devices are provided by libinput and KWin integrates with that for quite some time. What it didn’t do yet is to configure them and keep track of them.

So as a first step we created a Device wrapper class which encapsulates a libinput device in a way that we can easily use it from inside KWin. The Device allows us to gain knowledge about what it supports and also to configure it.

The used Devices and their states can be queried through KWin’s new Debug Console:
Input Devices in Debug console

The Debug Console is also new in Plasma 5.7 and allows us to have a deeper introspection into KWin. It can be opened through KRunner (Alt+Space) and then with the command “KWin”. The functionality set is inspired by tools like xprop (one tab allows to read all window properties), xwininfo -tree (one tab shows a (sub)surface tree), xev (input events).

Back to input devices. Once we had the Device wrapper class in place we could start making use of it to configure devices. And today I landed a series of commits in KWin master which bring us back many features from X11. From the configuration point of view it’s really easy for us as we can just reuse our existing configuration tools. For example the mouse configuration module writes to a config file which KWin can read. When the settings change the configuration module sends out a DBus signal which KWin uses to trigger a reconfigure.

So KWin now supports setting pointer devices (mouse) to left-handed usage and modifying the pointer acceleration.
Toggle Touchpad

In addition KWin started to integrate changes for touchpads. KWin listens to the global shortcut for touchpad toggle and uses this to enable/disable the touchpad. Of course this comes with triggering the new on-screen-display notification for enabling the touchpad. Next step will be to integrate more options for touchpads, but this will require writing a kwin-backend for our touchpad control module.

March Wayland update: ASCII Art rendering

Another busy month, another round of great features being worked on. March featured our Plasma 5.6 release which had lots of last minute (aka bug fixing) focus. We had our sprint at CERN. But of course also new code was added.

One of the features on my TODO list for Plasma 5.7 is the ASCII art output rendering. Mplayer supports that and while that is nice, it’s not really a good solution. It would be way more convenient to have the complete workspace rendered through ASCII art to not have to implement this feature in every application.

So without much further ado I’m happy to announce the caca platform for KWin:

Glxgears running in nested KWin on caca platform
Glxgears running in nested KWin on caca platform

Unfortunately it’s only supported by our QPainter based compositor. Thus to run it, you need to make sure to use KWIN_COMPOSE=Q to pick the QPainter backend (yeah we should improve that to have a better automatic selection). This is the first platform which support both nested and full mode. Thus you can run with –ascii in your normal X session which will open a window, just like our nested X11 platform plugin. The rendering is done through the great caca library.

Lock screen in KWin's caca platform
Lock screen in KWin’s caca platform

What I’m personally impressed about is how small this new platform plugin is. Less than 400 lines of code, great job by libcaca. But it also shows how easy it is to get KWin on a new platform.

February KWin/Wayland update: all about input

I haven’t blogged for quite some time about the progress on KWin/Wayland and had a few people requesting an update. As we are now approaching a feature freeze and I have most of the things I wanted to do for Plasma 5.6 done, it’s time to blog again. I use this also as a public service announcement: thanks to Let’s Encrypt my blog is also available through an encrypted connection.

Last month my development focus was on the input handling in KWin. That is the part between input events enter through libinput and are sent to the Wayland client. There are many things the compositor needs to consider for input events: updating the window which has focus, ensuring while the screen is locked to not pass events to normal windows, handling focus follows mouse, etc. etc. On X11 KWin has already code for most of these things, but the code is quite dependent on X11, so it needed to be partially adjusted and partially rewritten.

The code we had in KWin/Wayland for input handling so far already showed it’s age. It was written and designed before KWin really became a Wayland compositor, from the time when KWin could render X11 windows to another Wayland compositor. So it mostly cared about sending the events to the X server. Everything else continued to work as the X11 event handling was still in place.

So the first task was to untangle the code so that it’s easier to extend and at the same time guarantee that it won’t break. As we are now able to start KWin/Wayland on a virtual framebuffer, we can run it during our auto tests. This was a rather important corner stone for reworking the input as it allowed to write test cases for everything KWin does.

With that done existing features from X11 could be ported to Wayland including mouse actions (what to do when clicking inactive window), unrestricted move/resize with alt+(left/right) mouse button, focus follows mouse and auto raise, etc. All those features are now also under test coverage and as the code is mostly shared with the X11 implementation we now also have test coverage for these features on X11. That’s quite an improvement for our X11 implementation thanks to Wayland.

Another area of work is keyboard layout handling. So far KWin defaulted to use the us layout without any possibility to change. That was a huge drawback for my own usage as I couldn’t even write my name. Like with many other input related areas I’m not really familiar with the technology, so I had to look into it in more detail. I am very pleased with xkbcommon, it was really easy to get this working and hooked up properly in KWin. The result is that KWin/Wayland now fully supports keyboard layout switches and also the integration with Plasma’s keyboard layout configuration module. I was rather pleased to see that the configuration module was hardly X11 dependent and just works on Wayland. With KWin listening to the correct DBus signal it allows to reconfigure layouts. But there is still work in that area. So far I have not added support for compose keys, the systemtray applet for switching layouts is not ported yet, accessibility features are still lacking. If you are interested in these areas some help is appreciated.

In case you tried Plasma 5.5 on Wayland you might have noticed that the cursor was sometimes rather incorrect. Not anymore in Plasma 5.6. The cursor image handling got also redesigned and put under test coverage (unfortunately our CI system doesn’t like them yet, locally they pass). But having KWin handle cursors correctly is unfortunately not sufficient to have proper cursor images. Cursor images are set by the clients and if they set an incorrect cursor image, KWin cannot do anything about it. For example QtWayland doesn’t support animated cursors and doesn’t support custom cursors. With the feature freeze for Plasma 5.6 behind us I’m also looking into these issues now. Overall client bugs make it really hard to test features as you never know whether it’s a bug in your application or in the client (or XWayland). The fact that GTK+ and wayland-demos just crash, because KWin doesn’t support xdg-shell yet, doesn’t make it easier to test new features.

The last input area I looked at and landed just in time for feature freeze is drag’n’drop. The implementation is not yet using the updated protocol from Wayland 1.10 as we had already passed dependency freeze when I started to implement it.

Overall the improved input handling gives us a nice feature set for Plasma 5.6 on Wayland. On a single screen setup it’s quite useable already. Of course there are bugs and those need you. Please give it a try and report all bugs you see.

In the area of input there is also more work to be done. We need support for graphic tablets. If you are interested in it: I’m willing to mentor a GSoC project about it. We have a few GSoC ideas for Wayland, check them out!

So what’s next in Wayland world? Till the release of Plasma 5.6 I want to concentrate on bug fixes to make the experience better. Than there’s xdg-shell quite high on my priority list and making multi-screen work correctly (that’s blocking me to switch my main system, the single screen notebook is mostly used on Wayland nowadays).

Server-Side decorations coming to KWin/Wayland

As a kind of Christmas present to our Wayland users I’m happy to announce that over the last two weeks I worked on adding support for server-side decorations.

The main motivations for working on it was the fact that I want to switch to Wayland as primary driver for my system and the nested KWin running on top of another Wayland server, which I need for development, doesn’t have any decorations. Of course I could have implemented client-side decorations for it. But as my readers might know, I consider client-side decorations as an inferior solution. And KWin of course has support for server-side decorations anyway for X11 and thus it’s less work to go for server-side decoration than to go for client-side.

The second reason is that Qt’s default client-side decorations are comparable ugly and lack important features like a difference between active and inactive windows which makes using a Wayland session really hard.

In this case a possibility could have been to develop a plugin so that KDecoration based themes could be used for client-side decoration. But to get it really useable this would have required a complex protocol to get in on par with what KWin internally has.

So here’s the solution:

Server Side decoration support in KWin/Wayland
Server Side decoration support in KWin/Wayland

A core element is a protocol to negotiate whether a window should have server-, client-side or no decoration which got added to KWayland. KWin got an implementation for that both as server and client. I plan to submit the protocol for inclusion in Wayland next year. I do think that this can be a general solution: KWin won’t stay the only Wayland compositor preferring to not have client-side decorations. If we think about tiling and use cases like phones we see that client-side cannot be the ultimate solution. Thus I think it’s a useful extension. Of course it doesn’t forbid client-side decoration, that’s still possible with the protocol. So GTK+ applications build upon client-side decoration are still able to use it, but of course I highly recommend to use server-side decorations on a system that prefers server-side decorations (the protocol is also able to tell that).

The last part to get this working got implemented in our Qt Platform Theme plugin for Plasma. This plugin will move from frameworksintegration to Plasma with 5.6, so we can easily extend it and depend on KWayland. The plugin checks whether the server supports the protocol and if it does it disables Qt’s client side decorations. For each new created Wayland window it tells KWin to either use server-side decoration or no decoration (popup windows). As all of that is implemented in our platform theme plugin it means that it doesn’t affect other Wayland compositors. There the plugin does not get loaded and Qt’s client-side decorations will be used. So no fear: this won’t affect GNOME Shell at all. As the plugin is currently in the process of being moved, it’s only in a scratch repository and won’t make it to main this year. Our code deserves a Christmas break as well 🙂

Happy holidays and a successful Wayland year 2016!

Gaming on Linux: Move to next generation?

In this blog post I want to outline some thoughts on how the gaming experience can be improved on Linux.

Situation on X11

On X11 the big problem for gaming is the Compositor. Games need access to the GPU and need to pretty much exclusively use the GPU. Compare to a true console like a PlayStation: while the game is running you can be sure it’s the exclusive user of the GPU. The way how compositing works on X11 this cannot be provided. There is the compositor which needs to render the scene. The setup looks simplified like:

  1. Game renders through OpenGL/GLX
  2. X-Server notifies Compositor through XDamage extension
  3. Compositor schedules a repaint for changed area
  4. Compositor uses the XComposite extension to get a pixmap for the Game window
  5. Compositor binds pixmap to an OpenGL texture
  6. Compositor renders the texture using OpenGL/GLX to the composite overlay window
  7. X server presents the rendered image from the Compositor through kernel mode settings

In this setup we have a few things which are not optimal for a fullscreen game: the roundtrips through Xserver-compositor and the possible delay added by the compositor. In such a setup the compositor will always
slow down the game as it performs vsync, etc.

Workarounds on X11

There is an existing solution to improve this which is known as “unredirection of full-screen windows”. The idea is that if there is a full screen window the compositor won’t run for it and the “normal” non-composited functionality of the X server is used.

In my opinion this is the wrong solution to the problem, because the Compositor is still running, it still gets damage events (from possibly other windows) and might start to composite the scene at any time again (e.g. a notification pops up as a override-redirect window).

In KWin/Plasma we have a better solution: blocking compositing. We can do that as we do not require compositing, other environments which require compositing can only use unredirection as best measurement. In fact we even have high-level API for games to tell us that they want to block compositing. It’s just one X atom, please use it.

This also explains why in KWin/Plasma the option to fullscreen unredirection is not enabled by default. It has drawbacks for a non-gaming usage, while we offer a better solution for games. It also explains why I don’t care at all about gaming benchmarks through the PTS as that in my opinion tests an invalid setup for us. If we cared about it, it would be easy to just ensure that the games used by PTS disable compositing.

Situation on Wayland

On Wayland the setup is better as we don’t have X11 in between. So a setup looks like the following:

  1. Game renders through OpenGL/EGL
  2. Compositor gets notified through wl_surface damage
  3. Compositor directly presents the wl_buffer through KMS as it knows there is nothing else to see

So the situation is much improved in an ideal case. I need to point out that KWin doesn’t support this ideal case yet and still renders through OpenGL, but that’s where we are going to.

But I think there are still problems. Our Compositor still gets damage events from other windows, it gets woken up, etc. Running a game in a desktop environment means that there are other session processes which the game needs to share resources with. We want to have a PlayStation like setup: game everything, everyone else nothing. I don’t want KWin to take away important CPU/GPU time from the game.

Kernel Mode Settings in games

So what can we do? I thought about this and propose that we change gaming completely on Linux: remove the windowing system! Games should talk to kernel mode settings directly, games should interact with libinput directly. Let’s remove everything in between, we don’t need it, it only can worsen the gaming experience.

When a fullscreen game starts, it can create a “sub-session” on a new virtual terminal and become the logind- session controller for that session. This would allow the game to open the device files for rendering and for input just like a Wayland compositor does. Rendering could be done through EGL on top of DRM/GBM just like a Wayland compositor. The game would have full control over rendering, there is no desktop environment to slow it down any more. And it would have control over mode setting. Need a different resolution? No problem, just set it. On a desktop environment that’s always problematic (terrible on X11, better on Wayland). For games in windowed mode nothing should be changed, those should stay on the desktop environment.

Of course this would remove all interaction with the desktop environment. This is something which needs to be considered, like how to get Mumble to work in such a setup? Maybe the game could launch its own Wayland server?

That breaks Alt+Tab! Well not really. For one on X11 at least games often grab the keyboard, so Alt+Tab won’t work anyway. And of course one can still switch with ctrl+alt+f1 to the running session. Games should also have a common way to achieve this in my opinion.

There are certainly a few things which still need to be solved to get there, like how to start such a sub-session, how to return to the running session without needing to unlock the screen. The experience should be as good as on dedicated gaming console. So it’s still some work, but I think this is work well invested and better than trying to somehow make games work in Desktop Environments as that just cannot work as good as a dedicated gaming console.

Looking at the security of Plasma/Wayland

Our beta release of Plasma 5.5 includes a Wayland session file which allows to start in a Plasma on Wayland session directly from a login manager supporting Wayland session. This is a good point in time to look at the security of our Plasma on Wayland session.

Situation on X11

X11 is not secure and has severe conceptual issues like

  • any client connected to the X server (either remote or local) can read all input events
  • any client can get information about when another window rendered and get the content of the window
  • any client can change any X attribute of any other window
  • any window can position itself
  • many more issues

This can be used to create very interesting attacks. It’s one of the reasons why I for example think it’s a very bad idea to start the file manager as root on the same X server. I’m quite certain that if I wanted to I could exploit this relatively easily just through what X provides.

The insecurity of X11 also influenced the security design of applications running on X11. It’s pointless to think about preventing potential attacks if you could get the same by just using core X11 functionality. For example KWin’s scripting functionality allows to interact with the X11 windows. In general one could say that’s dangerous as it allows untrusted code to change aspects of the managed windows, but it’s nothing you could not get with plain X11.

Improvements on Wayland

Wayland removes the threats from the X11 world. The protocols are designed in a secure way. A client cannot in any way interact with windows from other clients. This implies:

  • reading of key events for other windows is not possible
  • getting window content of other windows is not possible

In addition the protocols do not allow to e.g. position windows themselves, or raise themselves, change stacking order and so on.

Security removed in Plasma

But lots of these security restrictions do not work for us in Plasma. Our desktop shell need to be able to get information of other windows (e.g. Tasks applet), be able to mark a panel as a panel (above other windows) and need to be able to position windows itself.

Given that we removed some of the security aspects again and introduced a few custom protocols to provide window management facilities and also to allow Plasma windows to position themselves. At the moment we have no security restrictions on that yet which gives this functionality to all clients.

We will address this in a future release. There are multiple ways how this could be addressed, e.g. using the Wayland security module library or use systemd in some way to restrict access. Overall I think it will require rethinking security on a Linux user session in general, more on that later on.

Security added in Plasma compared to X11

The most important change on the security front of the desktop session is a secure screen locker. With Plasma 5.5 on Wayland we are able to provide that and address some long standing issues from X11. The screen locks even if a context menu is open or anything else grabbing input. The compositor knows that the screen is locked and knows which window is the screen locker. This is a huge change compared to X11: the XServer has no concept of a screen locker. Our compositor can now do the right thing when the screen is locked:

  • don’t render other windows
  • ensure input events are only handled in the lock screen
  • prevent access to screen grabbing functionality while screen is locked

As a matter of fact the Wayland protocol itself doesn’t know anything about screen locking either. This is now something we added directly to KWin and doesn’t need any additional custom Wayland interfaces.

How to break the security?

Now imagine you want to write a key logger in a Plasma/Wayland world. How would you do it? I asked myself this question recently, thought about it, found a possible solution and had a key logger in less than 10 minutes: ouch.

Of course there is no way to get a client to act as a key logger. The Wayland protocol is designed in a secure way and also our protocol additions do not weaken that. So the key to get a key logger is to attack KWin.

So what can an attacker do with KWin if he owns it? Well pretty much anything. KWin internally has a very straight forward trust model: everything is trusted and everything can access anything. There is not much to do about that, this is kind of how binaries work.

For example as a Qt application each loaded plugin has access to the QCoreApplication::instance. From there one could just use Qt’s meta object inspection to e.g. get to the InputRedirection model and connect to the internal signal on each key press:

<code>void ExamplePlugin::installKeyLogger()
{
    const auto children = QCoreApplication::instance()-&gt;children();
    for (auto it = children.begin(); it != children.end(); ++it) {
        const QMetaObject *meta = (*it)-&gt;metaObject();
        if (qstrcmp(meta-&gt;className(), "KWin::InputRedirection") != 0) {
            continue;
        }
        connect(*it, SIGNAL(keyStateChanged(quint32,InputRedirection::KeyboardKeyState)), this, SLOT(keyPressed(quint32)), Qt::DirectConnection);
    }
}

void ExamplePlugin::keyPressed(quint32 key)
{
    qDebug() &lt;&lt; "!!!! Key: " &lt;&lt; key;
}
</code>

But Martin, why don’t you just remove the signal, why should any other aspect of KWin see the key events? Because this is just the example of the most trivial exploit. Of course it’s not the only one. If you have enough time and money you could write more sophisticated ones. For example look at this scenario:

KWin uses logind to open restricted files like the input event files or the DRM node. For this KWin registers as the session controller in logind. Now a binary plugin could just send a DBus call to logind to also open the input event files and read all events. Or open the DRM node and take over rendering from KWin. There is nothing logind could do about it: how should it be able to distinguish a valid from an invalid request coming from KWin?

How to secure again?

As we can see the threat is in loading plugins. So all we need to do is ensure that KWin doesn’t load any plugins from not trusted locations (that is not from any user owned locations). This is easy enough for QML plugins where we have the complete control. In fact it’s easy to ensure for any of KWin’s own plugins. We can restrict the location of all of them.

And even more: by default a system is setup in a way that no binary plugins are loaded from user’s home. So yeah, no problem after all? Well, unfortunately not. During session startup various scripts are sourced which can override the environment variables to influence the loading of plugins. And this allows to also use the well known LD_PRELOAD hack. My naive approach to circumvent this issue didn’t work out at all as I had to learn that already the session startup and the PAM interaction source scripts. So your session might be owned very early.

An approach to black list (unset) env variables is futile. There are too many libraries KWin relies on which in turn load plugins through custom env variables. Most obvious examples are Qt and Mesa. But there are probably many more. If we forget to unset one variable the protection is broken.

A different approach would be to white list some known secure env variables to be passed to KWin. But this also requires that at the point where we want to do the restriction the session is not already completely broken. This in turn means that neither PAM nor the session manager may load any variables into the session before starting the session startup. And that’s unfortunately outside what we can do in our session startup.

So for Plasma 5.5 I think there is nothing we can do to get this secure, which is fine given that the Wayland session is still under development. For Plasma 5.6 we need to rethink the approach completely and that might involve changing the startup overall. We need to have a secure and controlled startup process. Only once KWin is started we can think about sourcing env variables from user locations.

So how big is the threat? By default it’s of course secure. Only if there is already a malicious program running in the system there is a chance of installing a key logger in this way. If one is able to exploit e.g. a browser in a way that it can store an env variable script in one of the locations, you are owned. Or if someone is able to get physical access to your unencrypted hard disk, there is a threat. There are easy workarounds for a user: make all locations from where scripts are sourced during session startup non-writable and non-executable, best change ownership to root and encrypt your home location.

Overall it means that Plasma 5.5 on Wayland is not yet able to provide the security I would have liked to have, but it’s still a huge improvement over X11. And I’m quite certain that we will be able to solve this.

October Plasma on Wayland Update: all about geometry

Last month our Wayland efforts made a huge step forward. In KWin we are now at a state where I think the big underlying work is finished, we entered the finishing line of the KWin Wayland porting. The whole system though still needs a little bit more work.

The big remaining task which I worked on last month was geometry handling. That is simplified: moving and resizing windows. Sounds relatively easy, but isn’t. Moving and resizing windows or in general the geometry handling is one of the core aspects of a window manager. It’s where our expertise is, the code which makes KWin such a good window manager. Naturally we don’t want to throw that code out and want to reuse it in a Wayland world.

And that meant: refactoring. The big problem here was that the code in question is also highly X11 specific. Examples are: the sync protocol for synced resizing, window gravity handling, various X11 specific quirks for moving X11 window moving smooth (e.g. only update at end of move if compositing is active, direct updates if not active).

The task was now to separate the X11 specific geometry handling from generic geometry handling so that it can support both Wayland and X11 windows. With this now in place all the geometry handling works also for Wayland windows. This includes move/resize through the alt+f3 menu, but also triggered from the window itself. During move/resize the quick tiling areas are of course triggered correctly, show the outline and snap to the area if released. Resizing is automatically synced to the speed the client supports and that also in a much better way than on X11 (yay for Wayland and double-buffered state!). During move/resize the windows snap to screen borders and other windows.

Unfortunately our checks to ensure that the window titlebar is not moved outside the visible area doesn’t work, because of client side decorations. Yay, everything is awesome! Have I mentioned that client side decorations are a stupid idea because it breaks useful stuff? No, cannot be. Related to that: QtWayland enters move mode if you click the title bar to activate the window. Of course there should be a delay which KWin implements for it’s decorations. After all we have more than 15 years of experience in doing window decorations. Yay, client side decos! Given control to the application! Let’s leave them in charge of window management, what could go wrong? Wohoo! Totally awesome! Broken window management! That’s the way to go! Of course that could be fixed in Qt, but well it doesn’t fix the problem. We have seen this in Chromium years ago, we even had to adjust our quick tiling/maximization behavior because of it. Another fun fact: QtWayland deco has a minimize button which does nothing because it’s not supported in wl_shell protocol. Have I mentioned that client side decos are awesome?

All right, all right, I’ll end the sarcasm mode now. Our geometry handling has a few more very handy features like window packing: you can configure a shortcut to move the window left (or other direction) and it will move to the next window or the screen border. Similar we support growing or shrinking the window to the next window. I was hardly aware of that feature before doing the refactoring, so I thought I should point it out. It can be used from scripting, so should be useful for ideas like poor man tiling or resizing multiple quick tiled windows at the same time. As a nice addition this code is now covered by auto tests.

So with geometry handling in place it’s possible to do real testing and one of my systems (my notebook) migrated from X11 to Wayland. Actually I have been using Wayland on that system already for watching videos since April as it gives tear-free rendering. A clear plus when watching videos.

And this gives us already the outlook for what we will see in the November update: I’ll focus on stabilizing the current state of Plasma/Wayland and fix bugs, fix bugs and fix bugs. My aim is to have a useable early-adopters version ready for the Plasma 5.5 release. It’s looking good, so I’m confident that we will reach that state.

Upgrading libhybris

One of the most important dependencies for our phone project is libhybris. Libhybris is a neat technology to allow interfacing with Android drivers allowing for example to bring Wayland to a device where all we have are Android drivers.

Given that KWin provides a hwcomposer backend which uses libhybris to create an OpenGL context. All other applications need libhybris indirectly to have the Wayland OpenGL buffer exchange work automatically.

When we started the work on the hwcomposer backend we based it on the libhybris version used in Ubuntu (0.1.0+git20131207) as we used Ubuntu as the reference platform. Soon enough we noticed that this version diverged a lot from the upstream version. Lots of recent changes are missing and there are API incompatible changes.

This made working with it difficult. How much time should we invest in investigating issues? Should we write code which we know might break once Ubuntu decides to upgrade libhybris? How well is Wayland integrated in the Ubuntu version given that they don’t need it? If we need help, who to talk to? Ubuntu who will tell us that they don’t know anything about Wayland, or the libhybris devs who might just tell us: use later version?

Furthermore we want other distributions to provide Plasma for the phone. This means they need to provide libhybris. Of course this is difficult if we need to tell them that we need exactly the version used by Ubuntu. And even more it might conflict with other uses. Considering distributions like Mer would have to chose between a libhybris for Plasma and a libhybris for lipstick.

With that in mind we wanted to invest some time on upgrading libhybris in our stack in this release cycle and then fix the issues we were seeing in the stack. Our awesome packagers did the job of creating packages so that I can port KWin against it. And in deed after some hacking I had KWin rendering again. A more difficult task was to get other applications to work as we run into a problem that libwayland-egl does not use the alternatives system. Thus our packagers needed to do some ld tricks to get this worked around. But with that we had a nice rendering system.

A surprise in this exercise was that our input handling code in the hwcomposer backend didn’t compile any more. The code was gone. While that was in the first moment an unpleasant surprise, it soon turned into something wonderful. If that code is not needed at all on an Android powered device it means that we must be able to get libinput to work with it. 400 lines of code deleted and it’s using the shared input stack through libinput. I’m very happy about that!

With that all in place we finally were able to investigate the rendering issues we were seeing. My hope was that just upgrading libhybris would fix the visual tearing, but unfortunately not. While I’m still surprised that it’s possible to get tearing in the first place on Android devices (hey ever heard of things like Atomic Mode Settings, Android?), it at least gives us a vsync event. Unfortunately the only tear-free solution I could find invokes blocking till we get the event. I don’t like that and I think that’s a bad architecture. One can have blocking free and tear free rendering. Our DRM (kernel mode setting) backend can do so with an easy to use API. Really disappointing that the Android stack is in that regard not better than the glx backend. But well at least it’s tear free 🙂

As we now use upstream libhybris I hope to see distributions to pick up the work and provide a Plasma phone spin. I’d love to see an openSUSE phone or a Fedora phone (or any other distribution). Distributions: you can of course ask us on how to integrate 🙂