Plasma Wayland and Qt 5.9 and beyond

As you might know Qt 5.8 created challenging problems for our Wayland session and threw our efforts back quite a bit. In this post I want to discuss the actual problems it created, how we are addressing them and looking into the future.

How our integration used to work

Our integration uses additional Wayland protocols. We have a protocol for server side window decorations which we use in our Plasma integration plugin to inform KWin whether the window should have a decoration or not. We have a protocol for client provided shadows which is e.g. used by our widget style Breeze to add shadows to the context menus. We have a protocol for the desktop shell, so that it can mark windows as desktop, panel, auto hiding panel, position the window, etc. Also we have a few protocols for interacting with our effect system, e.g. sliding popups, blur behind.

To use these protocols we need to interact with Qt in a low level way. We use the native interface in the Qt Platform Abstraction to get a wl_surface pointer for the QWindow. In order to not have to keep this simple for our applications our KWayland::Client API provides an API point for it: Surface::fromWindow(QWindow*) -> Surface*.

But when exactly to inject our own integration? We found a very handy way which worked much better than what we had used in the past for X11 (and based on that also transitioned X11 code to use it). Qt emits an event once it has created the native platform surface (in case of Wayland the wl_surface*) for a QWindow. Verbatim quote of the documentation:

The QPlatformSurfaceEvent class is used to notify about native platform surface events.

Platform window events are synchronously sent to windows and offscreen surfaces when their underlying native surfaces are created or are about to be destroyed.

Applications can respond to these events to know when the underlying platform surface exists.

Awesome! We get an event when the surface is created and when it gets destroyed. This made it very simple to create the integration and what is really important for us is that we get this event before the window is shown. So we can prepare everything so that KWin gets a good state.

And the KWin side was to a large part implemented on assumptions on how the sequence will work. We first get the surface, then the (xdg) shell surface, then the integration bits. Sure it would be nice if KWin handled also other sequences, but as the only implementation of this is Qt it doesn’t make sense to really care about it. We know that it was not perfect, we even had the test cases for it, which expect failed.

What broke with Qt 5.8?

In Qt 5.8 our complete integration broke. When the platform surface created event was emitted the wl_surface was not created, see QTBUG 58423. This is in my humble opinion a clear violation of the documented behavior and thus a breakage of the stability guarantees Qt provides, but others might disagree. After some discussion, trying out patches by those who had a Qt 5.8 build we had a patch for Qt which made things mostly work together with a patch to KWin. But at the time we had the patch ready Qt 5.8 was already declared end of life with no prospect of a Qt 5.8.1 bug fix release. For our Wayland session it was just impossible to get Qt 5.8 compatible again. All we could do was to advise distributions to not combine Qt 5.8 with the Wayland session. From our side it was fine for distros to ship Qt 5.8, but if they do they should make it impossible to install the Wayland session. The state was just too broken.

Qt 5.9

With Qt 5.9 the situation looks better. The required patch is merged, it’s an LTS release and we already had the first bug fix release. Qt wants to create more bug fix releases for it and this allows us to use it as a new target for integration. But still the situation is not as good as it used to be. If you currently use our Wayland session with Qt 5.9 you will still see quite some rough corners compared to where we were with Qt 5.7.

The main problem for us is that the platform surface regression was not the only change affecting us. Pre Qt 5.7 a wl_surface lived as long as a QWindow. Now the wl_surface gets destroyed whenever the window gets hidden and a new one created on every show. Unfortunately without a platform surface created event. This means our integration breaks as soon as a window gets hidden. E.g. after closing KRunner the integration for KRunner is broken.

We tried to address this problem in various places, but it is challenging. On the show event we don’t have the wl_surface yet (too early), on the expose event the window is already mapped (too late). This creates problems for KWin which is not prepared for the integration bits to hit when the window is mapped. Our protocols were designed with the platform surface created event semantics. For example in KRunner we face an issue with the integration. KRunner is a panel, which accepts focus and allows windows to go below, also it positions manually. Now when it gets re-shown KWin doesn’t know that this window is supposed to position itself and positions it. Now we get the request to put it as panel, KWin adjusts and moves maximized windows around. And then we get the request that the panel allows windows to go below. KWin shuffles the windows again as the maximized area changed again.

This is a rather tricky situation as we cannot really do something about it. If the window is mapped it’s too late. Even if we improve our API to handle the situation better it will be too late.

There are two possibilities to handle this: Qt stops to destroy the surface or sends a platform surface created event when it recreates the surface. The latter would be my personally preferred solution as this would match the documentation again and allow us to just use the one event handler.

Other regression

The situation around the changed behavior in Qt 5.8 caused a few steps back. Our code needed to be adjusted and that sometimes caused issues. We had a few regressions which also affected the compositor, so the stability of the whole system suffered. These issues are luckily investigated and fixed. But there are still bugs lurking in the system.

For me personally the most annoying bug is a crash in Qt which affects the auto completion of kdevelop. This makes hacking in a Wayland session rather difficult. I’m running currently a patched KWin which disables the virtual keyboard integration to not hit this issue.

A huge problem is that context menus are not marked as transient windows. This means that the Wayland compositor does not know that it is a menu and positions the menu anywhere on the screen. It gives the system a very unfinished touch.

If KRunner is closed through the escape key, the key starts to repeat on the window constantly and due to that it is not possible to open KRunner again. Similar if you start an application in Kickoff through the enter key, when opening Kickoff again it automatically launches the currently selected item. This again makes it very difficult to use the session and gives the whole system an unfinished look. We are working on a workaround for this issue in the server.

Towards the future

Qt 5.9 is here to stay and that’s what we have to use as integration target. Given that Qt 5.9 and Qt 5.7 behave very differently it will become difficult for us to maintain support for both. My suggestion is that we drop support for Qt 5.7 and require Qt 5.9 for the Wayland session. In addition there is hope that we can improve the integration. Marco and David have been working on adding support for XDG Shell unstable v6 which is already supported by Qt and makes it easier to integrate with. Once this landed in KWin Qt will be switched from wl_shell to xdg shell. This will improve the situation for us quite a bit as we then have one code path for both Qt and GTK applications.

How to prevent such regressions in the future

The change of behavior in Qt 5.8 threw our Wayland efforts back a few months. This is something we communicated to Qt quite early and it’s something which worries me a lot. We cannot spend time on changing our integration every time Qt releases a new version. Given that we need to look into how to prevent that such a situation happens again.

I hope that we can improve the integration on the testing front between Qt and KDE. We have a huge test suite which can find regressions in Qt. If Qt would run KDE’s tests during the integration phase Qt would notice regressions before they hit the code base. Given that all our tests are free software it should be possible for Qt to integrate them.

But also the other side would be interesting: if we could get the latest Qt into our CI system we could also discover breakages early. We have now a new docker based CI system which allows running multiple builds of the same change (e.g. Plasma gets build on openSUSE tumbleweed and on FreeBSD) – an image with a daily or weekly Qt snapshot could help us and Qt a lot to detect breakages early.

I also hope in openQA which allows to test the full operating system. This would spot regressions like the misplaced context menus even if KWin’s own auto tests would not spot them (KWin doesn’t care about Qt there, only about the Wayland and X11 protocols). There we might need to invest some work to make sure that KWin/Wayland can be properly run in the openQA tests.

I hope that our Plasma devs can discuss this in more detail with Qt devs during Akademy in person. Unfortunately I cannot be at Akademy this year 🙁 so I cannot discuss in person.

Last but not least it is important that developers test. It would help a lot if the developers working on QtWayland test their changes in a running Plasma Wayland session. We are now overall in a state that the session is suited for hacking on. I do all my Wayland hacking in a Wayland session, experiencing all the glitches like kdevelop crashing.

Of course you might wonder what about us? Shouldn’t we KDE devs also test against the latest Qt? For me personally that is not the case. I’m working on the server side and not on the client side. I’m also not testing the latest GTK for example. Nevertheless I tried to use Qt 5.9 before it got released. Used the installer, spent a day to compile everything on top of it it, just to notice that it doesn’t have QtWayland and won’t get it. I didn’t give up that easily. So I tried to compile QtWayland myself. But when I tried to use it, it turned out to not have any keyboard support, because qtbase was compiled without support for xbkcommon. At that point I gave up. Not having QtWayland is one thing, but not being able to use keyboard is another, it’s rather pointless. The only other option is to compile Qt, but that is hardly an option as it’s really difficult to compile an actually working Qt with all components. The last times I tried, I failed, wasting days compiling. If there were usable weekly images for Qt I would be happy to try it, but of course only with a properly compiled and included QtWayland.

30. July 2015

Porting Qt applications to Wayland

During Akademy I hold a session about porting applications to Wayland. I collected some of the general problems I saw in various KDE projects and want to highlight them in this blog post, too.

Starting Wayland

First the basics: how to start a Wayland compositor? Assuming one is using Plasma 5.4 (aka master) the way to start a nested KWin is:

kwin_wayland --xwayland

Important: ensure that you have Xwayland installed!

If you run nested this will open a window, if it turns black, everything works. If it doesn’t turn black: contact me (please check whether you use free drivers first).

As an alternative you can also start Weston:

weston

If you run into issues with your application running under Wayland, please always test whether the problem is also visible on Weston before blaming me 😛

Starting applications on Wayland

Now we have our black KWin window and we want to get a window to it. There are several ways to start it. As an example I’ll use kate.

Environment variable

Maybe the easiest way is to just use an environment variable to switch the QPA platform to Wayland:

export QT_QPA_PLATFORM=wayland
kate

And a Kate window will open on the nested KWin.

The environment variable is set to wayland in our startplasmacompositor script to start a full Plasma on Wayland session on tty.

Command line switch

Every Qt application also has a command line switch to select the windowing system:

kate --platform wayland

Starting applications together with KWin

KWin supports starting applications once the Wayland and Xwayland servers are started. You can add the programs to start to the command which starts KWin:

kwin_wayland --xwayland "kate --platform wayland"

Or to simplify with environment variables:

export QT_QPA_PLATFORM=wayland
kwin_wayland --xwayland kate

Of course it’s also possible to just start a Konsole and then launch applications from inside the Konsole. Please note that any key combination with ctrl-key is unfortunately broken when running Konsole as a Wayland application.

X11 Specific code

Many applications have X11 specific code. Being it implicitly or explicitly. So we need to move away from it and in most cases it means also improving the situation on X11 because the solution is wrong.

Opening X connection

A common mistake I saw in Qt applications is to open a connection to the X server. E.g. like that:

Display *dpy = XOpenDisplay(NULL);

Now this code is already wrong on X11 as it opens another connection which is not the Qt display connection. It might be on the wrong head, it might be on the wrong X server. From an integration point of view it’s broken.

So the solution is to always use QX11Info from QtX11Extras to integrate with the Display:

Display *dpy = QX11Info::display();

Runtime checks in addition to compile time checks

Before Qt5 the way to support multiple windowing systems was to use compile time switches, like:

#if HAVE_X11
#include <QX11Info>

void doSomethingWithX11()
{
    XFlush(QX11Info::display());
}
#endif

Now this will crash on Wayland. It will be compiled in, but crashes when running on Wayland. So you need to improve. Let’s try:

#if HAVE_X11
#include <QX11Info>

void doSomethingWithX11()
{
    Display *dpy = QX11Info::display();
    if (!dpy) {
        return;
    }
    XFlush(dpy);
}
#endif

Now this code will also crash! The problem is that QX11Info::display() goes through Qt’s native interface and uses the key “display”. QtWayland also provides the key “display” and returns a wl_display for it, which gets casted into a void in the native interface and then to Display* in QX11Info and boom at interesting places.

The proper way is to check the platform name:

#if HAVE_X11
#include <QX11Info>

void doSomethingWithX11()
{
    if (!QX11Info::isPlatformX11()) {
        return;
    }
    XFlush(QX11Info::display());
}
#endif

My recommendation is to keep the check in a local variable if the check is needed multiple times as in the end it’s a string comparison each time. Ideally you should restructure your code to have platform specific subclasses or similar refactored solutions.

General Changes with Wayland

Now here it becomes interesting: runtime changes. The most important information for application developers is probably that there are no more global window ids.

No global window ids

This has quite some implications for your application. E.g. QWindow::winId() returns an internal incremented number. Neither the compositor nor any other application knows anything about your internal window id counter. Similarly QWindow::fromWinId is not able to map a window id to a QWindow any more. This of course affects use cases like sending a window id through DBus to another process to use it for setting the transient hint. Also it means that KWindowSystem API has a hard time to support the calls on Wayland and in most cases it will just print a warning (this might improve, though).

Setting window icon

On X11 the applications exported the window icon as pixmap data. This is no longer possible on Wayland. Instead the compositor maps the window icon from the desktop file and the application announces the desktop file for a window. In Qt this is done by using the reverse domain name combined with the binary name. The domain name is set with KAboutData and defaults to “kde.org”. So for example in the case of kate the desktop file should be called “org.kde.kate.desktop”. So please fix the naming of your desktop files. If you want to test please be aware that this requires Qt 5.5 to work properly.

No longer working API

Quite some of the API calls of QWindow cannot be supported by Wayland on default (some of them might work on KWin, but please don’t implement against KWin!).
A short list of those API calls I know that they cannot work:

QWindow::setPosition
QWindow::setMouseGrabEnabled
QWindow::setKeyboardGrabEnabled
QWindow::alert
QWindow::lower
QWindow::raise
QWindow::requestActivate

In addition it’s no longer possible to grab the image of the screen using e.g. QScreen::grabWindow (most obvious for the case of “root” window). Also warping the pointer using QCursor::setPos is no longer supported. Please note that warping the pointer is also a bad idea on X11 and you shouldn’t do that.

In case your application set the Qt::BypassWindowManagerHint on your QWindow/QWidget you need to do some porting as QtWayland doesn’t show such windows. Unfortunately that needs a case by case evaluation of the use case and the solution I presented during my Akademy talk should not be applied for applications.

Porting to Wayland

Don’t port

The solution to port to Wayland is: DON’T PORT! Mostly your X11 specific code or behavior is wrong and could be solved in better ways. Instead of porting you might want to rethink your problem or change to existing abstracted API in e.g. Qt or KWindowSystem. I have seen many cases where X11 API was used as a workaround by e.g. raising windows, force activating them and so on. If one things about it one realizes that there seems to be an obvious problem. So don’t try to port that but rethink your problem and improve not just on Wayland, but also on X11.

But I really need platform specific code

Ok, ok, I get it! Now let’s start with the obvious: your code needs to be compile time and runtime checked. Let’s start with how to not do it:

#if HAVE_X11
    if (!QX11Info::isPlatformX11()) {
         return;
    }
    // here goes the X11 specific code
#else
    // here goes the Wayland specific code
#endif

The problem with this code snippet is that X11 support is probably also available when you use Wayland. Also the else part is not just Wayland, but also any other platform.

As I said the proper solution is to make it compile and runtime checked:

#if HAVE_X11
    if (QX11Info::isPlatformX11()) {
        callX11SpecificImplementation();
    }
#endif
#if HAVE_WAYLAND
    if (QGuiApplication::platformName().startsWith(
            QLatin1String("wayland"), Qt::CaseInsensitive)) {
        callWaylandSpecificImplementation();
    }
#endif

Be aware that your code might also run on other platforms on Wayland, e.g. eglfs or other custom windowing systems. So don’t crash if you don’t hit any of the code paths 😉