To EGLStream or not

The announcement of KDE Neon dev/unstable switching to Wayland by default raised quite a few worried comments as NVIDIA’s proprietary driver is not supported. One thing should be clear: we won’t break any setups. We will make sure that X11 is selected by default if the given hardware setup does not support Wayland. Nevertheless I think that the amount of questions show that I should discuss this in more detail.

NVIDIA does support Wayland – kind-of. The solution they came up with is not compatible to any existing Wayland compositor and requires patches to make it work. For the reference implementation Weston there are patches provided by NVIDIA, but those have not been integrated yet. For KWin such patches do not exist and we have no plans to develop such an adaption as long as the patches are not merged into Weston. Even if there would be patches, we would not merge them as long as they are not merged into Weston.

The solution NVIDIA came up with requires different code paths. This is unfortunate as it would require driver specific adjustments and driver specific code paths. This is bad for everybody involved. For us developers, for the driver developers and most importantly for our users. It means that we developers have to spend time on implementing and maintaining a solution for one driver – time which could be spent on fixing bugs instead. We could do such an effort for one driver, but once it goes to every driver requiring adjustment it gets not manageable.

But also adjustments for one driver are problematic. The latest NVIDIA driver caused a regression in KWin. On Quadro hardware (other hardware seems to be not affected) our shader self test fails which results in compositing disabled. If one removes the shader self test everything works fine, though. I assume that there is a bug in KWin’s rendering of the self test which is triggered only with this driver. But as I don’t have such hardware I cannot verify. Yes, I did pass multiple patches for investigating and trying to fix it to a colleague with such hardware. No, please don’t donate me hardware.

In the end, after spending more than half a day on it, we had to do the worst option which is to add a driver and hardware specific check to disable the self test and ship it with the 5.7.5 release. It’s super problematic for the code maintainability to add such checks. We are hiding a bug and we cannot investigate it. We are now stuck with an implementation where we will never be able to say “we can remove that again”. Driver specific workarounds tend to stick around. E.g. we have such a check:

// Broken on Intel chips with Mesa 9.1 - BUG 313613
if (gl->driver() == Driver_Intel && gl->mesaVersion() >= kVersionNumber(9, 1) && gl->mesaVersion() < kVersionNumber(9, 2))
    return;

It's nowadays absolutely pointless to have such code around as nobody is using such a Mesa version. But the code is still there, makes it more complex and has a maintenance cost. This is why driver specific implementations are bad and is nothing we want in our code base.

People asked to be pragmatic, because NVIDIA is so important. I am absolutely pragmatic here: we don't have the resources to develop and maintain an NVIDIA specific implementation on Wayland.

Also some people complained that this is unfair because we do have an implementation for (proprietary) Android drivers. I need to point out that this does not compare at all.

First of all our Android implementation is not specific for a proprietary driver. It is written for the open source hwcomposer interface exposed through libhybris. All of that is open source. The fact that the actual driver might be proprietary is nothing we like, but also not relevant for our implementation.

In addition the implementation is encapsulated in a platform plugin and significantly reduced in functionality (only one screen, no cursor, etc.). This is something we would not be able to do for NVIDIA (you would want multi-screen, right?).

For NVIDIA we would have to add a deviation into the DRM platform plugin to create the OpenGL context in a different way. This is something our architecture does not support and was not created for. The general idea is that if creating the GBM based context fails, KWin will terminate. Adding support there for a different way to get an OpenGL context up and running would result in lots of added complexity in a very important code path. We have to ensure that KWin terminates if OpenGL fails. At the same time we have to make sure that llvmpipe is not picked if NVIDIA hardware is used. This would be a horrible mess to maintain - especially if developers are not able to test this without huge effort.

From what I understand from the patch set it would also require to significantly change the presenting of a frame on an output and by that also turn our lower level code more complex. This code is currently able to serve both our OpenGL and our QPainter based compositor, but wouldn't allow to support NVIDIA's implementation. Adding changes there would hinder us in future development of the platform plugin. This is an important area we are working on and KWin 5.8 contains a new additional implementation making use of atomic mode settings. We want to have atomic mode settings used everywhere in the stack to have every frame perfect. NVIDIA's implementation would make that difficult.

EGLStreams bring another disadvantage as the code to bind a buffer (what a window renders) to a texture (what the compositor needs to render) would need changes. Binding the buffer is currently performed by KWin core and not part of the plugin infrastructure. Given that new additional code would also be needed there. We don't need that for any other platform we currently support. E.g. for hwcomposer on Andrid libhybris takes care of allowing us to use EGL the same way as on any other platform. I absolutely do not understand why changes would be needed there. Existing code shows that it can be done differently. And here we see again why I think the situation with EGLStream does not compare at all to supporting hwcomposer.

Overall we are not thrilled by the prospect of two competing implementations. We do hope that at XDC the discussions will have a positive end and that there will be only one implementation. I don't care which one, I don't care whether one is better as the other. What I care about is only requiring one code path, the possibility to test with free drivers (Mesa) and the support for atomic mode settings. Ideally I would also prefer to not have to adjust existing code.

A circle closes…

At Desktop Summit 2011 in Berlin I did my first presentation on Wayland and presented the idea of Wayland to the KDE community and explained to the KDE community how we are going to port to Wayland. This year at QtCon in Berlin I was finally able to tell the KDE community that the port is finished and that our code is ready for testing.

In 2011 I used a half hour slot to mostly present the differences between X11 and Wayland and why we want Wayland. In addition I presented some of the to be expected porting steps and what we will have in the end. This year I only used a 10 min lightning talk slot to give the community an update on the work done the last year.

(Watch video on youtube, my talk starts at 15:04)

Of course the work on Wayland is not yet finished and Wayland is not yet fully ready for use. There are missing features and there must be bugs (new code base, etc.). But we are in a state to start the public beta.

What is interesting is comparing the slides from 2011 to what we have achieved. The plan presented there is to introduce “Window Manager Backends” in KWin. We wanted to identify windowing system independent areas and make our two most important classes Toplevel and Workspace X11 free and add a window manager abstraction. During the port this wasn’t really an aim, nevertheless we got there. We do have a window manager abstraction which would allow to add support for further windowing systems. Toplevel is (at runtime) X free. Workspace, though, is not yet X free, but that moved on my todo list.

Also we thoughts back in 2011 that this might be interesting for other platforms naming Android, WebOS and Microsoft Windows as examples. Android we kind of achieved by having support for Android’s hwcomposer and being able to run Wayland on top of an Android stack. Support for Android’s surfaceflinger is something we do not aim for. The example of WebOS doesn’t really fit any more as WebOS uses Wayland nowadays. And Windows is only in the area of theoretically possible (though with the new Linux support it would be interesting to try to get KWin running on it).

KWin nowadays has a platform abstraction and multiple platform plugins. This allows us to start a Wayland compositor on various software stacks. Currently we support:

  • DRM
  • fbdev
  • hwcomposer (through libhybris)
  • Wayland (nested)
  • X11 (nested)
  • virtual

Adding support for a new platform is quite straight forward and doesn’t need a lot of code. The main tasks of a Platform is to create the OpenGL context for the compositor and to present each frame on the Platform specific output. All platforms together are less than 10000 lines of code (cloc) and a single platform is around 400-3000 lines of code.

In order to add support for a new windowing system more work would be needed. It is very difficult to estimate how much code would be needed as it all depends on how well the concept can be mapped to Wayland. Ideally adding support for a new windowing system would be done by creating an external application which maps the windowing system to Wayland. Just like XWayland maps X11 to Wayland. But as we can see with XWayland this might not be enough. KWin also needs to be an X11 window manager to fully support X11 applications. Given that it really depends on the windowing system how much work is needed.

One could also add a new windowing system the same way as we added support for Wayland. This would require to implement our AbstractClient to have a representation for a managed window of the windowing system and add support for creating a texture from the window content. In addition various places in KWin need to be adjusted to also consider these windows. Not a trivial task and going through a mapping to Wayland is always the better solution. But still it’s possible and this makes KWin future proof for possible other windowing systems. In general KWin doesn’t care any more about the windowing system of a window. We can have X11 windows on Wayland and Wayland windows on X11 (only experimental branch, not yet merged).

This brings me back to my presentation from 2011. Back then we expected to have three phases of development. The first phase adding Wayland support to the existing X11 base. That was what we experimented with back then and as I just wrote still experiment with it. As it turned out that was not the proper approach for development.

As a second phase we expected to remove X and have a Wayland only system. At the moment we still require XWayland to start KWin/Wayland. During the development it showed that this is not something really needed. It was easier to move the existing X11 code to interact through XWayland – we could keep the X code and move faster.

wayland-architecture-rootless-x.png

The third and final phase was about adding back XWayland support, so that KWin can support both X11 and Wayland windows. That’s the phase we developed directly. Which is kind of interesting that we went to the final step although we thought we need easier intermediate steps.

KDE Neon dev/unstable switching to Wayland by default

During this year’s Akademy we had a few discussions about Wayland, and the Plasma and Neon team decided to switch Neon developer unstable edition to Wayland by default soonish.

There are still a few things in the stack which need to be shaken out – we need a newer Xwayland in Neon, we want to wait for Plasma 5.8 to be released, we need to get the latest QtWayland 5.7 build, etc. etc.

This is really exciting. It’s probably the biggest step towards Wayland by default the KDE community has ever taken. I hope that other continuous delivery systems will follow so that we can get many enthusiastic users to try Wayland.