Tag Archives: Wayland

Libinput integration in KWin/Wayland

Today I pushed my outstanding branch to get libinput support into kwin_wayland. Libinput is a very important part for the work to get a full Wayland session in Plasma which means we reached a very important milestone. As the name suggests it allows us to process input events directly. KWin needs to forward the input events to the currently active application(s) and also interpret them before any other application gets them. E.g. if there is a global shortcut KWin should intercept it and not send it to an application.

Why libinput integration in KWin?

KWin/Wayland already supported input handling by being a Wayland client and connecting to a Seat. But especially for pointer events this was not sufficient at all. We have quite some code where we warp the pointer and Wayland doesn’t support this (and shouldn’t). Warping the pointer is normally considered evil as it can introduce quite some problems if applications are allowed to warp the pointer. E.g. it can create security issues if you start typing your password and a malicious applications warps the pointer to trick you entering your password into a password field of the malicious application. Also from a usabililty perspective it can be problematic as it makes the system behave in an unpredictable way.

On the other hand if the application is a window manager/compositor the need for warping cursors arises. For example the screen edge handling pushes the cursor slightly back which needs cursor warping. Or you can move a window with the cursor keys (hold Control key to have very precise moving) and in these cases we need to warp the pointer. With libinput this is possible again as KWin gets put in control of the input events directly. It completely bypasses the currently used Wayland compositor.

Libinput is also an important piece in the puzzle for a full Wayland session which does not rely on another Wayland compositor. So far KWin/Wayland can only be used in a nested scenario – which is important for development and interesting new possibilities like the idea for a SoK project – but we also want full support without the need for a Wayland session. This means we need to handle input (which libinput does) and need to interact with DRM directly. DRM support is still missing. This could be an interesting GSoC project next year ;-)

The merged implementation does not support all of libinput yet. Most important touch screen support is omitted as I don’t have a touch enabled device. I plan to sit down with fellow KDE developers who have a touchscreen enabled device and implement that part together. Also I will contact the VDG to define some global touch gestures to interact with the system (I’m quite interested in having a touch gesture to activate Present Windows). There’s lots of work to be done here and I would welcome any helping hand.

Security challenges

Processing input events directly comes with a slight problem, though: one needs to be root to read the events. And that’s obviously an absolute no-go for KWin. KWin may never ever be executed with root privileges and also not an suid which drops privileges again (which also wouldn’t help in that case but so what). The application has not been designed for running as root. The same is the case for Weston and obviously I looked at how it’s solved there and there is a very neat solution to support the use case we have in logind. The session controller can ask logind to open devices and logind provides a file descriptor to the opened device. In addition logind automatically takes care to close the file descriptors when a VT switch happens, which is extremely convenient for the use cases of Wayland compositors. So obviously I went for this solution as all it needs is connecting to very few D-Bus calls. This means the libinput integration in kwin_wayland will have a runtime dependency to a logind D-Bus interface. Of course this does not affect kwin_x11, neither does it affect kwin_wayland without libinput integration, but of course without libinput you won’t get support for all features. There is one caveat though: logind will blank the VT when the session controller goes away. So make sure to not run kwin_wayland with libinput support on your primary X session. Because of that libinput support must be explicitly enabled with the –libinput command line switch of kwin_wayland.

Current state and downsides of libinput and logind

As libinput does not yet have a stable release, the dependency is still optional and it’s possible to build kwin_wayland without libinput support. This is currently very important for the non-Linux operating systems, which might want to provide kwin_wayland, as libinput only supports Linux.

I hope that libinput will become available on other platforms. At XDC during the BSD presentations I heard at least one presenter touch the topic. So I’m optimistic that in the long run this will happen as we also see that DRM and KMS is nowadays in quite a good shape on the BSDs. For KWin development it’s of course important that we have only one library to interact with. Otherwise it means platform dependent code which is hard to develop and extremely difficult to test for the main developers not using such a platform. So if you want to get kwin_wayland on non-Linux, please consider putting the energy into getting libinput working (challenge is udev) as that will help all systems and not just KWin. After all we want to stand on the shoulders of giants ;-)

Logind is in a similar situation. It is developed as a component in systemd, which isn’t available on all systems which run KWin. Luckily we don’t depend on logind directly but only use a subset of a well defined D-Bus interface and that interface can be provided by other tools as well. Something like that is already being worked on for the BSD’s.
Like with libinput, I would much prefer to keep KWin lean and efficient and not complicate the code base and development by including libraries for specific platforms or having security relevant code around. As written above: using suid wrappers is very much a no-no to me. But of course it would be possible to implement the subset of the D-Bus in an independent project and provide it. KWin would happily use it, it just needs someone to write the code. So if enough people care, I’m quite sure that there will be a developer stepping up and writing the code.

I decided to put out a small FAQ here for those who have questions about the implications of the above:

FAQ

Does that mean KWin (and Plasma) depend on systemd?

No.

But it depends on logind?

No. It uses one D-Bus interface provided by logind. It doesn’t care which program is providing this D-Bus interface. It can be logind or logind-shim or the implementation being worked on for the BSDs. Even a small binary just providing the used D-Bus interfaces would work.

You should not use logind, there must be a different solution!

I’m sorry I did not find any solution which was as efficient and secure as the one provided by logind. Of course there are solutions like weston-launch, but they introduce a lot of complexity – both on the coding side and on the installation side. As such a solution would need to be suid, I’m very reluctant to the idea. We shouldn’t introduce such possible security risks, if there are better solutions available. Logind is simply providing a feature which is needed by kwin_wayland.

Does that affect KWin on X11?

No, that only affects kwin_wayland.

But there is no logind for the BSDs! So I won’t be able to run kwin_wayland on BSD systems?

Unfortunately the fact that logind is missing is least of your problems on BSD. Logind support is only needed for libinput which right now is not available on BSD. The kwin_wayland binary on BSD will not try to interact with logind. I’m sorry I don’t have a solution for the input stack on BSDs. I really hope the BSD developers can come up with a solution for this as we don’t have the resources to build a separate input solution for one platform.

How can I change KWin to not use logind?

As I noted, it is important to me that KWin is secure and that the code base is as easy to understand as possible. I don’t like the idea of having ifdefs all over the place and multiple solutions as that results in bitrot. When I pushed the libinput change it directly failed to build on the CI system as the ifdefs introduced a variation which I couldn’t test on my system. Each ifdef and each platform-specific solution increases the development and maintenance costs significantly. This means that I will only accept patches which don’t introduce the above mentioned problems. Preferrable a small wrapper binary could provide the needed D-Bus interface for KWin and other applications which need this functionality. This would not need changes in KWin at all and would be from my perspective the perfect solution.

Why won’t you implement such a wrapper binary?

Honestly there are a million things I would do if I had time, but a day has only 24 h and I have to prioritize my work. Just check the Wayland TODO list for what we all need to do to get KWin/Wayland ready. Why don’t you open your editor and get some work done? ;-)

But if KWin uses logind, Slackware will drop all of KDE!

Yes, I have read that (see comments). Luckily the situation for Slackware is similar to the BSDs: it doesn’t matter right now. Slackware doesn’t provide Wayland packages yet, so the logind support won’t be used as there is no kwin_wayland binary which could be built. And if enough people care, as I said, one or more of them can write the wrapper binary and KWin and other compositors will work just fine.

How can i help?

Best by writing code :-) See the TODO list I linked in an above answer. Also it would be good if someone documented the steps to get kwin_wayland running and how to develop on it cough.

An Update on kwin_wayland

With the initial release of Plasma 5.0 behind us I also started to look more in the direction of Wayland again. Now I’m kind of in full flow on Wayland work and kwin_wayland is progressing nicely. Yes, KWin 5.1 will introduce a new binary called kwin_wayland to complement the kwin_x11 binary which got introduced in KWin 5.0.

Now I do not want to list all the changes as you can hardly express them all in a blog post, but I can point to my Akademy talk. I will provide a small overview of the current state, what is new in KWin 5.0, what will be new in KWin 5.1 and where the journey is going.

I'm going to Akademy 2014

Of course there is lots of work going on and help is always appreciated. We started to use a public available task tracker on todo.kde.org. Also I have to say that there are still quite some open tasks for kdecoration2. Please help as I cannot split myself and it would be super important to release KWin 5.1 with kdecoration2.

Running frameworks powered applications on Wayland

After I got KWindowSystem to no longer require X11 on Linux I dared to start some of our applications which got ported to frameworks 5 with the magic “-platform wayland” command line switch. And look there: they started and were useable. But there were some obvious problems: e.g. our platform integration plugin was not loaded. But with a small change to Qt 5.3 this problem is solved. Now if you run a Wayland compositor in a KDE session you will get the KDE integration as one would expect. But this just openend a new set of problems: our default QStyle (Oxygen) was crashing any application on startup as it was still assuming that it’s on X11 if compiled with X11 support. Easy enough to fix and now Oxygen works as one would expect.

This experience highlighted an obvious problem: our applications will crash if they are compiled with X11 support and do not perform a runtime check. So I started to hunt down possible crashers in our libraries. In the perfect world there should not be any X11 specific code in the libraries (except KWindowSystems). And luckily our libraries supported non-X11 platforms for quite some time, so all the X specific code should be in HAVE_X11 blocks. So a small reminder for everybody who thinks that we should never have supported Windows in the first place: thanks to Windows support most of our applications just work on Wayland. Have a look at this nice collection of applications which are running just fine on Wayland:

Aus Weston

Right now we have reached a state where I am no longer able to crash applications at random and most applications which I give a try just work. We had a few problems in kde4support which prevented any applications which still used K(Unique)Application to start at all on Wayland and that framework had stronger usage of HAVE_X11 than I expected. Although the framework is deprecated I fixed all the issues I could find in it, to get applications to support Wayland faster. Basically if your application compiles on frameworks it will run on Wayland out of the box. Also thanks to that work our applications will just work on any proprietary windowing system (so no need to be upset that we put more work on e.g. Windows support than on your small pet project). My personal aim is to have Wayland support at least on par with Windows support in our first release of frameworks. Later on we will improve it to get it on the same level as X11 support.

Of course it’s possible that I haven’t spotten all crashes in the libraries yet. And I encourage our application developers to test their applications on Wayland. That’s quite simple in fact: just start Weston (can be nested in the X session) and prepare the setup:

source kf5_env.sh # your kf5 environment variables setup script
unset DISPLAY # to make sure that any XOpenDisplay call will fail
export WAYLAND_DISPLAY=wayland-0 # adjust to whatever you used in Weston

Now all you need is to pass “-platform wayland” to the command line args to start your application on Weston. In case you are using a KApplication and KCmdLineArgs you need to use “--platform wayland” instead. Update: as an alternative one can also export the environment variable “QT_QPA_PLATFORM” and set it to “wayland”. If you run into any problems with it, feel free to ping me on IRC. In case you get your application to crash in one of the frameworks and it’s a crash due to using X11, please report a bug and set me on CC. In case you hit a crash in your application there is a very easy pattern to solve the problem:

#if HAVE_X11
if (QX11Info::isPlatformX11()) {
    // do your X specific calls here
}
#endif

If you are not using QX11Info you can also do:

#if HAVE_X11
if (QGuiApplication::platformName() == QStringLiteral("xcb")) {
    // do your X specific calls here
}
#endif

As you can see it’s performing a string comparison so you don’t want to have that in a code path which gets called often. Recommendation: move it into a member variable which gets initialized only once in app life cycle and also ifdef the variable – no need to do these checks on e.g. Windows. If you have any questions with it, feel free to ask me.

In the workspace module I hit a few crashers in modules which make absolutely no sense on non-X11. The code is too X specific and most likely needs a complete rewrite if we want to have it on non-X11. Of course we don’t want to have modules which just crash in systemsettings. So I introduced a new method to KService to check whether the plugin makes sense on the current Qt platform. This is honored by ::noDisplay() so systemsettings (and also kcmshell5) won’t pick them. To specify one can use e.g.

X-KDE-OnlyShowOnQtPlatforms=xcb

in the plugin’s desktop file. There’s also a “X-KDE-NotShowOnQtPlatforms”.

KWindowSystem in Frameworks 5

KWindowSystem is a tier 1 framework which allows to interact with the windowing system. Historically it provided an implementation of NETWM on X11. It provides a NETRootInfo for accessing the global state (all that’s set on the root window) and NETWinInfo for all information about a specific window. The classes have a window manager and client perspective. This is the foundation which powers our window manager and various parts of the desktop shell such as the taskmanager.

On top of those X11-specific classes we have a convenient API KWindowInfo and KWindowSystem which provides a windowing system independent API for our applications. Thus we have different implementations depending of the platform it’s compiled on. On X11 the implementation depends on named NET* classes, on Windows and Mac what makes sense is implemented using the platform specific API.

In the good old days of Qt 4 this was a sufficient solution. If it’s build on unix-like systems we have X11 and we know it’s X11, on Windows it’s Windows and so on. With Qt 5 this no longer works. Just because we built with X11 support doesn’t mean the software will run on X11. Due to the introduction of the QPA it’s possible that another platform is used – most obvious Wayland. But there are more platforms for Linux like Android. With the solution from Qt 4 our applications would just crash as soon as they access KWindowSystem as that tries to interact with XLib/xcb unconditionally.

Over the last weeks I spent quite some time on making sure that KWindowSystem works as expected (that is doing nothing at the moment) when run on Wayland. Just adding a new implementation as in the Qt 4 days is no solution as we want our framework to support both X11 and Wayland at the same time. Otherwise our distributions would have a hard time packaging our software. The approach was to introduce an internal abstraction in KWindowInfo and KWindowSystem and have a platform implementation. This is now done for X11 together with a dummy implementation which is used as a fallback if we do not have an implementation for the currently used windowing system (e.g. Wayland). Unfortunately this has a side-effect: it broke the backend for Windows and MacOS. I don’t feel very happy about it as I don’t like to break the work of others, but I cannot fix it. Windows and MacOS are proprietary systems for which either a license or even specific hardware is required. I do hope that the specific teams will re-add the support till the release of frameworks 5. Please note: at the time of writing this blog post not all patches are merged yet.

A nice side-effect for this work was that I started to write unit tests for KWindowInfo on X11. This is far from trivial as it interacts with X11 and the running window manager. And the test kind of depends on the used window manager. Obviously given that it’s KDE it would make sense to write the test against KWin, but that’s not sufficient for our CI system as kwindowsystem is a dependency of KWin and thus kwindowsystem cannot depend on KWin (not even on runtime). Thus the tests are now performed against openbox on the CI system, but also succeed when running against KWin. They are quite a stress test for a window manager and found one very unlikely crash condition in KWin (of course already fixed).

The framework provides more functionality which is kind of X11 specific. For example there is the KSelectionOwner and KSelectionWatcher which implements a manager selection as described in ICCCM, section 2.8. Again a very important building block for our window manager. While it’s clearly X11 specific code which only gets built if X11 is available, there is no reason to crash if it’s not run on X11. So I went through all of our sources and tried to make sure that it correctly checks whether the runtime platform is xcb. Thus we don’t have to change all applications using it, but can rely on the library not to crash. Still if your application is using these X11 specific functionality I highly recommend to check for the platform as you might run into runtime errors. E.g. claiming a manager selection will fail, don’t rely on it.

As all of this is kind of a requirement to running frameworks based applications on Wayland, I have to do the obligatory screenshot of Kate on Weston:

Aus Weston

An unexpected journey

Since my last post quite some progress has been made in getting KWin working on top of a Wayland compositor. My main focus of work has been on the input stack. This is something I am not really familiar with as so far we did not have to care about it.

As some might know input handling in X11 is very insecure. Every application is able to listen to every key event. And in the KDE workspaces we obviously make use of these “features”. For example the global shortcut handling is implemented as a kded module listening to all key events and notifying the application via D-Bus that the shortcut got triggered. In a post-X11 world this will not work any more: applications are no longer able to listen to all key events.

One of the important tasks therefore is to not send all input events to the
X-Server but just to the window which should get it, or to handle the input events inside KWin and discard them without passing to the applications. My current branch already supports activating screen edges and using fullscreen effects like Present Windows completely without passing the input events through X. This means that some of the longstanding issues are automatically resolved. We no longer “steal” the screen edges from applications and starting the screen locker should be possible even if present windows is active (not tested and still needs some screen locker support in KWin).

But this also means that features like the global shortcut handling start to break. Mostly because kded is not listening on my virtual X but that’s just detail ;-) So I also had to start working on global shortcut support inside KWin – otherwise I would have had a hard time to use some of the important features like switching desktops or using Alt+Tab. At the moment the latest code only supports KWin’s internal shortcuts, but global support for kglobalacceld will have to follow. We had some ideas on how to improve global shortcut handling in general anyway.

When the infrastructure was in place I couldn’t resist the temptation to work on features which had been requested for a long time and were not really possible to do with X11 (or only with huge hacks): mouse shortcuts. Ctrl+Alt+left click activates desktop cube. But that’s not enough. While sitting in the Kubuntu Developer Summit I added also mouse wheel support which allows us to ctrl+alt+wheel to switch desktops and also to use the zoom effect. This work can be found in branch kwin/global-shortcuts on my personal workspace clone.

I hope that we will soon have a decision on the “what’s master” discussion and I can start merging the changes in. Once this is done I plan to switch focus to the Qt 5 port and concentrate on this for the next few weeks.

Nevertheless this should not be a reason to stop the work on the Wayland efforts. There are so many small things which can be done and lots of low hanging fruit. I will try to setup a trello board with a few tasks, so that interested developers could start picking up some easy tasks and get their hands dirty. Once something is in place I’ll write another post.

More Software Compositing

One of the most often repeated misconceptions about Wayland is that it requires hardware acceleration. I would have thought that this issues would have been resolved once the reference compositor, Weston, supported rendering through Pixman. The reason for this misconception is most likely that the earlier versions of Weston required hardware acceleration.

Now KWin in 4.11 introduces the same mistake. Our experimental Wayland support is only available in the OpenGL compositor – even OpenGL ES is not supported (something is broken on my system I cannot start it). I think this is a bad situation. One of the huge advantages of KWin is the exchangeable compositor allowing to switch to software based XRender in case there are no proper drivers available. In fact KWin switches automatically to XRender if it detects a driver which recommends the XRender backend (e.g. software rasterizer).

So last week I started to work on some new Wayland related features and extended the XRender compositor in a way that it renders to a Wayland surface. It’s a pretty simple setup. Our XRender compositor uses an off-screen pixmap for rendering. The adjusted backend uses a pixmap created through XShm, so we get the frame rendered into a shared memory segment. From there we just copy it into a shared memory Wayland buffer to share it with the Wayland compositor. It would be better if we could create the shared Wayland buffer directly from the XShm segment, but I did not see a solution on how to achieve this.

In the end it’s also not really relevant. After all we want to use Wayland and the last thing we want to do on Wayland is using an X11 rendering API for compositing. So we can consider this more as a proof-of-concept and as a task so that I can get more familiar with Wayland ;-) It helped me to improve the code, so it was worth the effort. Even if it is a more academic solution it serves a concrete use-case: KWin uses XRender as a fallback and we can expect that the backend works. Given that we have to enforce compositing in future, it’s good to know that all backends work. The current state of this work can be found in branch kwin/wayland-backend of my clone (note: the link might change in case I force push).

Video on YouTube

If XRender is an academic solution, we are still facing the problem that we lost our software compositor. So I thought I need to also do something about this problem and started to work on a new compositing backend based on QPainter. It’s a quite simple compositor which uses a QImage as the rendering target. The QImage is created directly from the Wayland buffer, so in opposite to the XRender backend we don’t have the needless memory copy from one to another buffer. This is a very interesting compositor as it’s the first part of KWin which has been written from scratch only with Wayland in mind. It doesn’t support X11 (at the moment). This is the first new compositor written in years, so I found some things to improve and which are already under review. And as it was much easier to write a new compositing backend than I expected, I decided to document it in our wiki. It would be totally awesome if someone could have a look at it and write a dedicated compositor for the RaspberryPi ;-) The new compositor can be found in branch kwin/qpainter-scene.

The switch to Wayland does not only affect our software compositors, but compositing in general. So far KWin has always had the possibility to not require compositing. This doesn’t make any sense in a Wayland world. So I needed to teach KWin that we might need to enforce compositing. Of course on X11 we still want to have the option to disable compositing, so the changes only affect the Wayland backend. When starting KWin with the Wayland display environment variable set, KWin will require compositing. Even if it is disabled in the config options it will be enabled. Even more it ignores the shortcut to suspend compositing and options like unredirection of fullscreen windows. But still the backend might fail and in this case KWin is just exiting. There is no need to continue execution if one cannot see the output. This work can be found in branch kwin/enforce-compositing (which also contains all the other mentioned branches).

Starting a full KDE Plasma session in Wayland

This week there will be the release of KDE SC 4.11 Beta 1 and this will come with an interesting new feature: an experimental Wayland backend inside KWin. This backend does not allow to manage Wayland clients, but uses another Wayland compositor as the rendering target. Instead of rendering to an X window, KWin renders to a Wayland surface. From an architecture point of view this means that there is a Wayland system compositor and KWin is running as a Wayland session compositor (although KWin is not yet a Wayland compositor).

Disclaimer: this is a highly experimental feature and not intended for productive usage. Please do not report any bug reports in the bug tracker. If you find an issue please open your editor, hack and submit a review request.

Now I expect that you are also excited about Wayland and that you want to run your KDE Plasma session on top of Wayland instead of X, right? So here are the instructions: First of all you need to change the “Tearing Prevention (VSync)” in KWin’s Advanced desktop effects settings to “Full scene repaints”. This is needed because KWin does not yet support the buffer-age extension needed to properly repaint in Wayland. Note: when using X11 you probably don’t want to use this option, it wastes quite some power.

The KDE Plasma session startup is controlled by a script called “startkde”. In order to start the KDE Plasma session with Wayland we also need this script, but need to have a few environment variables set to tell KWin that it should use Wayland. So best copy this script and call it” startkde-wayland”. Edit this file and add the following exports after the shebang:

export DISPLAY=:99
export WAYLAND_DISPLAY=wayland-system-0
export KWIN_OPENGL_INTERFACE=egl_wayland

Now all we need is another small shell script to start Wayland, Xvfb and the modified startkde script. Let’s call it kwayland:

#!/bin/sh
weston-launch -- --socket=wayland-system-0 &
Xvfb -screen 0 1366x768x24 :99 &
startkde-wayland &

Please change the screen resolution to the one used by your screen. KWin is not yet able to update the resolution of the X server to what Weston uses. This also means that you cannot change the resolution or add multiple screens (as I said: it’s an experimental feature).

Now log out of your running KDE session, switch to a tty and stop the still running X server. With X out of the way you can start the kwayland script. You should see Weston starting and shortly afterwards KWin should take over. If everything is configured correctly you should see the normal splash screen (seems like my system is not configured correctly).

Once the system is fully started you can just use it. If everything works fine, you should not even notice any difference, though there are still limitations, like only the three mouse buttons of my touchpad are supported ;-) I would post a screenshot but it’s fairly uninteresting as one cannot see a difference.

This blog post was written in a KDE Plasma session running in Wayland.

KWin running in Weston

This week I decided to do some research for the Wayland porting of the KDE Plasma workspaces. One of the features we will need in future is a Wayland session compositor which runs nested on a Wayland system compositor. Of course one could think of setups without a system compositor, but overall I think that a nested compositor simplifies the setup and allows to have all the low level technologies in one place without duplication in all the various compositors. +1 for working together.

After three days of work I already have something to demo (video on youtube):

Sorry for the bad audio. I’ll just explain what one can see. The video starts with the normal X-Server. After that it switches to a VT and we start Weston there. On Weston I’m starting KWin with some environment variables set to pick the correct libraries and force KWin into Wayland mode. KWin creates a connection to Wayland, creates a Wayland surface and uses it for OpenGL output. All the windows from the running X Server are rendered into this surface just as if it were a normal X11 output.

KWin also gets input from Wayland and passes it to the X Server. That’s the reason why we can see mouse interaction and working keyboard.

How it works

The OpenGL backend

KWin supports multiple backends for providing an OpenGL context and doing the texture from pixmap operation. At the moment we have an GLX and an EGL backend. Both create the OpenGL context on the XComposite overlay window and provide the texture from pixmap in the GLX case through the GLX_EXT_TEXTURE_FROM_PIXMAP extension and in the EGL case through the the EGL_KHR_image_pixmap extension.

A new backend is added which creates the OpenGL context on a Wayland surface. The backend started as a fork of the existing EGL backend with the X code stripped out. What’s a little bit tricky is getting the texture from pixmap working. The extension used in the normal EGL on X11 backend is not available. The proper solution would be a setup with XWayland, but that’s still too early as KWin does not yet support Wayland clients.

The solution I came up with is inspired by a fallback mechanism in KWin from the time when GLX_EXT_TEXTURE_FROM_PIXMAP was not guaranteed to be around: XShm to copy the pixmap content into an OpenGL texture. Not a nice solution but it works.

Input handling

Input is currently also a rather hackish solution until we have XWayland up and running. We just take all input events and forward them to the XServer with the XTest extension to inject fake events. It’s a huge hack and one can see how old X is there and how limited. I was rather surprised that it works at all. At the time of this writing the code supports keyboard events and the left, middle and right mouse button. Wheel events are tricky as X uses mouse buttons for them and for more mouse buttons I have problems with mapping them as I’m lacking a multi button mouse.

We are also not able to back sync the mouse position from X to Wayland. As far as I understood the Wayland protocol there is nothing like XWarpPointer, so if something in X warps the pointer we have a mismatch. I agree that warping is evil, but we use it in KWin for activating the screen edges :-(

Next Steps

Cursor

What is to do next is to get the changes to the cursor in X11 and set the cursor on the Wayland surface. That should not be really difficult as the XFixes extension provides everything one would need for that.

Thread

A rather huge limitation at the moment is that the connection to the Wayland display is hold in the main thread. We cannot block there, so we only get events when we actively check for them. This is currently during repainting the screen. So if you wondered why the ShowFPS effect in the video is turned on: it’s to force repaints and to keep the connection alive. This connection needs to go into a thread so that we can block there.

Buffer age

Currently the code forces as to do full-screen repaints. The two solutions we have for non-fullscreen repaints in the EGL backend do not work in the EGL on Wayland backend. The Wayland demo code shows that the EGL_EXT_buffer_age implementation could be used. We wanted to have support for that one anyway in KWin.

Giving it a try

Building from source

I just pushed the code into branch “kwin/wayland-egl-backend” on my clone kde:clones/kde-workspace/graesslin/kde-workspace. Be aware that I intend to force push to this branch.

To build you need to have the Wayland libraries around. The CMake module tries to find it through package kit. Watch the output of CMake, Wayland is only an optional dependency! If you build Wayland and Weston from source, please follow the instructions. You might also need to build Mesa from source.

How to start

Starting is rather simple. Just have an Xserver running somewhere, start Weston on a VT (don’t do nested on X, you would only get a black screen once KWin started) and open a terminal. Setup your environment variables to be able to start KWin and then start KWin in the following way:

DISPLAY=:0 KWIN_DIRECT_GL=1 KWIN_OPENGL_INTERFACE=egl_wayland kwin --replace &

KWIN_DIRECT_GL makes KWin skip the OpenGL Test application which is currently only supporting GLX and seems to fail if one is on a different VT. KWIN_OPENGL_INTERFACE tells KWin which backend to use. So by using “egl_wayland” one forces KWin to use the Wayland backend. In future we will probably detect whether the Wayland display environment variable is defined and just pick it directly (though that is dangerous in case of nested Weston on X).

When will it be available in a release

I want to get this work into 4.11 as an experimental feature for multiple reasons. I wanted to have a build dependency to Wayland in 4.11 for quite some time. So that’s a nice excuse. Unfortunately feature freeze is approaching.

Of course I want to give users something to play with. We have talked about Wayland for such a long time and there is nothing to really see that we are walking the way. Last but not least that might be a rather important solution for Plasma Active as that hopefully allows to run on hardware where we currently would not get OpenGL. With libhybris we should be able to get KWin working with OpenGL and being an X11 compositor even if the driver doesn’t support X11. Though I have not tested this case yet and have no devices to play with. But I know a few people in the community who might be interested to play with that.

Mir in Kubuntu

As you might have seen in Jonathan’s blog post we discussed Mir in Kubuntu at the “Mataro Sessions II”. It’s a topic I would have preferred to not have to discuss at all. But the dynamics in the free software world force us to discuss it and obviously our downstream needs to know why we as an upstream do not consider Mir adoption as a valid option.

This highlights a huge problem Canonical created with Mir. I cannot just say “Canonical sucks”[1] to discard Mir as an option, I have to provide proper technical arguments why we won’t integrate Mir. I have to invest time to investigate the differences, advantages and disadvantages. As I have those arguments, I thought it might be a good idea to share them in a blog post.

The discussion started during a presentation about X11 and Wayland to my fellow team mates at Blue Systems. I decided to first explain X11 as I think one cannot understand the needs for Wayland without understanding X11. I did not intend to discuss Mir at all, but somehow the discussion drifted into the direction and the valid questions were raised about what are the differences and advantages of Mir or Wayland. What followed was kind of a rant about Ubuntu and Canonical [2]. So later the week we discussed “Mir in Kubuntu” in more detail to try to find answers to the many questions this raises for our downstream.

Introduction

Frustration and lost Motivation

Before I go into more detail I want to make one thing clear: Canonical is totally allowed to develop whatever they want. I’m totally fine with this and don’t care whether they develop another display server, an own os kernel or yet another desktop shell. I couldn’t care less. It’s Canonical/Mark’s money and he can invest it in any way he considers as useful. I wouldn’t even care if it would be proprietary software, that’s all fine.

What is not fine is causing a major disruption in the free software ecosystem by giving false technical arguments and doing bold statements about software Canonical does not contribute to. This is not acceptable. This was very frustrating and destroyed lots of trust I had in Canonical. It will be difficult to rebuild this trust. Canonical can be glad that it is the free software world and not the normal corporate world. There were quite some statements which could have raised the legal department in the normal corporate world[3]. It also cost lots of motivation at least on my side and I even questioned whether it’s still worth to be a member of the free software ecosystem. Instead of working together we now have a situation where members of the ecosystem become a competitor and which badmouth part of the software stack. A very frustrating situation.

There certainly are valid reasons for developing Mir which also make sense. Unfortunately they have not been presented so far. I’m quite sure that I know the reasons and if they would have been said straight away it would have been for me and other projects probably much easier. It would have taken away the frustration which the announcement caused and we would not need to discuss it at all, because those question marks would not exist. But apparently Canonical decided to give false technical arguments over the real ones.

Not ready yet

At the moment Mir is not there yet, this is important to remember. With the announcement we basically had four options on how to handle the situation.

  1. Continue with the Wayland plan and ignore Mir
  2. Switch to Mir and ignore Wayland
  3. Support Mir and Wayland
  4. Delay decision until Mir is ready

If I map our time line for Plasma Workspaces 2 against the time line of Mir I see no overlap. We want to support Wayland before Mir is ready. So delaying the decision would be a rather bad idea. It would just throw us back. This also means that option 2 is not valid especially as we would need to delay until Mir is ready for this to happen. So the only valid options are supporting both Mir and Wayland or only Wayland. At the moment the code is not ready yet to properly decide whether supporting Mir in addition to Wayland is a valid approach or not. Last time I checked the source base I hit a few stubs and then obviously stopped looking at the code as it’s not worth the effort yet. So we have to evaluate on the knowledge we already have and that doesn’t look good on the Mir side.

Wayland vs Mir

Possible Advantages of Mir over Wayland

The differences between Mir and Wayland are rather minimal. One of the differences is that Mir uses server allocated buffers while Wayland uses client side buffer allocation. I cannot judge whether this is an advantage or disadvantage. But I trust Kristian and the Wayland team more on that topic.

Another difference is that Mir uses test-driven development. To me development methodology is not a technical argument. I rather use a working system without unit tests than a system with unit tests that doesn’t work [4]. Also KWin does not use TDD. If I would consider TDD superior I would have to question my own development methodology.

But that’s it. That are the differences I found so far which could count as an advantage for Mir. But of course there is the advantage that Mir is going to be awesome. For the disadvantages I will spend a complete section on each point.

Distro specific

So far Mir is a one-distribution solution. So far no other distribution has shown any interest in packaging Mir even if it would become a working solution. Unfortunately I don’t have the ability to see into the future, but I can use the past and the present to get ideas for the future. The past tells me that there are other Canonical specific solutions which are not available in other distributions. I do not know of any distribution which packages Unity and from all I have heard it’s even impossible to package Unity on non-Ubuntu distributions. Given that it is quite likely that Mir will go the same road. It’s designed as a solution for Unity and if distros don’t package Unity there is no need to package Mir.

This has quite some influence on a possible adoption. I do not know of any kde-workspace developer using (K)Ubuntu. I do not see how anyone would work on it or how we should be able to review code or even maintain code. It would mean all the adoption would have to go into ifdef sections nobody compiles and nobody runs. This is the best way to ensure that it starts to bit-rot. Even more our CI system runs on openSUSE so not even the CI would be able to detect breakage. Of course a downstream like Kubuntu could develop the adoption and carry it as a patch on top of upstream, but I would highly recommend them to not do this as KWin’s source code churn is too high. Also we all agree that downstream patches are evil and we would no longer be able to help in any way downstream’s user from a support perspective.

Architecture

Mir’s architecture is centered around Unity. It is difficult to really understand the architecture of Mir as the specification is so full of buzz-words that I don’t understand it [5]. From all I can see and understand Unity Next is a combination of window manager and desktop shell implemented on top of Mir. How exactly this is going to look like I do not know. Anyway it does not fit our design of having desktop shell and window manager separated and we do not know whether Mir would support that. We also do not know whether Mir would allow any other desktop shell except Unity Next, given that this is the main target. Wayland on the other hand is designed to have more than one compositor implementations. Using KWin as a session compositor is an example in the spec.

License

Wayland is licensed like X under the MIT license, which served us well for a display server. I think this is a very good choice and I am glad that the Wayland developers decided for this license. Mir is licensed under GPLv3-only with CLA. I think this is very unsuited for such a part of the stack and would render quite a risk for usage in KDE Plasma. KWin (and most KDE software) is GPLv2-or-later, this would no longer be possible, it would turn our code into GPLv3-only as KWin (or any other software which would depend on mir-server) would be a derived work of Mir. I do not consider GPLv3-only software as a possible dependency of any core part of our application stack. It renders a serious threat for the future in case of a GPLv4 which is not compatible with GPLv3. I also dislike the CLA [6]. So from a licensing perspective Mir is hardly acceptable.

Unity Specific/No Protocol

One of the most important aspects from Wayland for us is the ability to extend the protocol. This has already been a quite important feature in X and we are using our own extensions over ICCCM and EWMH to implement additional functionality. Of course our workspace has own ideas and it is important for us to be able to “standardize” those and also make them available to others if they are interested. This is possible thanks to protocol extensions.
Mir doesn’t have a real protocol. The “inner core” is described as “protocol-agnostic”. This renders a problem to us if we would want to use it. Our architecture is different (as described above) and we need a protocol between the desktop shell and the compositor. If Mir doesn’t provide that we would need to use our own protocol. And that already exists, it is called “Wayland”. So even if we would support Mir, we would need the Wayland protocol?!? That doesn’t make any sense to me. If we need to run Wayland on top of Mir just to get the features we need, why should we run Mir at all?

But it gets worse, the protocol between Mir server and Mir clients is defined as not being stable. In fact it’s promised that it will break. That’s a huge problem, I would even call it a showstopper. For Canonical that’s fine – they control the complete stack and can just adjust all bits using the protocol like QMir.

For us this looks quite different. Given that the protocol may change any time and given that the whole thing is developed for the needs of Unity we have to expect that the server libraries are not binary compatible or that old version of the server libraries cannot talk with the latest client libraries. We would constantly have to develop against an unstable and breaking base. I know that this sounds overly pessimistic but I know of one case where a change got introduced in a Canonical protocol late in the release cycle completely breaking an application in Kubuntu which wanted to use the protocol. Given this experience I would not trust that the protocol doesn’t change one day before the release meaning that Kubuntu cannot ship.

This is not awesome, it’s awful. It means KWin will not work just fine on Mir.

I hope this shows that using Mir inside the KDE Plasma workspaces is not an option. There are no advantages which would turn Mir into a better solution than Wayland and at the same time there are several showstoppers which mean that we cannot integrate Mir – not even optionally in addition to Wayland. The unstable protocol and the licensing choice are clearly not acceptable.

What this means to Kubuntu

Question marks

For Kubuntu the Mir switch by Canonical created quite some questions. One of those questions is answered: Upstream has no interest in supporting it and would most likely not accept patches for support. With upstream not using Mir the question is how the graphics stack for Kubuntu will look like once Ubuntu switched to Mir? The questions cannot be answered right now but it doesn’t look good.

Patches to the stack

Ubuntu has always had one of the worst graphics stack in the free software world. I can see this in the bug tracker. The quality of the Mesa stack in Ubuntu is really bad. For Mir Ubuntu will have to patch the Mesa stack even further. This is nothing which I would like to see. Also Mesa needs to be packaged with Wayland support. But will Canonical continue to do this? If not, would Kubuntu (and other Ubuntu flavors) need to ship their own Mesa stack? What if the changes by Canonical are so large that a standard Mesa stack doesn’t run on top of the Ubuntu stack?

Switching Sessions

One of the advantages of free software is that one can select the desktop environment in the login manager. This looks like no longer be possible in a Mir world. Unity will run with a Mir system compositor with LightDM nested underneath. We will need either the X Server or a Wayland system compositor. So from the login manager it will not be possible to start directly into a session using a different system compositor. How will it continue to be possible to use both Unity and KDE Plasma on the same system? Running a Unity and a KDE Plasma (or GNOME or XFCE or anything) session at the same time seems to no longer be possible.

System Compositor

How deep into the system is the system compositor going to be? Will it be possible to disable the Mir system compositor and replace it with X or Wayland? What if the packages start to conflict? Will it still be possible to install Kubuntu and Ubuntu on the same system? Will Canonical care about it? Will the system compositor mean that one has to decide in Grub whether to boot Ubuntu or Kubuntu?

Packages from Where

So far X, Wayland and Mesa have been packaged by Canonical. But what about the future? Will there still be packages for X, will there be packages for Wayland? If not, where to take them from? Debian unstable, most likely. But Debian might be frozen. Will it be possible at all to use the Debian packages for X and Wayland in the Ubuntu stack? Will they meet the requirements for KDE Plasma[7]? If Canonical doesn’t provide Wayland packages, they would drop to universe, so Mesa in main cannot depend on them. How to get then Mesa with Wayland support?

Only Future can tell

Those questions cannot be answered right now. It will have to wait until Mir is integrated into the Ubuntu stack. Then Kubuntu developers will see how far the stack broke. I’m not really optimistic that it will still be possible to provide the Ubuntu flavors once the transition to Mir is done. I don’t think that Canonical has any interest in the community provided distributions on top of Ubuntu any more. There are many small changes in the direction which indicate that. But we will see, maybe I’m too pessimistic.

[1] Given how Canonical introduced Mir with incorrect information about Wayland I consider this as a valid approach to dismiss the technology.

[2] I was very fed up with Ubuntu at the time anyway because our bug tracker once again exploded after the Ubuntu release.

[3] I do admit that I thought about asking KDE e.V. to send an Abmahnung after the statement that KWin would just work fine on Mir.

[4] In fact I consider TDD as utter non-sense and as a useless methodology though some aspects are useful.

[5] “with our protocol- and platform-agnostic approach, we can make sure that we reach our goal of a consistent and beautiful user experience across platforms and device form factors”

[6] Yes I know that Qt also has a CLA, which I have signed. But for Qt there is also the KDE Free Qt Foundation agreement.

[7]Last week a feature hit KWin which I cannot test/use because the X-Server is too old in Debian testing.

The History on Wayland Support inside KWin

Ever since a certain free software company decided to no longer be part of the larger ecosystem, I have seen lots of strange news postings whenever one of the KDE workspace developers mentioned the word “Wayland”. Very often it goes in the direction of “KDE is now also going on Wayland”. Every time I read something like that, I’m really surprised.

For me Wayland support has been the primary goal I have been working on over the last two years. This doesn’t mean that there is actual code for supporting Wayland (there is – the first commit for Wayland support in our git repositories is from June 11, 2011 (!)).

The Wayland research projects two years ago had been extremely important for the further development of KWin since then. First of all it showed that adding support for Wayland surfaces inside KWin’s compositor is rather trivial. Especially our effect system did not care at all about X11 or Wayland windows. So this is not going to be a difficult issue.

The more important result from this research project was that it’s impossible to work against an always changing target. At that time Wayland had not yet seen the 1.0 release, so the API was changing. Our code broke and needed adjustments for the changing API. It also meant that we could not merge the work into our master branch (distributions would kill us), we needed to be on a different branch for development. Tracking one heavily changing project is difficult enough, but also KWin itself is changing a lot. So the work needed to be on top of two moving targets – it didn’t work and the branch ended in the to be expected state. Now with Wayland 1.0 and 1.1 releases the situation changed completely.

The next lesson we learned from that research project was that the window manager part is not up to the task of becoming a Wayland compositor. It was designed as an X11 window manager and the possibility that there would not be X11 had never been considered. We started to split out functionality from the core window manager interface to have smaller units and to be able to add abstractions, where needed, to support in future more than just X11. That had been a huge task and is still ongoing and it comes with quite some nice side-effects like the rewrite of KWin scripting (helped to identify the interface of a managed Client inside KWin), the possibility to run KWin with OpenGL on EGL since 4.10, the new screen edge system in 4.11 and many many more. All these changes were implemented either directly or indirectly with Wayland in mind. That means we have been working on it for quite some time even if it is not visible in the code.

My initial plannings for adding Wayland support around October/November last year was to start hacking on it in January. I was so confident about it that I considered to submit a talk for FOSDEM which would demo KWin running on Wayland. In the end I decided against it as it would have meant working on some of our very important foundations under time pressure, which I don’t think is good for maintainability of the code base.

In December though I decided to adjust my plans and focus first on the Qt 5 port as that would allow us to use the Qt Wayland bindings which are a little bit more convenient for usage in a Qt based application than the native C library. This is not something I just come up with, I discussed this small adjustment with a few people (for example Aaron Seigo) at the Qt dev days last year.

On January, 22nd 2013 sebas outlined the time line for the transition to Qt 5 and Wayland of the KDE Plasma Workspaces. It clearly states that KWin will become a Wayland compositor and that this is a target we are working on with a clearly defined time line.

Given that I am really surprised to see media writing again and again that we started to work on Wayland because other projects deciding against or for Wayland. It’s not something we decided on recently and it is quite clear that our work does not depend on any decisions or announcements our competitors do. We are an independent project, which does it’s own decisions for long term planning. The fact that our work now shifts towards Wayland just at the time our competitors decide for or against Wayland is pure coincidence.

Last week we had a Plasma developer sprint in Nuremberg (thanks to SUSE) and of course Wayland was an important topic for discussion. We had many points on the agenda. After all it’s the first sprint for us since we work on Plasma Workspaces 2 and it is needed to define our direction for the next year. Given all that I wrote so far it’s rather obvious that we would have discussed Wayland even if a certain project would not have done a certain announcement. Some topics on the agenda were based on discussions we had on the mailing list before the certain announcement was made (I’m currently sitting in a train with not sufficient Internet connection so I cannot look up the reference). Which also shows that we had long term plans which were decided on long before the ecosystem shifted.

Summary: we had been working on Wayland for years and it is our long term strategy. Our strategy is of course not based on any announcements of our competitors. We of course need to evaluate new solutions available in the ecosystem when they come up. As I already pointed out in the past, this is not yet possible at the moment and all we can see so far is that Wayland is a better solution for our needs in the KDE Plasma Workspaces than other windowing systems which might be tailored towards the needs of our competitor’s desktop shell. One of the advantage of free software projects is that the development and discussions are open and that it’s quite easy to reach out to the developers and architects of the software.