After three days of work I already have something to demo (video on youtube):

Sorry for the bad audio. I’ll just explain what one can see. The video starts with the normal X-Server. After that it switches to a VT and we start Weston there. On Weston I’m starting KWin with some environment variables set to pick the correct libraries and force KWin into Wayland mode. KWin creates a connection to Wayland, creates a Wayland surface and uses it for OpenGL output. All the windows from the running X Server are rendered into this surface just as if it were a normal X11 output.

KWin also gets input from Wayland and passes it to the X Server. That’s the reason why we can see mouse interaction and working keyboard.

How it works

The OpenGL backend

KWin supports multiple backends for providing an OpenGL context and doing the texture from pixmap operation. At the moment we have an GLX and an EGL backend. Both create the OpenGL context on the XComposite overlay window and provide the texture from pixmap in the GLX case through the GLX_EXT_TEXTURE_FROM_PIXMAP extension and in the EGL case through the the EGL_KHR_image_pixmap extension.

A new backend is added which creates the OpenGL context on a Wayland surface. The backend started as a fork of the existing EGL backend with the X code stripped out. What’s a little bit tricky is getting the texture from pixmap working. The extension used in the normal EGL on X11 backend is not available. The proper solution would be a setup with XWayland, but that’s still too early as KWin does not yet support Wayland clients.

The solution I came up with is inspired by a fallback mechanism in KWin from the time when GLX_EXT_TEXTURE_FROM_PIXMAP was not guaranteed to be around: XShm to copy the pixmap content into an OpenGL texture. Not a nice solution but it works.

Input handling

Input is currently also a rather hackish solution until we have XWayland up and running. We just take all input events and forward them to the XServer with the XTest extension to inject fake events. It’s a huge hack and one can see how old X is there and how limited. I was rather surprised that it works at all. At the time of this writing the code supports keyboard events and the left, middle and right mouse button. Wheel events are tricky as X uses mouse buttons for them and for more mouse buttons I have problems with mapping them as I’m lacking a multi button mouse.

We are also not able to back sync the mouse position from X to Wayland. As far as I understood the Wayland protocol there is nothing like XWarpPointer, so if something in X warps the pointer we have a mismatch. I agree that warping is evil, but we use it in KWin for activating the screen edges

Next Steps

Cursor

What is to do next is to get the changes to the cursor in X11 and set the cursor on the Wayland surface. That should not be really difficult as the XFixes extension provides everything one would need for that.

Thread

A rather huge limitation at the moment is that the connection to the Wayland display is hold in the main thread. We cannot block there, so we only get events when we actively check for them. This is currently during repainting the screen. So if you wondered why the ShowFPS effect in the video is turned on: it’s to force repaints and to keep the connection alive. This connection needs to go into a thread so that we can block there.

Buffer age

Currently the code forces as to do full-screen repaints. The two solutions we have for non-fullscreen repaints in the EGL backend do not work in the EGL on Wayland backend. The Wayland demo code shows that the EGL_EXT_buffer_age implementation could be used. We wanted to have support for that one anyway in KWin.

Giving it a try

Building from source

I just pushed the code into branch “kwin/wayland-egl-backend” on my clone kde:clones/kde-workspace/graesslin/kde-workspace. Be aware that I intend to force push to this branch.

To build you need to have the Wayland libraries around. The CMake module tries to find it through package kit. Watch the output of CMake, Wayland is only an optional dependency! If you build Wayland and Weston from source, please follow the instructions. You might also need to build Mesa from source.

How to start

Starting is rather simple. Just have an Xserver running somewhere, start Weston on a VT (don’t do nested on X, you would only get a black screen once KWin started) and open a terminal. Setup your environment variables to be able to start KWin and then start KWin in the following way:

DISPLAY=:0 KWIN_DIRECT_GL=1 KWIN_OPENGL_INTERFACE=egl_wayland kwin --replace &

KWIN_DIRECT_GL makes KWin skip the OpenGL Test application which is currently only supporting GLX and seems to fail if one is on a different VT. KWIN_OPENGL_INTERFACE tells KWin which backend to use. So by using “egl_wayland” one forces KWin to use the Wayland backend. In future we will probably detect whether the Wayland display environment variable is defined and just pick it directly (though that is dangerous in case of nested Weston on X).

When will it be available in a release

I want to get this work into 4.11 as an experimental feature for multiple reasons. I wanted to have a build dependency to Wayland in 4.11 for quite some time. So that’s a nice excuse. Unfortunately feature freeze is approaching.

Of course I want to give users something to play with. We have talked about Wayland for such a long time and there is nothing to really see that we are walking the way. Last but not least that might be a rather important solution for Plasma Active as that hopefully allows to run on hardware where we currently would not get OpenGL. With libhybris we should be able to get KWin working with OpenGL and being an X11 compositor even if the driver doesn’t support X11. Though I have not tested this case yet and have no devices to play with. But I know a few people in the community who might be interested to play with that.

This highlights a huge problem Canonical created with Mir. I cannot just say “Canonical sucks”[1] to discard Mir as an option, I have to provide proper technical arguments why we won’t integrate Mir. I have to invest time to investigate the differences, advantages and disadvantages. As I have those arguments, I thought it might be a good idea to share them in a blog post.

The discussion started during a presentation about X11 and Wayland to my fellow team mates at Blue Systems. I decided to first explain X11 as I think one cannot understand the needs for Wayland without understanding X11. I did not intend to discuss Mir at all, but somehow the discussion drifted into the direction and the valid questions were raised about what are the differences and advantages of Mir or Wayland. What followed was kind of a rant about Ubuntu and Canonical [2]. So later the week we discussed “Mir in Kubuntu” in more detail to try to find answers to the many questions this raises for our downstream.

Introduction

Frustration and lost Motivation

Before I go into more detail I want to make one thing clear: Canonical is totally allowed to develop whatever they want. I’m totally fine with this and don’t care whether they develop another display server, an own os kernel or yet another desktop shell. I couldn’t care less. It’s Canonical/Mark’s money and he can invest it in any way he considers as useful. I wouldn’t even care if it would be proprietary software, that’s all fine.

What is not fine is causing a major disruption in the free software ecosystem by giving false technical arguments and doing bold statements about software Canonical does not contribute to. This is not acceptable. This was very frustrating and destroyed lots of trust I had in Canonical. It will be difficult to rebuild this trust. Canonical can be glad that it is the free software world and not the normal corporate world. There were quite some statements which could have raised the legal department in the normal corporate world[3]. It also cost lots of motivation at least on my side and I even questioned whether it’s still worth to be a member of the free software ecosystem. Instead of working together we now have a situation where members of the ecosystem become a competitor and which badmouth part of the software stack. A very frustrating situation.

There certainly are valid reasons for developing Mir which also make sense. Unfortunately they have not been presented so far. I’m quite sure that I know the reasons and if they would have been said straight away it would have been for me and other projects probably much easier. It would have taken away the frustration which the announcement caused and we would not need to discuss it at all, because those question marks would not exist. But apparently Canonical decided to give false technical arguments over the real ones.

Not ready yet

At the moment Mir is not there yet, this is important to remember. With the announcement we basically had four options on how to handle the situation.

1. Continue with the Wayland plan and ignore Mir
2. Switch to Mir and ignore Wayland
3. Support Mir and Wayland
4. Delay decision until Mir is ready

If I map our time line for Plasma Workspaces 2 against the time line of Mir I see no overlap. We want to support Wayland before Mir is ready. So delaying the decision would be a rather bad idea. It would just throw us back. This also means that option 2 is not valid especially as we would need to delay until Mir is ready for this to happen. So the only valid options are supporting both Mir and Wayland or only Wayland. At the moment the code is not ready yet to properly decide whether supporting Mir in addition to Wayland is a valid approach or not. Last time I checked the source base I hit a few stubs and then obviously stopped looking at the code as it’s not worth the effort yet. So we have to evaluate on the knowledge we already have and that doesn’t look good on the Mir side.

Wayland vs Mir

Possible Advantages of Mir over Wayland

The differences between Mir and Wayland are rather minimal. One of the differences is that Mir uses server allocated buffers while Wayland uses client side buffer allocation. I cannot judge whether this is an advantage or disadvantage. But I trust Kristian and the Wayland team more on that topic.

Another difference is that Mir uses test-driven development. To me development methodology is not a technical argument. I rather use a working system without unit tests than a system with unit tests that doesn’t work [4]. Also KWin does not use TDD. If I would consider TDD superior I would have to question my own development methodology.

But that’s it. That are the differences I found so far which could count as an advantage for Mir. But of course there is the advantage that Mir is going to be awesome. For the disadvantages I will spend a complete section on each point.

Distro specific

So far Mir is a one-distribution solution. So far no other distribution has shown any interest in packaging Mir even if it would become a working solution. Unfortunately I don’t have the ability to see into the future, but I can use the past and the present to get ideas for the future. The past tells me that there are other Canonical specific solutions which are not available in other distributions. I do not know of any distribution which packages Unity and from all I have heard it’s even impossible to package Unity on non-Ubuntu distributions. Given that it is quite likely that Mir will go the same road. It’s designed as a solution for Unity and if distros don’t package Unity there is no need to package Mir.

This has quite some influence on a possible adoption. I do not know of any kde-workspace developer using (K)Ubuntu. I do not see how anyone would work on it or how we should be able to review code or even maintain code. It would mean all the adoption would have to go into ifdef sections nobody compiles and nobody runs. This is the best way to ensure that it starts to bit-rot. Even more our CI system runs on openSUSE so not even the CI would be able to detect breakage. Of course a downstream like Kubuntu could develop the adoption and carry it as a patch on top of upstream, but I would highly recommend them to not do this as KWin’s source code churn is too high. Also we all agree that downstream patches are evil and we would no longer be able to help in any way downstream’s user from a support perspective.

Architecture

Mir’s architecture is centered around Unity. It is difficult to really understand the architecture of Mir as the specification is so full of buzz-words that I don’t understand it [5]. From all I can see and understand Unity Next is a combination of window manager and desktop shell implemented on top of Mir. How exactly this is going to look like I do not know. Anyway it does not fit our design of having desktop shell and window manager separated and we do not know whether Mir would support that. We also do not know whether Mir would allow any other desktop shell except Unity Next, given that this is the main target. Wayland on the other hand is designed to have more than one compositor implementations. Using KWin as a session compositor is an example in the spec.

License

Wayland is licensed like X under the MIT license, which served us well for a display server. I think this is a very good choice and I am glad that the Wayland developers decided for this license. Mir is licensed under GPLv3-only with CLA. I think this is very unsuited for such a part of the stack and would render quite a risk for usage in KDE Plasma. KWin (and most KDE software) is GPLv2-or-later, this would no longer be possible, it would turn our code into GPLv3-only as KWin (or any other software which would depend on mir-server) would be a derived work of Mir. I do not consider GPLv3-only software as a possible dependency of any core part of our application stack. It renders a serious threat for the future in case of a GPLv4 which is not compatible with GPLv3. I also dislike the CLA [6]. So from a licensing perspective Mir is hardly acceptable.

Unity Specific/No Protocol

One of the most important aspects from Wayland for us is the ability to extend the protocol. This has already been a quite important feature in X and we are using our own extensions over ICCCM and EWMH to implement additional functionality. Of course our workspace has own ideas and it is important for us to be able to “standardize” those and also make them available to others if they are interested. This is possible thanks to protocol extensions.
Mir doesn’t have a real protocol. The “inner core” is described as “protocol-agnostic”. This renders a problem to us if we would want to use it. Our architecture is different (as described above) and we need a protocol between the desktop shell and the compositor. If Mir doesn’t provide that we would need to use our own protocol. And that already exists, it is called “Wayland”. So even if we would support Mir, we would need the Wayland protocol?!? That doesn’t make any sense to me. If we need to run Wayland on top of Mir just to get the features we need, why should we run Mir at all?

But it gets worse, the protocol between Mir server and Mir clients is defined as not being stable. In fact it’s promised that it will break. That’s a huge problem, I would even call it a showstopper. For Canonical that’s fine – they control the complete stack and can just adjust all bits using the protocol like QMir.

For us this looks quite different. Given that the protocol may change any time and given that the whole thing is developed for the needs of Unity we have to expect that the server libraries are not binary compatible or that old version of the server libraries cannot talk with the latest client libraries. We would constantly have to develop against an unstable and breaking base. I know that this sounds overly pessimistic but I know of one case where a change got introduced in a Canonical protocol late in the release cycle completely breaking an application in Kubuntu which wanted to use the protocol. Given this experience I would not trust that the protocol doesn’t change one day before the release meaning that Kubuntu cannot ship.

This is not awesome, it’s awful. It means KWin will not work just fine on Mir.

I hope this shows that using Mir inside the KDE Plasma workspaces is not an option. There are no advantages which would turn Mir into a better solution than Wayland and at the same time there are several showstoppers which mean that we cannot integrate Mir – not even optionally in addition to Wayland. The unstable protocol and the licensing choice are clearly not acceptable.

What this means to Kubuntu

Question marks

For Kubuntu the Mir switch by Canonical created quite some questions. One of those questions is answered: Upstream has no interest in supporting it and would most likely not accept patches for support. With upstream not using Mir the question is how the graphics stack for Kubuntu will look like once Ubuntu switched to Mir? The questions cannot be answered right now but it doesn’t look good.

Patches to the stack

Ubuntu has always had one of the worst graphics stack in the free software world. I can see this in the bug tracker. The quality of the Mesa stack in Ubuntu is really bad. For Mir Ubuntu will have to patch the Mesa stack even further. This is nothing which I would like to see. Also Mesa needs to be packaged with Wayland support. But will Canonical continue to do this? If not, would Kubuntu (and other Ubuntu flavors) need to ship their own Mesa stack? What if the changes by Canonical are so large that a standard Mesa stack doesn’t run on top of the Ubuntu stack?

Switching Sessions

One of the advantages of free software is that one can select the desktop environment in the login manager. This looks like no longer be possible in a Mir world. Unity will run with a Mir system compositor with LightDM nested underneath. We will need either the X Server or a Wayland system compositor. So from the login manager it will not be possible to start directly into a session using a different system compositor. How will it continue to be possible to use both Unity and KDE Plasma on the same system? Running a Unity and a KDE Plasma (or GNOME or XFCE or anything) session at the same time seems to no longer be possible.

System Compositor

How deep into the system is the system compositor going to be? Will it be possible to disable the Mir system compositor and replace it with X or Wayland? What if the packages start to conflict? Will it still be possible to install Kubuntu and Ubuntu on the same system? Will Canonical care about it? Will the system compositor mean that one has to decide in Grub whether to boot Ubuntu or Kubuntu?

Packages from Where

So far X, Wayland and Mesa have been packaged by Canonical. But what about the future? Will there still be packages for X, will there be packages for Wayland? If not, where to take them from? Debian unstable, most likely. But Debian might be frozen. Will it be possible at all to use the Debian packages for X and Wayland in the Ubuntu stack? Will they meet the requirements for KDE Plasma[7]? If Canonical doesn’t provide Wayland packages, they would drop to universe, so Mesa in main cannot depend on them. How to get then Mesa with Wayland support?

Only Future can tell

Those questions cannot be answered right now. It will have to wait until Mir is integrated into the Ubuntu stack. Then Kubuntu developers will see how far the stack broke. I’m not really optimistic that it will still be possible to provide the Ubuntu flavors once the transition to Mir is done. I don’t think that Canonical has any interest in the community provided distributions on top of Ubuntu any more. There are many small changes in the direction which indicate that. But we will see, maybe I’m too pessimistic.

[1] Given how Canonical introduced Mir with incorrect information about Wayland I consider this as a valid approach to dismiss the technology.

[2] I was very fed up with Ubuntu at the time anyway because our bug tracker once again exploded after the Ubuntu release.

[3] I do admit that I thought about asking KDE e.V. to send an Abmahnung after the statement that KWin would just work fine on Mir.

[4] In fact I consider TDD as utter non-sense and as a useless methodology though some aspects are useful.

[5] “with our protocol- and platform-agnostic approach, we can make sure that we reach our goal of a consistent and beautiful user experience across platforms and device form factors”

[6] Yes I know that Qt also has a CLA, which I have signed. But for Qt there is also the KDE Free Qt Foundation agreement.

[7]Last week a feature hit KWin which I cannot test/use because the X-Server is too old in Debian testing.

I have never seen an explanation on why Compositing should matter at all. It mostly boils down to “OpenGL is evil” and “I don’t want 3D”. So let’s leave the “educated guesses” behind us and have a proper look to the question whether Compositing matters for “lightweight”. (Remember: lightweight is a buzz-word without any meaning.)

Let’s start from one of the “lightweight” aspects: Memory. For this it’s important to know that KWin is a Window Manager and Compositor. You cannot just have the Window Manager – even if compositing is disabled. It’s not like xcompmgr which runs in another process. For the binary that gets loaded into memory it doesn’t matter whether compositing is enabled or not. The pure binary containing both window manager and compositor has a size of 1.3 M (on Debian testing, that should be KDE SC 4.8). But when you enable compositing an additional library gets loaded containing the Effects. They are all in one library to not have to open 40 different files. (Again on Debian testing) This library has a size of 783 K. The effects do not link any library which is not already linked by KWin core, so that’s what we are talking about from additional memory costs when using any desktop effect from binary size: 783 K!

Obviously the binary mapped into memory is just one aspect. There are also additional data structures which need to be created on the heap. I have no idea how much is really needed and cannot properly look at my system because I have a KWin with debugging symbols loaded. What’s important to know is that only Effects one uses will get loaded and use memory for their data structures. Also the Effects do not load everything they need but wait till they get activated. E.g. the cube effect will only load a background texture if the cube gets activated. But of course effects do not have anything to do with just using compositing. If you don’t want an effect, just disable it.

The biggest cost from compositing in the memory perspective is probably using the XComposite extension which redirects the rendering into an X pixmap. That’s something which will go away once we are on Wayland as application and compositor share the buffer the application is rendering to.

But even so the usage of the additional memory is not just something which is thrown out. It’s just another case of the time-memory tradeoff. Unrelated windows do not need to repaint if a window get’s unmapped, moved or resized. Just look at this comment from the non-composited switching of virtual desktops:

During virt. desktop switching, desktop areas covered by windows that are going to be hidden are first obscured by new windows with no background ( i.e. transparent ) placed right below the windows. These invisible windows are removed after the switch is complete. Reduces desktop ( wallpaper ) repaints during desktop switching

Without compositing we move the time-memory tradeoff towards using more CPU (create those windows) to prevent even more CPU usage. With compositing we don’t have to do anything of it. So disabling compositing is obviously not the silver bullet for being “lightweight”. It just means moving the time-memory tradeoff slider towards CPU.

What remains is the OpenGL question. I know that many people think it’s about having a “3D desktop”. But the desktop I’m using is quite 2D-ish. There are people saying that 2D is just a special case of 3D. Looking at OpenGL that is certainly the case. As long as you don’t use any effect which needs the “3D aspect” of OpenGL, KWin just does 2D with OpenGL. Using OpenGL means to make use of hardware which is designed to do these operations instead of doing it on the CPU which is not designed for it. But what if one doesn’t have the hardware for it? A valid question, but just not in the context of KWin. If your system only provides software rendering KWin automatically falls back to XRender based compositing.

The last aspect to mention in the context of OpenGL is texture from pixmap. I don’t know how it’s implemented in the drivers so I don’t know whether that needs additional memory or not. But if you are concerned about it: XRender does not need a texture from pixmap.

So as you can see compositing comes with a little bit more memory usage but reduces CPU usage. It’s nothing I could say which one is generally better, but less CPU usage means a sleeping CPU. Granted some effects do animations – that also requires CPU usage. If you are concerned about that it’s better to disable the effects or set the global animation speed to instant than to just disable compositing completely.

And if you try to be “lightweight” and play with the compositing settings: one unrelated change can ruin all your benefit. Just imagine you would disable compositing because you want to be “lightweight” and then you enable a window decoration which uses QML (that is interpreted code) and does animations?

As it turned out that was KWin sending out notifications. Which immediately raised the question of why? Why would a user want a notification that he started/finished moving a window? After all it’s an action the user triggered. What should be done with the notification? Show a message? “You successfully moved a window!”, yes thank you I can see that on the screen. Play an annoying sound? Pling! Hopefully not.

Looking at what KNotify supports only logging to file or running a script make sense in response to the notifications emitted by KWin. But for logging to file it’s rather questionable why one would want that and why one would do that from inside a window manager. So what remains is running a script – fair enough that can be useful.

A closer look to the notifications emitted by KWin showed a few more things. None of them is configured to do something with the notification. By default KNotify just discards the notifications. This means we do a communication with kded via D-Bus for nothing. Two applications are getting woken up, context switches and so on for exactly nothing. Some notifications have a sound file connected to them, but are not configured to play that sound by default. We made a small quiz by me playing the sound file and letting the rest of the group guess what it’s for – I think nobody got one right. Nobody of the core workspace developers knew these sounds.

Looking at the code where the notifications are emitted highlighted another interesting fact. At all places we also emit a Qt signal which is mapped into the KWin scripting environment. And that’s actually quite awesome. Because unlike the notifications it has context. A KWin script does not only get informed about a window that gets moved, but about which window gets moved. Let’s say you are only interested in movements of Firefox windows – with KWin scripting that’s possible, with the notifications it isn’t. Given that we already identified that scripting (and maybe sound) is the only use case for the notifications emitted by KWin it’s getting more questionable why they are still around. Obviously they had been added long before we had KWin scripts. I was unable to really figure out when it got added, as git blame ends in one of the moving around code commits years ago (with more patience one can figure it out, but knowing that it’s old, was enough).

Of course we do not want to break our users’ workflow, so removing the feature is not easy. We have to properly judge whether the users’ workflow is valid enough to penalize all users by the additional overhead in code and in communication especially the context switches. Given that KWin scripting also allows to perform D-Bus calls, it should be possible to rework each of the previously existing notifications in KWin scripts.

That said: in case you were a user of one of those notifications and the removal breaks your workflow: please let me know. Please tell me how you used it and best provide your script. I will have a look at it and try to ensure that it’s still possible with KWin scripting (if it’s simple enough I will just port it).

For me Wayland support has been the primary goal I have been working on over the last two years. This doesn’t mean that there is actual code for supporting Wayland (there is – the first commit for Wayland support in our git repositories is from June 11, 2011 (!)).

The Wayland research projects two years ago had been extremely important for the further development of KWin since then. First of all it showed that adding support for Wayland surfaces inside KWin’s compositor is rather trivial. Especially our effect system did not care at all about X11 or Wayland windows. So this is not going to be a difficult issue.

The more important result from this research project was that it’s impossible to work against an always changing target. At that time Wayland had not yet seen the 1.0 release, so the API was changing. Our code broke and needed adjustments for the changing API. It also meant that we could not merge the work into our master branch (distributions would kill us), we needed to be on a different branch for development. Tracking one heavily changing project is difficult enough, but also KWin itself is changing a lot. So the work needed to be on top of two moving targets – it didn’t work and the branch ended in the to be expected state. Now with Wayland 1.0 and 1.1 releases the situation changed completely.

The next lesson we learned from that research project was that the window manager part is not up to the task of becoming a Wayland compositor. It was designed as an X11 window manager and the possibility that there would not be X11 had never been considered. We started to split out functionality from the core window manager interface to have smaller units and to be able to add abstractions, where needed, to support in future more than just X11. That had been a huge task and is still ongoing and it comes with quite some nice side-effects like the rewrite of KWin scripting (helped to identify the interface of a managed Client inside KWin), the possibility to run KWin with OpenGL on EGL since 4.10, the new screen edge system in 4.11 and many many more. All these changes were implemented either directly or indirectly with Wayland in mind. That means we have been working on it for quite some time even if it is not visible in the code.

My initial plannings for adding Wayland support around October/November last year was to start hacking on it in January. I was so confident about it that I considered to submit a talk for FOSDEM which would demo KWin running on Wayland. In the end I decided against it as it would have meant working on some of our very important foundations under time pressure, which I don’t think is good for maintainability of the code base.

In December though I decided to adjust my plans and focus first on the Qt 5 port as that would allow us to use the Qt Wayland bindings which are a little bit more convenient for usage in a Qt based application than the native C library. This is not something I just come up with, I discussed this small adjustment with a few people (for example Aaron Seigo) at the Qt dev days last year.

On January, 22nd 2013 sebas outlined the time line for the transition to Qt 5 and Wayland of the KDE Plasma Workspaces. It clearly states that KWin will become a Wayland compositor and that this is a target we are working on with a clearly defined time line.

Given that I am really surprised to see media writing again and again that we started to work on Wayland because other projects deciding against or for Wayland. It’s not something we decided on recently and it is quite clear that our work does not depend on any decisions or announcements our competitors do. We are an independent project, which does it’s own decisions for long term planning. The fact that our work now shifts towards Wayland just at the time our competitors decide for or against Wayland is pure coincidence.

Last week we had a Plasma developer sprint in Nuremberg (thanks to SUSE) and of course Wayland was an important topic for discussion. We had many points on the agenda. After all it’s the first sprint for us since we work on Plasma Workspaces 2 and it is needed to define our direction for the next year. Given all that I wrote so far it’s rather obvious that we would have discussed Wayland even if a certain project would not have done a certain announcement. Some topics on the agenda were based on discussions we had on the mailing list before the certain announcement was made (I’m currently sitting in a train with not sufficient Internet connection so I cannot look up the reference). Which also shows that we had long term plans which were decided on long before the ecosystem shifted.

Summary: we had been working on Wayland for years and it is our long term strategy. Our strategy is of course not based on any announcements of our competitors. We of course need to evaluate new solutions available in the ecosystem when they come up. As I already pointed out in the past, this is not yet possible at the moment and all we can see so far is that Wayland is a better solution for our needs in the KDE Plasma Workspaces than other windowing systems which might be tailored towards the needs of our competitor’s desktop shell. One of the advantage of free software projects is that the development and discussions are open and that it’s quite easy to reach out to the developers and architects of the software.

Plasma Workspaces 2 will be released at an interesting point of time. We don’t want to do the transition to Qt 5 and Wayland at the same time, so it will still be X based. But we all agree that our future will be on Wayland and even if we use X as the windowing system our primary focus is on Wayland. With Wayland quite a few things will change. KWin will play a more important role as it will be the Wayland compositor – we do not plan to use Weston.

Given that we know that the Wayland shell interface only covers part of what Plasma needs and some of our needs are extremely Plasma specific (for example Activities) it would be tempting to say that we tie KWin to Plasma. Let’s face it: which other compositor will be there to replace KWin? The reference compositor will probably never accept Plasma specific patches for things like Activities, Compiz won’t be ported to Wayland and GNOME Shell will probably never be a solution for Plasma. For the small window managers we do not know whether they will go to Wayland at all, but I expect rather not, though I expect that we will see Weston forks/extensions for substitutions of tiling window managers.

We decided to resist the temptation to go the easy road, but instead will develop all our integration bits in a way that one could replace KWin by a different Wayland compositor, even if that is just a theoretical option. Of course we will not do any fallback modes for the case that one is using e.g. Weston without Plasma integration bits. So the features which we need might then just be disabled. Adding fallback modes would most likely just result in bit rotting code as nobody would use it.

Of course to make it possible that others can provide compatibility features we need to properly document our extensions and additional interfaces. Luckily Wayland implicitly forces us to do so. The general plan is to publish our extensions and also try to standardize what makes sense to be standardized and we hope that this would also benefit other projects. What we especially had in mind is of course Razor-Qt which already supports using KWin. By properly documenting all our Plasma-KWin communication channel, they can also use what is useful to them and it ensures that we don’t break KWin in a way that it gets unusable for Razor-Qt.

I was considering various things like “being memory efficient” which I discarded for obvious reasons. First of all it’s difficult to measure memory usage correctly (I haven’t seen anyone, who provides numbers, doing it correctly, this includes especially Phoronix). And then it’s comparing mostly apples to oranges. Loading a high-resolution wallpaper might make all the difference in the memory usage. Also if desktop environment Foo provides features which are not provided by Bar it’s obvious that Foo uses more memory. But still it’s apples vs oranges. It’s not a comparison on memory, it’s a comparison of features. And of course one might consider the Time-memory-tradeoff.

So is it all about features? Obviously not. If there is a feature a user needs and uses it cannot be bloat. The fact that a desktop environment has only few features cannot be the key to being lightweight. Being evil now: many people say GNOME is removing features, but nobody would say that GNOME is lightweight.

What about support for old systems? That’s not lightweight, that’s support for old hardware. And it’s something which doesn’t make any sense given Moore’s law. Which raises the first question: what is old hardware? One year, two years, ten years? Is it a moving target or is a Pentium III for all time the reference? Optimizing for old hardware means not making use of modern hardware capabilities. But does that make sense to not use modern hardware if it is available? Using the GPU for things the GPU can do better than the CPU is a good thing, isn’t it? Parallelize a computation on multi-core if possible is a good thing, isn’t it? But if you do so, you are optimizing for modern hardware and not for old hardware. So saying you are good for old hardware, implies you are bad on new hardware? Also I’m wondering how one can optimize for old hardware? Developers tend to have new hardware to not have problems like this. And how can one keep support for old hardware when the complete stack is moving towards new hardware? Who tests the kernel against old hardware? Who provides DRI drivers for obsoleted hardware which doesn’t fit into modern mainboards (who remembers AGP or PCI)? Who ensures that software is still working on 32 bit systems, who would notice such a breakage for example in the X-Server? So lightweight cannot be fit for old hardware. And remember: optimizing for old hardware is not the same as optimizing for modern low-end hardware. Even the Raspberry Pi has a stronger CPU (700 MHz) than the oldest Pentium III (450 MHz) – not to mention things like OpenGL…

What’s it then? Let’s ask Wikipedia. For Xfce it tells us, that “it aims to be fast and lightweight, while still being visually appealing and easy to use”. Unfortunately there’s no link on lightweight and also no reference. Let’s try again, LXDE: “The goal of the project is to provide a desktop environment that is fast and energy efficient”. Again no real definition and just stating a goal. But it goes on:

LXDE is designed to work well with computers on the low end of the performance spectrum such as older resource-constrained machines, new generation netbooks, and other small computers, especially those with low amounts of RAM.

I have no idea what a “new generation netbook” is, but it sounds like something modern half a decade ago. But we are down to the “being good on old hardware”, which we just discarded. Interestingly Wikipedia has references to prove that LXDE is good on RAM – unfortunately it’s references to Phoronix. Shame on you, Wikipedia, that’s a “benchmark” which has been considered seriously flawed by people understanding the topic.

Ok Phoronix aside, there is one more of the lightweight desktops to check. Razor-qt: “is a lightweight, Qt-based free software desktop environment for the X Window System. Unlike the KDE desktop environment which is also based on Qt, Razor-qt usually performs well even when used in computers with older hardware”. Damn, it again only claims and no reference for anything. Who says that KDE desktop environment (what’s that again?) is not performing well on older hardware? And what does the “usually” mean in that sentence?

So all Wikipedia gives us, is buzzword bingo and various claims without any proof (and not even a “reference missing”). Let’s try different and go to the projects directly.

Xfce tells us that it “is a lightweight desktop environment for UNIX-like operating systems. It aims to be fast and low on system resources, while still being visually appealing and user friendly.” Which sounds very similar to what Wikipedia wrote. Unfortunately nothing that tells us what lightweight means or what fast is or low on system resources. Fuzzy is fuzzy.

LXDE also has the same definition as shown in Wikipedia (maybe we should mark those wiki articles as advertisement?) but at least gives us a definition for what lightweight means to them: “It needs less CPU and performs extremely well with reasonable memory.” Fuzzy is fuzzy. What’s “reasonable memory” and “less CPU”? It goes on about various other things like “fast” and “energy saving” (which even are contradicting, e.g. fast is defined as working on hardware from 1999 and not require 3D – how that can be energy saving on new hardware one has to show me). The point energy saving is quite funny as it just says they are better than “other systems”. Oh well fuzzy is fuzzy.

Last but not least: Razor-qt. Razor doesn’t say it’s lightweight:

Razor-qt is an advanced, easy-to-use, and fast desktop environment based on Qt technologies. It has been tailored for users who value simplicity, speed, and an intuitive interface. Unlike most desktop environments, Razor-qt also works fine with weak machines.

Fuzzy is fuzzy. What’s a “weak machine”, what are the “most desktop environments”? Where’s the proof for other DEs not working well on those not-defined “weak machines”?

And here, poor fool! with all my lore I stand, no wiser than before. All what I could derive from studying what lightweight means is, that one just has to claim to be lightweight. Bonus points if you include it into your name. It seems like if you repeat it often enough people will repeat it and it will turn out into truth.

The effects are so similar that they even share the code to lay out the windows. Nevertheless there are quite some differences in the functionality. The following features are only available in Present Windows:

• Filtering of windows
• Icon and caption on each window
• Closing windows through a button shown on hover
• Enlarging windows on hover
• Mouse Actions
• External activation (used by Netbook Shell)

On the other hand there are features in Desktop Grid not available in Present Windows:

• Drag windows from one screen to another
• Add/remove virtual desktops

The difference in functionality cannot be explained – there is just no reason for it. This is clearly a problem for our users – the effects look identical, but they behave differently. From all I learned about usability this is pretty bad because it teaches users to not trust systems.

The obvious answer would be to merge those two effects. But here we have a problem. Present Windows has been implemented to be Present Windows – no matter how we tweak it, it will be Present Windows. And Desktop Grid has been implemented as a Desktop Grid – no matter how we tweak it, it will be Desktop Grid.

I know that people will tell me that nothing is impossible. Of course one could hack around and make the effects be the same. But we have been there and tried to bring non-Present Windows functionality into Present Windows. I once implemented an Alt+Tab mode in Present Windows – a huge mistake. All it did was adding lots and lots of special case handling to make it work. And it castrated the effect. It only supported one screen (needed of course special hacks), it didn’t support filtering (keyboard grab hold by Alt+Tab framework) and so on and on. It never made any sense and I am very glad that I realized this error and removed the Alt+Tab mode again.

But also with Desktop Grid we made the experience by adding the layout mode of Present Windows. It wasn’t good for the code base. Nowadays, after I implemented that addition, I compare the code base of Desktop Grid effect to a Jenga tower which is already shaking and I fear that if I touch it to change something it will completely collapse.

A merge would only be possible by removing features and that is always difficult. People were really upset when I removed the Alt+Tab mode of Present Windows. Arguments do not work, giving replacements does not help and trying to improve the workflow is of course not welcome.

What remains would be a complete rewrite from scratch. But that’s a task so large that I consider it as a possibility for a Google Summer of Code project. Together the two effects have more than 4000 lines of code. It’s nothing we could do in parallel to our normal work and rewriting as a KWin Effect does not really make sense. Yes one can use Effects for that but it’s rather limited.

One of the most clear disadvantages is that Effects require Compositing. While having always composited systems becomes more and more the reality, there is still no reason why there should be the break in the user workflow when turning off Compositing and I have had many discussions with users who did not understand that.

But the real problem is that we do not have a toolkit inside the Effects. We are down to key press/release events and mouse motion/press/release events. That’s rather limited. For the close button inside Present Windows we do quite some hacks to get the mouse events injected into the QWidget. It’s a hack and nothing that scales. Key events are not composed correctly – for example on my English keyboard I’m not able to write letters like “ä, ß” in Present Windows. But the real problem is the handling of things like the drag and drop of windows in Desktop Grid. The code is extremely nested (seven indentation levels) and has to cover many, many corner cases. It just shows how difficult it is to get that handled correctly with different modes.

But since recently we have a technology which solves many of these problems: QML. With QML we get a toolkit which we can use to properly interact with the windows and which is supported in both composited and non-composited setups. In combination with our KWin scripting framework we have capabilities which are quite similar to what our effects can do. As a nice side-effect QML forces us to separate logic from visualization which means that the logic of Present Windows/Desktop Grid could be reused and combined with new ideas of how to represent the data.

This week the logic part entered KWin’s source repository in form of a QAbstractItemModel which is exported to QML. The model is quite flexible and allows to sort the windows in multiple levels – currently it supports screens, virtual desktops and activities. In addition one can specify whether a window should be excluded. That’s for example important to have the different modes of Present Windows – there is one mode for only current desktop and one mode for only windows of the currently active application.

Having a model makes it rather easy to support filtering – we just need to connect it to a QSortFilterProxyModel and such a model is already provided and exported to KWin’s QML binding.

Implementing the logic for Desktop Grid inside QML becomes now as simple as:

ClientFilterModel {
    id: filterModel
    clientModel: ClientModelByScreenAndDesktop {
        exclusions: ClientModel.NotAcceptingFocusExclusion | ...
    }
    filter: filterItem.text
}

The idea is now to use this as a base for what I call “Present Windows 2″. It’s nothing I target for 4.11, because of the complexity of Present Windows and Desktop Grid. I hope that many things become easier once we are on Qt 5 and QtQuick 2. So it’s something I want to have for Plasma Workspaces 2. I’m still wondering whether to optionally include the current proof of concept QML script which renders a screen like this:

Aus KWin

One can already see the improvements like an always visible filter area. That’s one of the real disadvantages of the existing Present Windows effect: one doesn’t get any hint that it is possible to filter the shown windows. Also a clear improvement in this variant is the always visible close button for each of the windows.

The new models included in KWin might also get new tasks I haven’t thought about when I started to work on it. Our Plasma developers noticed that this would be a rather good base for the tasks applet. Whether that’s possible or not we do not yet know, but it would have clear advantages to have a shared code base for all the different ways of getting an overview of open windows.

(Sorry for having once again written such a long blog post)

I got the memo about Linux is for normal users and not for LEET, but I do not understand why it has to be so difficult to setup a device which is meant for development. In the past it was as simple as dd an image to an SD-card, plug it in and done. Well those times are over.

My requirement for a base distributions are rather small:

• Up to date kdelibs, because compiling is slow on the device
• Working drivers for GL hardware

With this combination we can rule out most distributions like for example Debian (issues with both) or openSUSE (no drivers). I decided to try Linaro 13.02 which offers an image for Pandaboard and is Ubuntu 12.10 based, which means we can easily install KDE packages.

Linaro is still rather simple: dd to SD card, plug in and go. Just that you don’t get any output on the screen. I already thought my Pandaboard was broken. What’s a little bit tricky is that the Pandaboard has two HDMI connectors (one as HDMI, one as DVI) and at least openSUSE reports that only the real HDMI works. But with Linaro I did not get anything on either screen or TV.

So I had to connect to the Serial port to get some output. And look there: it boots. Once I was logged in I was able to figure out that the system is pretty basic, e.g. no X installed. But even after installing X I did not get anything on the screen: it complained about missing /dev/fb0. That was then the point where I considered trying a different distribution. (Search did not help).

Next choice was Kubuntu. This also used to be rather simple: dd to SD card. Downloaded daily build of 13.04, dd to SD card, plugged in and screen turns on. But instead of starting the system, the installer is started. Well since 12.10 you need to install. Obviously the system is not able to install to the SD card which is plugged in. So I got an USB stick, dd to stick, plugged in and nothing. The pandaboard doesn’t boot from USB stick. Now it got difficult: search for a second SD card. Found one, dd image to the second card, moved the first SD card to a card reader, plugged everything in and installed to the SD card at the card reader.

After installation, I swapped the SD cards, plugged in and nothing. System doesn’t boot. Well maybe the system expects at a different device, so I plugged the card reader in again, tried to boot and nothing.

OK, I installed a daily built of an unreleased version, so maybe I should try 12.10. Not so up to date kdelibs, but probably good enough for KWin. Did the same procedure again, installed and doesn’t boot.

Now what? I could try a base Ubuntu instead of Kubuntu. But as the installer is to my knowledge shared I do not expect that it will work. And also I’m too scared that Ubuntu will come without drivers and with Compiz. I don’t want llvmpipe on the desktop, I definitely don’t want llvmpipe on ARM. And I had seen Compiz with drivers on the pandaboard – so no that’s too scary.

While searching for a solution to the problems I was facing, I also stumbled over Ubuntu Core. It didn’t look like anything I want to try, but at this stage I was at the point to decide whether I use a distro like openSUSE without drivers or try something which might give me drivers.

Now with the linked wiki page I would not have been able to install it, because it’s an interesting system. Luckily the omappedia contains an instruction. Basically the system comes with nothing. No editor, no network, no console, no way to log in, just nothing. But even with the introduction it gets interesting. For example it links a PPA which is not available for quantal or raring. Do I not need it anymore? Will my system be broken afterwards? (it turned out that the packages are nowadays included).

The most interesting aspects were that it doesn’t start with networking enabled. The wiki page lists a command to start networking, but I did not manually adjust resolv.conf because I thought DHCP would set it. Well it didn’t and in the running system you cannot change it because there’s no editor.

After quite some fiddling with the system I finally was at a state where startx succeeded and loaded into the KDE Plasma desktop. Looked good even compositing working OOTB (of course only XRender). So I enabled my keyboard/mouse combo and nothing. Searching for this problem recommended to check whether the usb module is loaded. Yeah, come on, I’m not stupid. Well better be sure:

lsmod | grep usb

nothing. Hmm get’s interesting. Modprobe fails, depmod fails – /lib/modules/ does not exist. The linux image was not installed. That was the point where I decided to install full kubuntu-desktop instead of just kde-workspace. The device is slow, it took a few hours.

By now I have the system running and KWin master built from source (took another few hours). It looks like it has drivers for the system. At least a shipped demo application for OpenGL ES 1.1 is working – the demo application for OpenGL ES 2.0 is failing because it has a hard coded path to the shader file. Installing the driver was also fun, it comes as a DKMS package and the first installation try failed with a crashing g++. That’s something I already knew from my last try with the pandaboard. Gcc might crash, restarting solves it.

Unfortunately I don’t have KWin running on OpenGL ES yet on this hardware. It fails somewhere in the init buffer configs code. But it was already worth the effort. The code is already extended with much more error checks as I needed to find where exactly it is failing. Also compiling on ARM found a few more C++11 narrowing conversion warnings (floats). Given that we want to enable C++11 for KWin in 4.11 it’s good to know that there is still some work to do on non x86 hardware. Would be nice to get that CI covered.

Last week we had to investigate one issue which quite surprised me. On Sunday a cleanup to our GLX initialization code was pushed to master. This had a small regression preventing compositing to work on NVIDIA. What surprised me is that nobody noticed till Thursday, when I setup an old system with NVIDIA to test something completely different. Is nobody running master any more or are our KDE developers no longer using the NVIDIA driver? Overall it’s not good that we have regressions in master for four days without anybody noticing. And no: CI cannot catch such issues.

Summary

Crash Fixes

• 315528: KWin crashes when switching windows
  This change will be available in version 4.11
  Git Commit

Critical Bug Fixes

Bug Fixes

• 259640: Deleted windows not unrefed when restarting compositing
  This change will be available in version 4.10.2
  Git Commit
• 317068: keepInArea does not work if window dimensions match the area dimensions
  This change will be available in version 4.10.2
  Git Commit
• 305781: Option “Suspend desktop effects for fullscreen windows” makes the image freeze when going fullscreen in applications
  This change will be available in version 4.10.2
  Git Commit
• 316040: Dual screen issue caused by d6b3f6983efebc42abd6028ece9c3ec7facea2d0
  This change will be available in version 4.11
  Git Commit
• 283309: Attaching windows to activities is too clumsy
  This change will be available in version 4.10.2
  Git Commit
• 316033: Switching from one aurorae theme to another results in no decoration
  This change will be available in version 4.10.2
  Git Commit
• 314532: Plastik (QML) decoration: poor performance with desktop effects
  This change will be available in version 4.10.2
  Git Commit
• 296076: Fix fullscreen state handling: NETWM says it’s bound to focus and not stacking order, also see bug #224600
  This change will be available in version 4.11
  Git Commit
• 313061: “Login” effect does not fade in on secondary monitor(s) with a multi screen setup
  This change will be available in version 4.11
  Git Commit
• 307965: Upper part of windows tears when moving it left/right ONLY in upper part of display
  This change will be available in version 4.11
  Git Commit
• 299245: Get rid of “Display borders on maximized windows” setting
  This change will be available in version 4.11
  Git Commit
• 237260: shortcuts for switching windows don’t work with multihead
• 317025: Java Swing Apps do not receive Deiconify event if window is shaded
  This change will be available in version 4.11
  Git Commit
• 91703: vertical maximization doesn’t work as expected
  This change will be available in version 4.11
  Git Commit

New Features

Tasks