Posted on

KWin running in Weston

This week I decided to do some research for the Wayland porting of the KDE Plasma workspaces. One of the features we will need in future is a Wayland session compositor which runs nested on a Wayland system compositor. Of course one could think of setups without a system compositor, but overall I think that a nested compositor simplifies the setup and allows to have all the low level technologies in one place without duplication in all the various compositors. +1 for working together.

After three days of work I already have something to demo (video on youtube):

Sorry for the bad audio. I’ll just explain what one can see. The video starts with the normal X-Server. After that it switches to a VT and we start Weston there. On Weston I’m starting KWin with some environment variables set to pick the correct libraries and force KWin into Wayland mode. KWin creates a connection to Wayland, creates a Wayland surface and uses it for OpenGL output. All the windows from the running X Server are rendered into this surface just as if it were a normal X11 output.

KWin also gets input from Wayland and passes it to the X Server. That’s the reason why we can see mouse interaction and working keyboard.

How it works

The OpenGL backend

KWin supports multiple backends for providing an OpenGL context and doing the texture from pixmap operation. At the moment we have an GLX and an EGL backend. Both create the OpenGL context on the XComposite overlay window and provide the texture from pixmap in the GLX case through the GLX_EXT_TEXTURE_FROM_PIXMAP extension and in the EGL case through the the EGL_KHR_image_pixmap extension.

A new backend is added which creates the OpenGL context on a Wayland surface. The backend started as a fork of the existing EGL backend with the X code stripped out. What’s a little bit tricky is getting the texture from pixmap working. The extension used in the normal EGL on X11 backend is not available. The proper solution would be a setup with XWayland, but that’s still too early as KWin does not yet support Wayland clients.

The solution I came up with is inspired by a fallback mechanism in KWin from the time when GLX_EXT_TEXTURE_FROM_PIXMAP was not guaranteed to be around: XShm to copy the pixmap content into an OpenGL texture. Not a nice solution but it works.

Input handling

Input is currently also a rather hackish solution until we have XWayland up and running. We just take all input events and forward them to the XServer with the XTest extension to inject fake events. It’s a huge hack and one can see how old X is there and how limited. I was rather surprised that it works at all. At the time of this writing the code supports keyboard events and the left, middle and right mouse button. Wheel events are tricky as X uses mouse buttons for them and for more mouse buttons I have problems with mapping them as I’m lacking a multi button mouse.

We are also not able to back sync the mouse position from X to Wayland. As far as I understood the Wayland protocol there is nothing like XWarpPointer, so if something in X warps the pointer we have a mismatch. I agree that warping is evil, but we use it in KWin for activating the screen edges 🙁

Next Steps

Cursor

What is to do next is to get the changes to the cursor in X11 and set the cursor on the Wayland surface. That should not be really difficult as the XFixes extension provides everything one would need for that.

Thread

A rather huge limitation at the moment is that the connection to the Wayland display is hold in the main thread. We cannot block there, so we only get events when we actively check for them. This is currently during repainting the screen. So if you wondered why the ShowFPS effect in the video is turned on: it’s to force repaints and to keep the connection alive. This connection needs to go into a thread so that we can block there.

Buffer age

Currently the code forces as to do full-screen repaints. The two solutions we have for non-fullscreen repaints in the EGL backend do not work in the EGL on Wayland backend. The Wayland demo code shows that the EGL_EXT_buffer_age implementation could be used. We wanted to have support for that one anyway in KWin.

Giving it a try

Building from source

I just pushed the code into branch “kwin/wayland-egl-backend” on my clone kde:clones/kde-workspace/graesslin/kde-workspace. Be aware that I intend to force push to this branch.

To build you need to have the Wayland libraries around. The CMake module tries to find it through package kit. Watch the output of CMake, Wayland is only an optional dependency! If you build Wayland and Weston from source, please follow the instructions. You might also need to build Mesa from source.

How to start

Starting is rather simple. Just have an Xserver running somewhere, start Weston on a VT (don’t do nested on X, you would only get a black screen once KWin started) and open a terminal. Setup your environment variables to be able to start KWin and then start KWin in the following way:

DISPLAY=:0 KWIN_DIRECT_GL=1 KWIN_OPENGL_INTERFACE=egl_wayland kwin --replace &

KWIN_DIRECT_GL makes KWin skip the OpenGL Test application which is currently only supporting GLX and seems to fail if one is on a different VT. KWIN_OPENGL_INTERFACE tells KWin which backend to use. So by using “egl_wayland” one forces KWin to use the Wayland backend. In future we will probably detect whether the Wayland display environment variable is defined and just pick it directly (though that is dangerous in case of nested Weston on X).

When will it be available in a release

I want to get this work into 4.11 as an experimental feature for multiple reasons. I wanted to have a build dependency to Wayland in 4.11 for quite some time. So that’s a nice excuse. Unfortunately feature freeze is approaching.

Of course I want to give users something to play with. We have talked about Wayland for such a long time and there is nothing to really see that we are walking the way. Last but not least that might be a rather important solution for Plasma Active as that hopefully allows to run on hardware where we currently would not get OpenGL. With libhybris we should be able to get KWin working with OpenGL and being an X11 compositor even if the driver doesn’t support X11. Though I have not tested this case yet and have no devices to play with. But I know a few people in the community who might be interested to play with that.

Posted on

Mir in Kubuntu

As you might have seen in Jonathan’s blog post we discussed Mir in Kubuntu at the “Mataro Sessions II”. It’s a topic I would have preferred to not have to discuss at all. But the dynamics in the free software world force us to discuss it and obviously our downstream needs to know why we as an upstream do not consider Mir adoption as a valid option.

This highlights a huge problem Canonical created with Mir. I cannot just say “Canonical sucks”[1] to discard Mir as an option, I have to provide proper technical arguments why we won’t integrate Mir. I have to invest time to investigate the differences, advantages and disadvantages. As I have those arguments, I thought it might be a good idea to share them in a blog post.

The discussion started during a presentation about X11 and Wayland to my fellow team mates at Blue Systems. I decided to first explain X11 as I think one cannot understand the needs for Wayland without understanding X11. I did not intend to discuss Mir at all, but somehow the discussion drifted into the direction and the valid questions were raised about what are the differences and advantages of Mir or Wayland. What followed was kind of a rant about Ubuntu and Canonical [2]. So later the week we discussed “Mir in Kubuntu” in more detail to try to find answers to the many questions this raises for our downstream.

Introduction

Frustration and lost Motivation

Before I go into more detail I want to make one thing clear: Canonical is totally allowed to develop whatever they want. I’m totally fine with this and don’t care whether they develop another display server, an own os kernel or yet another desktop shell. I couldn’t care less. It’s Canonical/Mark’s money and he can invest it in any way he considers as useful. I wouldn’t even care if it would be proprietary software, that’s all fine.

What is not fine is causing a major disruption in the free software ecosystem by giving false technical arguments and doing bold statements about software Canonical does not contribute to. This is not acceptable. This was very frustrating and destroyed lots of trust I had in Canonical. It will be difficult to rebuild this trust. Canonical can be glad that it is the free software world and not the normal corporate world. There were quite some statements which could have raised the legal department in the normal corporate world[3]. It also cost lots of motivation at least on my side and I even questioned whether it’s still worth to be a member of the free software ecosystem. Instead of working together we now have a situation where members of the ecosystem become a competitor and which badmouth part of the software stack. A very frustrating situation.

There certainly are valid reasons for developing Mir which also make sense. Unfortunately they have not been presented so far. I’m quite sure that I know the reasons and if they would have been said straight away it would have been for me and other projects probably much easier. It would have taken away the frustration which the announcement caused and we would not need to discuss it at all, because those question marks would not exist. But apparently Canonical decided to give false technical arguments over the real ones.

Not ready yet

At the moment Mir is not there yet, this is important to remember. With the announcement we basically had four options on how to handle the situation.

  1. Continue with the Wayland plan and ignore Mir
  2. Switch to Mir and ignore Wayland
  3. Support Mir and Wayland
  4. Delay decision until Mir is ready

If I map our time line for Plasma Workspaces 2 against the time line of Mir I see no overlap. We want to support Wayland before Mir is ready. So delaying the decision would be a rather bad idea. It would just throw us back. This also means that option 2 is not valid especially as we would need to delay until Mir is ready for this to happen. So the only valid options are supporting both Mir and Wayland or only Wayland. At the moment the code is not ready yet to properly decide whether supporting Mir in addition to Wayland is a valid approach or not. Last time I checked the source base I hit a few stubs and then obviously stopped looking at the code as it’s not worth the effort yet. So we have to evaluate on the knowledge we already have and that doesn’t look good on the Mir side.

Wayland vs Mir

Possible Advantages of Mir over Wayland

The differences between Mir and Wayland are rather minimal. One of the differences is that Mir uses server allocated buffers while Wayland uses client side buffer allocation. I cannot judge whether this is an advantage or disadvantage. But I trust Kristian and the Wayland team more on that topic.

Another difference is that Mir uses test-driven development. To me development methodology is not a technical argument. I rather use a working system without unit tests than a system with unit tests that doesn’t work [4]. Also KWin does not use TDD. If I would consider TDD superior I would have to question my own development methodology.

But that’s it. That are the differences I found so far which could count as an advantage for Mir. But of course there is the advantage that Mir is going to be awesome. For the disadvantages I will spend a complete section on each point.

Distro specific

So far Mir is a one-distribution solution. So far no other distribution has shown any interest in packaging Mir even if it would become a working solution. Unfortunately I don’t have the ability to see into the future, but I can use the past and the present to get ideas for the future. The past tells me that there are other Canonical specific solutions which are not available in other distributions. I do not know of any distribution which packages Unity and from all I have heard it’s even impossible to package Unity on non-Ubuntu distributions. Given that it is quite likely that Mir will go the same road. It’s designed as a solution for Unity and if distros don’t package Unity there is no need to package Mir.

This has quite some influence on a possible adoption. I do not know of any kde-workspace developer using (K)Ubuntu. I do not see how anyone would work on it or how we should be able to review code or even maintain code. It would mean all the adoption would have to go into ifdef sections nobody compiles and nobody runs. This is the best way to ensure that it starts to bit-rot. Even more our CI system runs on openSUSE so not even the CI would be able to detect breakage. Of course a downstream like Kubuntu could develop the adoption and carry it as a patch on top of upstream, but I would highly recommend them to not do this as KWin’s source code churn is too high. Also we all agree that downstream patches are evil and we would no longer be able to help in any way downstream’s user from a support perspective.

Architecture

Mir’s architecture is centered around Unity. It is difficult to really understand the architecture of Mir as the specification is so full of buzz-words that I don’t understand it [5]. From all I can see and understand Unity Next is a combination of window manager and desktop shell implemented on top of Mir. How exactly this is going to look like I do not know. Anyway it does not fit our design of having desktop shell and window manager separated and we do not know whether Mir would support that. We also do not know whether Mir would allow any other desktop shell except Unity Next, given that this is the main target. Wayland on the other hand is designed to have more than one compositor implementations. Using KWin as a session compositor is an example in the spec.

License

Wayland is licensed like X under the MIT license, which served us well for a display server. I think this is a very good choice and I am glad that the Wayland developers decided for this license. Mir is licensed under GPLv3-only with CLA. I think this is very unsuited for such a part of the stack and would render quite a risk for usage in KDE Plasma. KWin (and most KDE software) is GPLv2-or-later, this would no longer be possible, it would turn our code into GPLv3-only as KWin (or any other software which would depend on mir-server) would be a derived work of Mir. I do not consider GPLv3-only software as a possible dependency of any core part of our application stack. It renders a serious threat for the future in case of a GPLv4 which is not compatible with GPLv3. I also dislike the CLA [6]. So from a licensing perspective Mir is hardly acceptable.

Unity Specific/No Protocol

One of the most important aspects from Wayland for us is the ability to extend the protocol. This has already been a quite important feature in X and we are using our own extensions over ICCCM and EWMH to implement additional functionality. Of course our workspace has own ideas and it is important for us to be able to “standardize” those and also make them available to others if they are interested. This is possible thanks to protocol extensions.
Mir doesn’t have a real protocol. The “inner core” is described as “protocol-agnostic”. This renders a problem to us if we would want to use it. Our architecture is different (as described above) and we need a protocol between the desktop shell and the compositor. If Mir doesn’t provide that we would need to use our own protocol. And that already exists, it is called “Wayland”. So even if we would support Mir, we would need the Wayland protocol?!? That doesn’t make any sense to me. If we need to run Wayland on top of Mir just to get the features we need, why should we run Mir at all?

But it gets worse, the protocol between Mir server and Mir clients is defined as not being stable. In fact it’s promised that it will break. That’s a huge problem, I would even call it a showstopper. For Canonical that’s fine – they control the complete stack and can just adjust all bits using the protocol like QMir.

For us this looks quite different. Given that the protocol may change any time and given that the whole thing is developed for the needs of Unity we have to expect that the server libraries are not binary compatible or that old version of the server libraries cannot talk with the latest client libraries. We would constantly have to develop against an unstable and breaking base. I know that this sounds overly pessimistic but I know of one case where a change got introduced in a Canonical protocol late in the release cycle completely breaking an application in Kubuntu which wanted to use the protocol. Given this experience I would not trust that the protocol doesn’t change one day before the release meaning that Kubuntu cannot ship.

This is not awesome, it’s awful. It means KWin will not work just fine on Mir.

I hope this shows that using Mir inside the KDE Plasma workspaces is not an option. There are no advantages which would turn Mir into a better solution than Wayland and at the same time there are several showstoppers which mean that we cannot integrate Mir – not even optionally in addition to Wayland. The unstable protocol and the licensing choice are clearly not acceptable.

What this means to Kubuntu

Question marks

For Kubuntu the Mir switch by Canonical created quite some questions. One of those questions is answered: Upstream has no interest in supporting it and would most likely not accept patches for support. With upstream not using Mir the question is how the graphics stack for Kubuntu will look like once Ubuntu switched to Mir? The questions cannot be answered right now but it doesn’t look good.

Patches to the stack

Ubuntu has always had one of the worst graphics stack in the free software world. I can see this in the bug tracker. The quality of the Mesa stack in Ubuntu is really bad. For Mir Ubuntu will have to patch the Mesa stack even further. This is nothing which I would like to see. Also Mesa needs to be packaged with Wayland support. But will Canonical continue to do this? If not, would Kubuntu (and other Ubuntu flavors) need to ship their own Mesa stack? What if the changes by Canonical are so large that a standard Mesa stack doesn’t run on top of the Ubuntu stack?

Switching Sessions

One of the advantages of free software is that one can select the desktop environment in the login manager. This looks like no longer be possible in a Mir world. Unity will run with a Mir system compositor with LightDM nested underneath. We will need either the X Server or a Wayland system compositor. So from the login manager it will not be possible to start directly into a session using a different system compositor. How will it continue to be possible to use both Unity and KDE Plasma on the same system? Running a Unity and a KDE Plasma (or GNOME or XFCE or anything) session at the same time seems to no longer be possible.

System Compositor

How deep into the system is the system compositor going to be? Will it be possible to disable the Mir system compositor and replace it with X or Wayland? What if the packages start to conflict? Will it still be possible to install Kubuntu and Ubuntu on the same system? Will Canonical care about it? Will the system compositor mean that one has to decide in Grub whether to boot Ubuntu or Kubuntu?

Packages from Where

So far X, Wayland and Mesa have been packaged by Canonical. But what about the future? Will there still be packages for X, will there be packages for Wayland? If not, where to take them from? Debian unstable, most likely. But Debian might be frozen. Will it be possible at all to use the Debian packages for X and Wayland in the Ubuntu stack? Will they meet the requirements for KDE Plasma[7]? If Canonical doesn’t provide Wayland packages, they would drop to universe, so Mesa in main cannot depend on them. How to get then Mesa with Wayland support?

Only Future can tell

Those questions cannot be answered right now. It will have to wait until Mir is integrated into the Ubuntu stack. Then Kubuntu developers will see how far the stack broke. I’m not really optimistic that it will still be possible to provide the Ubuntu flavors once the transition to Mir is done. I don’t think that Canonical has any interest in the community provided distributions on top of Ubuntu any more. There are many small changes in the direction which indicate that. But we will see, maybe I’m too pessimistic.

[1] Given how Canonical introduced Mir with incorrect information about Wayland I consider this as a valid approach to dismiss the technology.

[2] I was very fed up with Ubuntu at the time anyway because our bug tracker once again exploded after the Ubuntu release.

[3] I do admit that I thought about asking KDE e.V. to send an Abmahnung after the statement that KWin would just work fine on Mir.

[4] In fact I consider TDD as utter non-sense and as a useless methodology though some aspects are useful.

[5] “with our protocol- and platform-agnostic approach, we can make sure that we reach our goal of a consistent and beautiful user experience across platforms and device form factors”

[6] Yes I know that Qt also has a CLA, which I have signed. But for Qt there is also the KDE Free Qt Foundation agreement.

[7]Last week a feature hit KWin which I cannot test/use because the X-Server is too old in Debian testing.

Posted on

The History on Wayland Support inside KWin

Ever since a certain free software company decided to no longer be part of the larger ecosystem, I have seen lots of strange news postings whenever one of the KDE workspace developers mentioned the word “Wayland”. Very often it goes in the direction of “KDE is now also going on Wayland”. Every time I read something like that, I’m really surprised.

For me Wayland support has been the primary goal I have been working on over the last two years. This doesn’t mean that there is actual code for supporting Wayland (there is – the first commit for Wayland support in our git repositories is from June 11, 2011 (!)).

The Wayland research projects two years ago had been extremely important for the further development of KWin since then. First of all it showed that adding support for Wayland surfaces inside KWin’s compositor is rather trivial. Especially our effect system did not care at all about X11 or Wayland windows. So this is not going to be a difficult issue.

The more important result from this research project was that it’s impossible to work against an always changing target. At that time Wayland had not yet seen the 1.0 release, so the API was changing. Our code broke and needed adjustments for the changing API. It also meant that we could not merge the work into our master branch (distributions would kill us), we needed to be on a different branch for development. Tracking one heavily changing project is difficult enough, but also KWin itself is changing a lot. So the work needed to be on top of two moving targets – it didn’t work and the branch ended in the to be expected state. Now with Wayland 1.0 and 1.1 releases the situation changed completely.

The next lesson we learned from that research project was that the window manager part is not up to the task of becoming a Wayland compositor. It was designed as an X11 window manager and the possibility that there would not be X11 had never been considered. We started to split out functionality from the core window manager interface to have smaller units and to be able to add abstractions, where needed, to support in future more than just X11. That had been a huge task and is still ongoing and it comes with quite some nice side-effects like the rewrite of KWin scripting (helped to identify the interface of a managed Client inside KWin), the possibility to run KWin with OpenGL on EGL since 4.10, the new screen edge system in 4.11 and many many more. All these changes were implemented either directly or indirectly with Wayland in mind. That means we have been working on it for quite some time even if it is not visible in the code.

My initial plannings for adding Wayland support around October/November last year was to start hacking on it in January. I was so confident about it that I considered to submit a talk for FOSDEM which would demo KWin running on Wayland. In the end I decided against it as it would have meant working on some of our very important foundations under time pressure, which I don’t think is good for maintainability of the code base.

In December though I decided to adjust my plans and focus first on the Qt 5 port as that would allow us to use the Qt Wayland bindings which are a little bit more convenient for usage in a Qt based application than the native C library. This is not something I just come up with, I discussed this small adjustment with a few people (for example Aaron Seigo) at the Qt dev days last year.

On January, 22nd 2013 sebas outlined the time line for the transition to Qt 5 and Wayland of the KDE Plasma Workspaces. It clearly states that KWin will become a Wayland compositor and that this is a target we are working on with a clearly defined time line.

Given that I am really surprised to see media writing again and again that we started to work on Wayland because other projects deciding against or for Wayland. It’s not something we decided on recently and it is quite clear that our work does not depend on any decisions or announcements our competitors do. We are an independent project, which does it’s own decisions for long term planning. The fact that our work now shifts towards Wayland just at the time our competitors decide for or against Wayland is pure coincidence.

Last week we had a Plasma developer sprint in Nuremberg (thanks to SUSE) and of course Wayland was an important topic for discussion. We had many points on the agenda. After all it’s the first sprint for us since we work on Plasma Workspaces 2 and it is needed to define our direction for the next year. Given all that I wrote so far it’s rather obvious that we would have discussed Wayland even if a certain project would not have done a certain announcement. Some topics on the agenda were based on discussions we had on the mailing list before the certain announcement was made (I’m currently sitting in a train with not sufficient Internet connection so I cannot look up the reference). Which also shows that we had long term plans which were decided on long before the ecosystem shifted.

Summary: we had been working on Wayland for years and it is our long term strategy. Our strategy is of course not based on any announcements of our competitors. We of course need to evaluate new solutions available in the ecosystem when they come up. As I already pointed out in the past, this is not yet possible at the moment and all we can see so far is that Wayland is a better solution for our needs in the KDE Plasma Workspaces than other windowing systems which might be tailored towards the needs of our competitor’s desktop shell. One of the advantage of free software projects is that the development and discussions are open and that it’s quite easy to reach out to the developers and architects of the software.

Posted on

The relationship between Plasma and KWin in Workspaces 2

Yesterday during the Tokamak 6 sprint in Nuremberg we discussed the role of KWin in Plasma Workspaces 2. At the moment in Plasma Workspaces 1 KWin is of course the recommended window manager and compositor, but it’s also possible to use a different window manager. Back in the days there were quite a lot of users who run Plasma with Compiz. In theory that shouldn’t matter because everything is standardized with EWMH and ICCCM. Over the years we added more and more extensions to EWMH. It’s all open source so anyone can implement these extensions (Compiz used to do so), nevertheless right now there is probably no other window manager available to offer the full experience except KWin.

Plasma Workspaces 2 will be released at an interesting point of time. We don’t want to do the transition to Qt 5 and Wayland at the same time, so it will still be X based. But we all agree that our future will be on Wayland and even if we use X as the windowing system our primary focus is on Wayland. With Wayland quite a few things will change. KWin will play a more important role as it will be the Wayland compositor – we do not plan to use Weston.

Given that we know that the Wayland shell interface only covers part of what Plasma needs and some of our needs are extremely Plasma specific (for example Activities) it would be tempting to say that we tie KWin to Plasma. Let’s face it: which other compositor will be there to replace KWin? The reference compositor will probably never accept Plasma specific patches for things like Activities, Compiz won’t be ported to Wayland and GNOME Shell will probably never be a solution for Plasma. For the small window managers we do not know whether they will go to Wayland at all, but I expect rather not, though I expect that we will see Weston forks/extensions for substitutions of tiling window managers.

We decided to resist the temptation to go the easy road, but instead will develop all our integration bits in a way that one could replace KWin by a different Wayland compositor, even if that is just a theoretical option. Of course we will not do any fallback modes for the case that one is using e.g. Weston without Plasma integration bits. So the features which we need might then just be disabled. Adding fallback modes would most likely just result in bit rotting code as nobody would use it.

Of course to make it possible that others can provide compatibility features we need to properly document our extensions and additional interfaces. Luckily Wayland implicitly forces us to do so. The general plan is to publish our extensions and also try to standardize what makes sense to be standardized and we hope that this would also benefit other projects. What we especially had in mind is of course Razor-Qt which already supports using KWin. By properly documenting all our Plasma-KWin communication channel, they can also use what is useful to them and it ensures that we don’t break KWin in a way that it gets unusable for Razor-Qt.

Posted on

War is Peace

Today I got many questions about KWin and Mir, how it affects us, what it means for our Wayland plans and so on. I did not want to write anything about it because I think there is nothing to write about, but before answering the same question again and again I think it’s better to put down a few lines here. Wiki will be updated once Wayland wiki is updated so that we have something to link to.

First question: Does Mir affect us? Yes, obviously. Because of Mir I have to write this blog post, Wayland developers have to get the FUD out of the Mir documentation, it’s creating tension and it harms the development. We will have to face again and again the question whether Wayland is better or not. So yes it affects us and I’m not happy about it.

Second question: Does it affect our plans for Wayland? I think it would be very unprofessional if we would change our plans just because Canonical did an announcement that they want to do an own display server for Unity (we didn’t throw away Plasma because of Unity either). Whether Mir provides technical advantages or not cannot be judged right now, we will have to wait and see, then we can make a decision whether it’s worth to change our plans. So far I have not seen anything in the documentation that would look like an advantage over Wayland. Given the incorrect statements about Wayland I’m very skeptical whether there can be any advantages. I don’t want to go into detail, just look on the Internet there’s already enough information about that. Also one should consider that Canonical changes plans for their distribution every other day. Just consider the number of toolkits which have been used for Unity – given that I would not bet on Mir will be used next year and that means of course that we should not consider it for our planning.

Third question: Will KWin support Mir? No! Mir is currently a one distribution only solution and any adjustments would be distro specific. We do not accept patches to support one downstream. If there are downstream specific patches they should be applied downstream. This means at the current time there is no way to add support and even if someone would implement support for KWin on Ubuntu I would veto the patches as we don’t accept distro-specific code. If Mir becomes available on more distributions one can consider the second question. Given the extreme success of Unity on non-Ubuntu distributions I’m positively optimistic that we will never have to do the evaluation of the second question.

Posted on

A real update on the progress of Wayland in KWin and KDE

This week I have been at the XDC 2012 kindly hosted by SUSE in Nuremberg. It was a very useful conference for me and I’m very glad I went there. There is not much difference to a KDE conference, even the N9 density was kind of the same. Of course I was asked quite often about the current state of Wayland in KDE and I honestly replied which resulted in rather incorrect “news” postings about Wayland in KDE not happening “any time soon” (whatever that is supposed to mean) to “Wayland for KDE will be delayed” (given that we never had a schedule in the first place, it cannot be delayed).

Actually there is not much to write about as to a reader of my blog you are already aware of all the work I am currently putting into Wayland. It’s just that I never write that I do something with Wayland in mind, because I don’t want to hype the technology.

Nevertheless I had been asked recently rather often about a status update and also promised that I would write a blog post if we reach the fund raising aim for the Randa sprint (thanks a lot for the great support on the last day of the fund raising campaign). So I want to give here a general overview of all the things which need to be done for Wayland.

Porting KDE to Wayland

For a general “KDE on Wayland” we have to consider three areas:

  • Applications
  • Desktop Shell
  • Compositor

For those we have different things which needs to be done and so I will discuss them separately.

KDE Applications on Wayland

For most KDE Applications there is not much work to be done. Thanks to the Windows and OS X ports the code is no longer platform dependent, so all which is needed is having a Wayland backend instead of an X11 backend.

This problem is solved with Qt 5. Once our applications use Qt 5 and KDE Frameworks 5 they should run fine on any Wayland compositor, e.g. Weston. Of course there are a few applications which use X11 directly (e.g. ksnapshot), those needs to be adjusted in addition or be morphed into KDE’s Wayland Compositor (that is something I expect to happen with tools like ksnapshot or klipper).

KDE Plasma

Our Plasma Desktop Shells (Desktop/Netbook/Device) need more adjustments than a normal KDE Application. Plasma is still doing quite some X11 directly for various tasks, like e.g. the tasks manager or pretty much any communication with KWin.

Before we can start working on adjusting Plasma for Wayland we need to get it to Qt 5 first and that requires KDE Frameworks 5 and that requires libplasma2. Currently our Plasma hackers are meeting in Randa thanks to your support to work on libplasma2.

Once Plasma is running on top of libplasma2 we can start working on Wayland support. How that will look like, I don’t know. It’s not a problem we can tackle right now, so it does not make much sense to think a lot of it. We know we need to change a few things here and there to make it work.

But what is really important is that we Do not want to break the desktop. Neither with going to libplasma2 nor with going to Wayland. So any fast steps do not make any sense. It’s important to get this right before giving it to the users. Wayland is an awesome technology, we all want it, we all support it and we don’t want to damage the technology by going to it too early. This has happened too often in the past (not only in KDE) that great technology got damaged because it got released too early on the users. (NOTE: with this I do not mean that Wayland is not ready yet, but that going to Wayland will cause regressions inside Plasma/KWin and there is no need to expose these to the users if we have a wonderfully working X11 backend.)

KWin

Our current planning is to make KWin our Wayland Compositor. This seems like a useful approach as that can help us having a consistent window management experience no matter whether you run on X11 or on Wayland. Even in a Wayland world we will still need to support X11, it’s not like that will go away. I expect that the Linux world will have to support X11 at least for the next 25 years, we have many legacy applications around, which still use things like Motif, so that won’t change. We have with KWin a very good X11 based window manager so it seems quite logical to me that also in a Wayland world KWin will be responsible for being the X11 based window manager.

Given that our manpower is rather limited and given that X11 is what we have right now and we do not have the resources to develop two Compositors (one for Wayland, one for X11) it seems to me that extending KWin with Wayland support is the only possible solution right now.

Now KWin is an X11 based window manager and compositor. It’s development started in a time when nobody expected that there could be anything else than X11, so you could say that it expects that all that exists is X11. This means the difficulty in porting KWin to Wayland is not in adding Wayland support to KWin, but in making it possible to have a KWin without X11 support or at least to be able to start KWin without needing an X Server.

For adding the initial Wayland support to KWin which allowed to include a window in the scene graph and pass input events to it, I only needed around three days of development (that included getting used to the new windowing system and hitting my head against a wall because I was too stupid to find the documentation and too proud to ask for help).

Current State of Porting KWin

For me it is important that going to Wayland does not decrease the code quality of KWin. I don’t want to have many if (X11) { foo(); } else if (Wayland) { bar(); } code blocks. I want to have KWin in a state that it can be easily maintained, that we are able to maintain the X11 code base even once we all run and use Wayland.

There is a git branch with some patches to have KWin support Wayland clients. If you look into the branch you will notice that it does not contain any new commits for more than a year. So it seems that nothing has happened in that area.

Well that’s not true. All the work I have put into KWin over the last year have been with Wayland in mind. I have a pretty clear plan on what needs to be done inside KWin to go to Wayland and I tend to follow that plan. I get side-tracked from time to time to improve the KWin out there right now (e.g. performance improvements) but in general I follow that master plan.

An example for that is the support for KWin Scripting. That might be surprising, what is the relationship between Scripting and Wayland? To have Scripting support I had to add Q_PROPERTIES to the class representing a window (class Toplevel) and to the class representing a managed window (class Client). This means we kind of extracted the interface of how a window needs to look like. The Client class has never be meant to be used for inheritance, so we did not know how the interface looks like. But in future we will need to have a WaylandClient and an XClient. That’s quite a change and scripting helps a lot to know how we need to make the class abstract.

Furthermore I hope that scripting helps to allow us to perform unit testing. I do want to add a unit test for each method before turning it abstract (remember that we don’t want to break the desktop) and to ensure through testing that the X11 and Wayland based implementation behaves the same from a window management perspective.

Another example of work going into Wayland to have the KWin improved right now is this bug I fixed some time ago. For the time being it means that when a window closes it is not raised on top of other windows any more. What has that to do with Wayland? The bug has been that KWin only kept track of the stacking order of Clients, but not of the more general class Toplevel. In order to go to Wayland and to be able to properly integrate Wayland Clients into our stacking order this had to change to keep track of the stacking order of Toplevels instead. It was a difficult to fix bug, but a very important change for going to Wayland.

Also the refactoring work I recently blogged about is an important step on the road towards Wayland. Especially the splitting of the OpenGL compositor is very important. This means that the actual Compositor (SceneOpenGL) does no longer depend on X11. The dependency has been moved into an OpenGLBackend with currently an GlxBackend and EglOnXBackend. This makes it much easier to add further backends in future to e.g. support KWin on top of another Wayland compositor or KWin on top of KMS through libgbm.

Personally I’m very satisfied with the progress we have made over the last year in preparing KWin for the important task of adding Wayland support. I think we are completely on schedule and the state is really looking good.

The Schedule

There is actually no schedule for the porting of KWin or KDE to Wayland. Many things like KDE Applications and KDE Plasma completely depend on the progress of Qt 5 and KDE Frameworks 5.

So what about those dates one can read about in the Internets? Well that are dates coming from my presentation at last year’s Desktop Summit. I put up there a very tough time line for the case that we could get many people involved in working on Wayland. That hasn’t happened so far. Also parts of the time line were just stupid. I wanted to have a first prototype already in KWin at the time of 4.8. It just didn’t make any sense. Wayland was still a too strong moving target and distributions did not yet include the needed libraries any way, so it did not make much sense. It would not have been anything for users and in case anyone wants really to play with it or develop it, there was the branch. As that has not happened I did not see any advantage in merging the code into master.

The second item on the time line was an idea to have Plasma Active running on Wayland rather soon, but that actually did not make any sense without Qt 5/KDE Frameworks 5. So that was really a stupid thing to put into the time line.

The third item in the time line when we actually have everything in place had a strong “It’s done, when it’s done” and “Don’t break the desktop” note. So any references on when I said when KDE will be on Wayland is just wrong. And I do hope that I have clarified this with this blog post. The times I set out where ambitious and idiotic. I do apologize for putting them up in the first place and would kindly ask anyone to realize how idiotic they have been and no longer refer to them. Thank you.

What next?

I’m personally quite satisfied with the current state of preparing KWin for Wayland and I think we have reached a point where we can seriously start thinking about adding real code specific for Wayland to KWin.

Given that we have a nicely refactored OpenGL backend I want to experiment with having KWin run on top of KMS or on top of another Wayland compositor. When I will find time for that I do not know, but given the already upcoming feature freeze it will be hard to have anything in 4.10.

Also quite some required dependencies are not there yet. Libgbm will be out with Mesa 9.0, the release of Wayland 1.0 is expected somewhen soonish and XWayland might hit the next X-Server release which will happen around March. The dependencies are rather important as we cannot depend on unreleased code in KDE. Quite a lot people would complain if KWin would depend on an unreleased X-Server. So that is clearly something to keep in mind when adding the code. Not to mention that I run Debian Testing and don’t like building everything from code.

Posted on

Thoughts about Network Trancparency

Every time there is an article about Wayland you can see that there a lots of uneducated comments about the “fact” that Wayland does not support network trancparency and because of that it is completely wrong to go for network trancparency. These discussions contain a lot of myths and even FUD and I consider it important to share my thoughts about these concerns as I am belonging to those who actively work to bring the benefits of Wayland to the KDE Plasma Workspaces.

Wayland Could Use Network Transparency

Nothing in the Wayland protocol forbidds network transparency. It is not yet implemented, but it is possible to implement it. Wayland uses a Unix Socket for communication but I think it would be rather trivial to either add network transport to the Wayland protocol or to just forward the buffer in the compositor. Stating that Wayland does not support Network Transparency in general is just wrong. It’s not yet implemented but many things are not yet implemented. Obviously it’s true that the Wayland protocol does not support the X11 Network Transparency as it’s not X11 (and that is a good thing). Obviously even a direct X11 successor (let’s name it X12) would also not support the X11 Network Transparency.

X11 Network Transparency is not Suited for Modern Applications

The idea behind the network transparency is to send drawing commands over the wire (useful idee for the requirements of 30 years ago). Nowadays modern applications do not use X11 any more for rendering. They use technologies like Cairo, Clutter, QPainter (Raster) or OpenGL directly. Without using X11 for rendering you end in streaming pixels over the wire. And there are clearly better technologies to do that than X11. Face it: network transparency is going to break very soon even without Wayland. I want to see the Qt 5 used over the wire.

Modern Applications require DBus

Yes DBus is not network transparent and yes most modern application use it, for things like StatusNotifier (this one has fallback to Xembed) or moving the menus somewhere else. Now without network transparency these things are just shown on the wrong system or not at all. Damn stupid devs not thinking about network transparency… So face it: no modern app can be used without DBus which breaks implicitly also the X11 Network Transparency.

Wayland targets Mobile Devices

Those who attended my presentation at Desktop Summit or who have seen my slides might have already got the message: we don’t want to break the desktop and will therefore only target mobile devices in the beginning. The same is true for MeeGo (they don’t have a desktop) and hopefully also GNOME Shell and Unity. So do you really want to forward your X11 application from your (mouse+keyboard) desktop to your touch tablet/smartphone? Probably not. So why do you care about network transparency not available on mobile? For mobile the target groups are the billions of users out there who do not even know that network transparency existed and not the five geeks who cannot live without it. Face it: you are not in the target group of Wayland on mobile devices if you need network transparency. (You probably wouldn’t use an iPad either).

Desktop will continue to Support X11

We will only switch to Wayland as our primary windowing system if we can continue to support legacy X11 applications, because we don’t want to break the desktop. This means you can still run your legacy X11 windows over network even under Wayland and also X11 applications on the Wayland system can still be forwarded to a remote system. So I hear you already complaining about the Wayland windows not being able to be forwarded. Well if the toolkit (like Qt) supports multiple backends there is no reason why you should not use the X11 backend to forward the window. Apart from that: remember, it would not be working at the time when we switch to Wayland even under X11 (see above).

Network Transparency Does Not Belong Into The Windowing System

Adding Network Transparency in the Windowing System is the wrong layer. The windowing system only receives the pixels and that can never be performant. Network Transparency needs to be added to the layer which does the rendering. That used to work with X11 as X11 also did the rendering, but this is not true any more. We need support for Network Transparency in the toolkit. Just imagine the possibilites: remote applications picking up your local font settings, color themes, icon themes… That’s what needs to be done!

But Distros will remove X11 support in $X Years

As long as there is a demand for legacy support distros will still support it. Especially I don’t think that enterprise distributions will dare to remove it in the next 10 to 20 years. Remember that also Mac OS X still supports legacy X11. And even if the distro removes the support: who stopps you to package X11 yourself and provide it to everyone who needs X11? Btw we are talking here about at least three to five years – others have predicted the death of desktop till then.
I hope this post can help to bring future discussions about Wayland to not keep repeating what has already said and is just wrong.

=-=-=-=-=
Powered by Blogilo

Posted on

And There was Text

Short update from the Wayland efforts:

This is quite a success as it took me several days to work out how to activate the window and get keyboard events to it. Now with that done I can start looking at the real window managment bits.

=-=-=-=-=
Powered by Blogilo

Posted on

Discovering a New World

This weekend a group of KDE hackers met in a small mountain village, Randa, in Switzerland to discuss the future of the KDE Frameworks. I was not present, but started on Saturday an endevour for the future of the KDE Plasma Workspaces. Yesterday evening I arrived in the new world:

desktopgrid-with-wayland
This is not just the Desktop Grid effect. The gears window you can see on each of the desktops is a Wayland client, all other windows are X clients. For me this is a rather historic event, not just for KDE but also and especially for Wayland. Up to now there have only been the demo compositor and the QtCompositor as Wayland Compositors. They are more or less just proof of concept. But this is different event: we have here a normal X11 window manager and are able to integrate Wayland clients into the compositing scene just as if it were an X client. This is not a Wayland compositor, it is a X compositor with an implemented Wayland Server component.
This is the proof that there is a sensible migration path from X11 to Wayland. We can use our normal and known X system and manage Wayland clients in it. There is no need to implement the complete stack on top of Wayland first with possible regression. As we can see the Effect Framework does not need any adjustment to support Wayland Clients, which means we can give the user their known KDE Workspace experience. For the user it is irrelevant if a window is an X or Wayland client, it is just a window. For KDE this means that we don’t need and won’t break the desktop. We will switch to Wayland when we are ready, but at the same time we can offer Wayland to interested developers and users who want to experiment with it. For Plasma Active on the other hand we will switch to an X less Wayland-powered KWin as soon as possible.
Of course there is still a long way to go to get to Wayland. This is just rendering, we still need to implement the complete Window Management, transition the compositing away from X, break out X from KWin and lots more. It’s the result of four days where I spent a few hours each day on hacking on that topic. Which shows how simple the Wayland protocol is, although the documentation could be better 😉
Now the next steps will be to clean up the code, document it and push it into a git branch. I hope to see many developers to pick up on it and to start implementing the window manager functionality and extending e.g. Plasma Desktop to support Wayland Clients in the Tasks Applet and many more such things.
I will present the complete KWin Wayland strategy at Berlin Desktop Summit in August.

=-=-=-=-=
Powered by Blogilo