This week I decided to do some research for the Wayland porting of the KDE Plasma workspaces. One of the features we will need in future is a Wayland session compositor which runs nested on a Wayland system compositor. Of course one could think of setups without a system compositor, but overall I think that a nested compositor simplifies the setup and allows to have all the low level technologies in one place without duplication in all the various compositors. +1 for working together.
After three days of work I already have something to demo (video on youtube):
Sorry for the bad audio. I’ll just explain what one can see. The video starts with the normal X-Server. After that it switches to a VT and we start Weston there. On Weston I’m starting KWin with some environment variables set to pick the correct libraries and force KWin into Wayland mode. KWin creates a connection to Wayland, creates a Wayland surface and uses it for OpenGL output. All the windows from the running X Server are rendered into this surface just as if it were a normal X11 output.
KWin also gets input from Wayland and passes it to the X Server. That’s the reason why we can see mouse interaction and working keyboard.
How it works
The OpenGL backend
KWin supports multiple backends for providing an OpenGL context and doing the texture from pixmap operation. At the moment we have an GLX and an EGL backend. Both create the OpenGL context on the XComposite overlay window and provide the texture from pixmap in the GLX case through the GLX_EXT_TEXTURE_FROM_PIXMAP extension and in the EGL case through the the EGL_KHR_image_pixmap extension.
A new backend is added which creates the OpenGL context on a Wayland surface. The backend started as a fork of the existing EGL backend with the X code stripped out. What’s a little bit tricky is getting the texture from pixmap working. The extension used in the normal EGL on X11 backend is not available. The proper solution would be a setup with XWayland, but that’s still too early as KWin does not yet support Wayland clients.
The solution I came up with is inspired by a fallback mechanism in KWin from the time when GLX_EXT_TEXTURE_FROM_PIXMAP was not guaranteed to be around: XShm to copy the pixmap content into an OpenGL texture. Not a nice solution but it works.
Input handling
Input is currently also a rather hackish solution until we have XWayland up and running. We just take all input events and forward them to the XServer with the XTest extension to inject fake events. It’s a huge hack and one can see how old X is there and how limited. I was rather surprised that it works at all. At the time of this writing the code supports keyboard events and the left, middle and right mouse button. Wheel events are tricky as X uses mouse buttons for them and for more mouse buttons I have problems with mapping them as I’m lacking a multi button mouse.
We are also not able to back sync the mouse position from X to Wayland. As far as I understood the Wayland protocol there is nothing like XWarpPointer, so if something in X warps the pointer we have a mismatch. I agree that warping is evil, but we use it in KWin for activating the screen edges 🙁
Next Steps
Cursor
What is to do next is to get the changes to the cursor in X11 and set the cursor on the Wayland surface. That should not be really difficult as the XFixes extension provides everything one would need for that.
Thread
A rather huge limitation at the moment is that the connection to the Wayland display is hold in the main thread. We cannot block there, so we only get events when we actively check for them. This is currently during repainting the screen. So if you wondered why the ShowFPS effect in the video is turned on: it’s to force repaints and to keep the connection alive. This connection needs to go into a thread so that we can block there.
Buffer age
Currently the code forces as to do full-screen repaints. The two solutions we have for non-fullscreen repaints in the EGL backend do not work in the EGL on Wayland backend. The Wayland demo code shows that the EGL_EXT_buffer_age implementation could be used. We wanted to have support for that one anyway in KWin.
Giving it a try
Building from source
I just pushed the code into branch “kwin/wayland-egl-backend” on my clone kde:clones/kde-workspace/graesslin/kde-workspace. Be aware that I intend to force push to this branch.
To build you need to have the Wayland libraries around. The CMake module tries to find it through package kit. Watch the output of CMake, Wayland is only an optional dependency! If you build Wayland and Weston from source, please follow the instructions. You might also need to build Mesa from source.
How to start
Starting is rather simple. Just have an Xserver running somewhere, start Weston on a VT (don’t do nested on X, you would only get a black screen once KWin started) and open a terminal. Setup your environment variables to be able to start KWin and then start KWin in the following way:
DISPLAY=:0 KWIN_DIRECT_GL=1 KWIN_OPENGL_INTERFACE=egl_wayland kwin --replace &
KWIN_DIRECT_GL makes KWin skip the OpenGL Test application which is currently only supporting GLX and seems to fail if one is on a different VT. KWIN_OPENGL_INTERFACE tells KWin which backend to use. So by using “egl_wayland” one forces KWin to use the Wayland backend. In future we will probably detect whether the Wayland display environment variable is defined and just pick it directly (though that is dangerous in case of nested Weston on X).
When will it be available in a release
I want to get this work into 4.11 as an experimental feature for multiple reasons. I wanted to have a build dependency to Wayland in 4.11 for quite some time. So that’s a nice excuse. Unfortunately feature freeze is approaching.
Of course I want to give users something to play with. We have talked about Wayland for such a long time and there is nothing to really see that we are walking the way. Last but not least that might be a rather important solution for Plasma Active as that hopefully allows to run on hardware where we currently would not get OpenGL. With libhybris we should be able to get KWin working with OpenGL and being an X11 compositor even if the driver doesn’t support X11. Though I have not tested this case yet and have no devices to play with. But I know a few people in the community who might be interested to play with that.
Martin thank you so much for bringing us one step closer to X’less dream. 🙂 (well, mostly X’less) As I understand it, your experiment requires an already running full blown X server. In the future it would replaced by a root-less X server but otherwise your code will stay the same, correct?
kind of. With XWayland as the root-less X server we won’t need the code this patch set includes to do texture from pixmap. But yeah the general architecture will probably be rather similar.
What has the ENTER key ever done to you, to make you hit it so hard? 😀
Nice and interesting proof of concept :).
This means that kwin is playing proxy between Wayland/Weston and X, which I’d see as a proof of concept for KWin being able to render X windows in future on top of KWin (when it replaced Weston one day) – isn’t it?
I don’t think we will ever want to run KWin on top of KWin. I think Weston or some other shared piece of technology should be used as a system compositor.
I don’t understand. qtwayland will be useles for kwin?
no, we will use qtwayland for something else. We need it for getting Wayland surface integrated into our compositor. For using it as a “compositor” doesn’t make sense given our architecture.
“This week I decided to do some research for the Wayland porting of the KDE Plasma workspaces.”
YES this is what it all should be about ATM!
“After three days of work I already have something to demo”
YES this is what we have been waiting for all along!
“It’s a huge hack and one can see how old X is there and how limited.”
YES X11 for today needs is basically just one hack on top of another i agree and that is why it was decided it’s time to act and to go forward!
“Of course I want to give users something to play with. We have talked about Wayland for such a long time and there is nothing to really see that we are walking the way.”
YES exactly we need to see and experience it in early stages of development too!
“With libhybris we should be able to get KWin working with OpenGL and being an X11 compositor even if the driver doesn’t support X11. Though I have not tested this case yet and have no devices to play with. But I know a few people in the community who might be interested to play with that.”
YES there are a lot of Android device users out there waiting to do just that!
” I think Weston or some other shared piece of technology should be used as a system compositor.”
YES if it’s not to late push Weston as standard Linux compositor. I think this opportunity is lost but who knows it might come back if the pace you set will continue and broader FOSS community picks that pace up when it comes to Wayland and competitor Mir…
I strongly advocate using libhybris to use the OpenGL driver. This would make Linux porting so much easier to typical ARM hardware.
(My ARM Cheromebook agrees with me, it just loves running Ubuntu and it would like to do it more efficiently :D)
Regarding always using a system compositor:
How would you implement features like automatic overlay usage (rgb and yuv) and automatic unredirect for fullscreen windows that weston supports if all you do is send the final fullscreen image to the compositor?
Just curious, i generally like the idea of a system compositor but i think you lose a lot of optimization possibilities if the scene graph is so far away from the hardware.
obviously we would not send the final fullscreen image to the compositor in such cases, but pass on the overlay. In fact I expect that having a system compositor makes it much easier, because we don’t need to implement those parts and can use the excelent support in Weston.
Wouldn’t you need a new protocol for this to delegate compositing to the system compositor? What about the overhead of the IPC?
No, not really. For fullscreen we would just pass the buffer on as another fullscreen surface. And for sub-surfaces we can just pass it on as a sub-surface as long as it’s not transformed.