Another busy month, another round of great features being worked on. March featured our Plasma 5.6 release which had lots of last minute (aka bug fixing) focus. We had our sprint at CERN. But of course also new code was added.
One of the features on my TODO list for Plasma 5.7 is the ASCII art output rendering. Mplayer supports that and while that is nice, it’s not really a good solution. It would be way more convenient to have the complete workspace rendered through ASCII art to not have to implement this feature in every application.
So without much further ado I’m happy to announce the caca platform for KWin: Glxgears running in nested KWin on caca platform
Unfortunately it’s only supported by our QPainter based compositor. Thus to run it, you need to make sure to use KWIN_COMPOSE=Q to pick the QPainter backend (yeah we should improve that to have a better automatic selection). This is the first platform which support both nested and full mode. Thus you can run with –ascii in your normal X session which will open a window, just like our nested X11 platform plugin. The rendering is done through the great caca library.
Lock screen in KWin’s caca platform
What I’m personally impressed about is how small this new platform plugin is. Less than 400 lines of code, great job by libcaca. But it also shows how easy it is to get KWin on a new platform.
I haven’t blogged for quite some time about the progress on KWin/Wayland and had a few people requesting an update. As we are now approaching a feature freeze and I have most of the things I wanted to do for Plasma 5.6 done, it’s time to blog again. I use this also as a public service announcement: thanks to Let’s Encrypt my blog is also available through an encrypted connection.
Last month my development focus was on the input handling in KWin. That is the part between input events enter through libinput and are sent to the Wayland client. There are many things the compositor needs to consider for input events: updating the window which has focus, ensuring while the screen is locked to not pass events to normal windows, handling focus follows mouse, etc. etc. On X11 KWin has already code for most of these things, but the code is quite dependent on X11, so it needed to be partially adjusted and partially rewritten.
The code we had in KWin/Wayland for input handling so far already showed it’s age. It was written and designed before KWin really became a Wayland compositor, from the time when KWin could render X11 windows to another Wayland compositor. So it mostly cared about sending the events to the X server. Everything else continued to work as the X11 event handling was still in place.
So the first task was to untangle the code so that it’s easier to extend and at the same time guarantee that it won’t break. As we are now able to start KWin/Wayland on a virtual framebuffer, we can run it during our auto tests. This was a rather important corner stone for reworking the input as it allowed to write test cases for everything KWin does.
With that done existing features from X11 could be ported to Wayland including mouse actions (what to do when clicking inactive window), unrestricted move/resize with alt+(left/right) mouse button, focus follows mouse and auto raise, etc. All those features are now also under test coverage and as the code is mostly shared with the X11 implementation we now also have test coverage for these features on X11. That’s quite an improvement for our X11 implementation thanks to Wayland.
Another area of work is keyboard layout handling. So far KWin defaulted to use the us layout without any possibility to change. That was a huge drawback for my own usage as I couldn’t even write my name. Like with many other input related areas I’m not really familiar with the technology, so I had to look into it in more detail. I am very pleased with xkbcommon, it was really easy to get this working and hooked up properly in KWin. The result is that KWin/Wayland now fully supports keyboard layout switches and also the integration with Plasma’s keyboard layout configuration module. I was rather pleased to see that the configuration module was hardly X11 dependent and just works on Wayland. With KWin listening to the correct DBus signal it allows to reconfigure layouts. But there is still work in that area. So far I have not added support for compose keys, the systemtray applet for switching layouts is not ported yet, accessibility features are still lacking. If you are interested in these areas some help is appreciated.
In case you tried Plasma 5.5 on Wayland you might have noticed that the cursor was sometimes rather incorrect. Not anymore in Plasma 5.6. The cursor image handling got also redesigned and put under test coverage (unfortunately our CI system doesn’t like them yet, locally they pass). But having KWin handle cursors correctly is unfortunately not sufficient to have proper cursor images. Cursor images are set by the clients and if they set an incorrect cursor image, KWin cannot do anything about it. For example QtWayland doesn’t support animated cursors and doesn’t support custom cursors. With the feature freeze for Plasma 5.6 behind us I’m also looking into these issues now. Overall client bugs make it really hard to test features as you never know whether it’s a bug in your application or in the client (or XWayland). The fact that GTK+ and wayland-demos just crash, because KWin doesn’t support xdg-shell yet, doesn’t make it easier to test new features.
The last input area I looked at and landed just in time for feature freeze is drag’n’drop. The implementation is not yet using the updated protocol from Wayland 1.10 as we had already passed dependency freeze when I started to implement it.
Overall the improved input handling gives us a nice feature set for Plasma 5.6 on Wayland. On a single screen setup it’s quite useable already. Of course there are bugs and those need you. Please give it a try and report all bugs you see.
In the area of input there is also more work to be done. We need support for graphic tablets. If you are interested in it: I’m willing to mentor a GSoC project about it. We have a few GSoC ideas for Wayland, check them out!
So what’s next in Wayland world? Till the release of Plasma 5.6 I want to concentrate on bug fixes to make the experience better. Than there’s xdg-shell quite high on my priority list and making multi-screen work correctly (that’s blocking me to switch my main system, the single screen notebook is mostly used on Wayland nowadays).
As a kind of Christmas present to our Wayland users I’m happy to announce that over the last two weeks I worked on adding support for server-side decorations.
The main motivations for working on it was the fact that I want to switch to Wayland as primary driver for my system and the nested KWin running on top of another Wayland server, which I need for development, doesn’t have any decorations. Of course I could have implemented client-side decorations for it. But as my readers might know, I consider client-side decorations as an inferior solution. And KWin of course has support for server-side decorations anyway for X11 and thus it’s less work to go for server-side decoration than to go for client-side.
The second reason is that Qt’s default client-side decorations are comparable ugly and lack important features like a difference between active and inactive windows which makes using a Wayland session really hard.
In this case a possibility could have been to develop a plugin so that KDecoration based themes could be used for client-side decoration. But to get it really useable this would have required a complex protocol to get in on par with what KWin internally has.
So here’s the solution: Server Side decoration support in KWin/Wayland
A core element is a protocol to negotiate whether a window should have server-, client-side or no decoration which got added to KWayland. KWin got an implementation for that both as server and client. I plan to submit the protocol for inclusion in Wayland next year. I do think that this can be a general solution: KWin won’t stay the only Wayland compositor preferring to not have client-side decorations. If we think about tiling and use cases like phones we see that client-side cannot be the ultimate solution. Thus I think it’s a useful extension. Of course it doesn’t forbid client-side decoration, that’s still possible with the protocol. So GTK+ applications build upon client-side decoration are still able to use it, but of course I highly recommend to use server-side decorations on a system that prefers server-side decorations (the protocol is also able to tell that).
The last part to get this working got implemented in our Qt Platform Theme plugin for Plasma. This plugin will move from frameworksintegration to Plasma with 5.6, so we can easily extend it and depend on KWayland. The plugin checks whether the server supports the protocol and if it does it disables Qt’s client side decorations. For each new created Wayland window it tells KWin to either use server-side decoration or no decoration (popup windows). As all of that is implemented in our platform theme plugin it means that it doesn’t affect other Wayland compositors. There the plugin does not get loaded and Qt’s client-side decorations will be used. So no fear: this won’t affect GNOME Shell at all. As the plugin is currently in the process of being moved, it’s only in a scratch repository and won’t make it to main this year. Our code deserves a Christmas break as well 🙂
Happy holidays and a successful Wayland year 2016!
Last month our Wayland efforts made a huge step forward. In KWin we are now at a state where I think the big underlying work is finished, we entered the finishing line of the KWin Wayland porting. The whole system though still needs a little bit more work.
The big remaining task which I worked on last month was geometry handling. That is simplified: moving and resizing windows. Sounds relatively easy, but isn’t. Moving and resizing windows or in general the geometry handling is one of the core aspects of a window manager. It’s where our expertise is, the code which makes KWin such a good window manager. Naturally we don’t want to throw that code out and want to reuse it in a Wayland world.
And that meant: refactoring. The big problem here was that the code in question is also highly X11 specific. Examples are: the sync protocol for synced resizing, window gravity handling, various X11 specific quirks for moving X11 window moving smooth (e.g. only update at end of move if compositing is active, direct updates if not active).
The task was now to separate the X11 specific geometry handling from generic geometry handling so that it can support both Wayland and X11 windows. With this now in place all the geometry handling works also for Wayland windows. This includes move/resize through the alt+f3 menu, but also triggered from the window itself. During move/resize the quick tiling areas are of course triggered correctly, show the outline and snap to the area if released. Resizing is automatically synced to the speed the client supports and that also in a much better way than on X11 (yay for Wayland and double-buffered state!). During move/resize the windows snap to screen borders and other windows.
Unfortunately our checks to ensure that the window titlebar is not moved outside the visible area doesn’t work, because of client side decorations. Yay, everything is awesome! Have I mentioned that client side decorations are a stupid idea because it breaks useful stuff? No, cannot be. Related to that: QtWayland enters move mode if you click the title bar to activate the window. Of course there should be a delay which KWin implements for it’s decorations. After all we have more than 15 years of experience in doing window decorations. Yay, client side decos! Given control to the application! Let’s leave them in charge of window management, what could go wrong? Wohoo! Totally awesome! Broken window management! That’s the way to go! Of course that could be fixed in Qt, but well it doesn’t fix the problem. We have seen this in Chromium years ago, we even had to adjust our quick tiling/maximization behavior because of it. Another fun fact: QtWayland deco has a minimize button which does nothing because it’s not supported in wl_shell protocol. Have I mentioned that client side decos are awesome?
All right, all right, I’ll end the sarcasm mode now. Our geometry handling has a few more very handy features like window packing: you can configure a shortcut to move the window left (or other direction) and it will move to the next window or the screen border. Similar we support growing or shrinking the window to the next window. I was hardly aware of that feature before doing the refactoring, so I thought I should point it out. It can be used from scripting, so should be useful for ideas like poor man tiling or resizing multiple quick tiled windows at the same time. As a nice addition this code is now covered by auto tests.
So with geometry handling in place it’s possible to do real testing and one of my systems (my notebook) migrated from X11 to Wayland. Actually I have been using Wayland on that system already for watching videos since April as it gives tear-free rendering. A clear plus when watching videos.
And this gives us already the outlook for what we will see in the November update: I’ll focus on stabilizing the current state of Plasma/Wayland and fix bugs, fix bugs and fix bugs. My aim is to have a useable early-adopters version ready for the Plasma 5.5 release. It’s looking good, so I’m confident that we will reach that state.
One of the most important dependencies for our phone project is libhybris. Libhybris is a neat technology to allow interfacing with Android drivers allowing for example to bring Wayland to a device where all we have are Android drivers.
Given that KWin provides a hwcomposer backend which uses libhybris to create an OpenGL context. All other applications need libhybris indirectly to have the Wayland OpenGL buffer exchange work automatically.
When we started the work on the hwcomposer backend we based it on the libhybris version used in Ubuntu (0.1.0+git20131207) as we used Ubuntu as the reference platform. Soon enough we noticed that this version diverged a lot from the upstream version. Lots of recent changes are missing and there are API incompatible changes.
This made working with it difficult. How much time should we invest in investigating issues? Should we write code which we know might break once Ubuntu decides to upgrade libhybris? How well is Wayland integrated in the Ubuntu version given that they don’t need it? If we need help, who to talk to? Ubuntu who will tell us that they don’t know anything about Wayland, or the libhybris devs who might just tell us: use later version?
Furthermore we want other distributions to provide Plasma for the phone. This means they need to provide libhybris. Of course this is difficult if we need to tell them that we need exactly the version used by Ubuntu. And even more it might conflict with other uses. Considering distributions like Mer would have to chose between a libhybris for Plasma and a libhybris for lipstick.
With that in mind we wanted to invest some time on upgrading libhybris in our stack in this release cycle and then fix the issues we were seeing in the stack. Our awesome packagers did the job of creating packages so that I can port KWin against it. And in deed after some hacking I had KWin rendering again. A more difficult task was to get other applications to work as we run into a problem that libwayland-egl does not use the alternatives system. Thus our packagers needed to do some ld tricks to get this worked around. But with that we had a nice rendering system.
A surprise in this exercise was that our input handling code in the hwcomposer backend didn’t compile any more. The code was gone. While that was in the first moment an unpleasant surprise, it soon turned into something wonderful. If that code is not needed at all on an Android powered device it means that we must be able to get libinput to work with it. 400 lines of code deleted and it’s using the shared input stack through libinput. I’m very happy about that!
With that all in place we finally were able to investigate the rendering issues we were seeing. My hope was that just upgrading libhybris would fix the visual tearing, but unfortunately not. While I’m still surprised that it’s possible to get tearing in the first place on Android devices (hey ever heard of things like Atomic Mode Settings, Android?), it at least gives us a vsync event. Unfortunately the only tear-free solution I could find invokes blocking till we get the event. I don’t like that and I think that’s a bad architecture. One can have blocking free and tear free rendering. Our DRM (kernel mode setting) backend can do so with an easy to use API. Really disappointing that the Android stack is in that regard not better than the glx backend. But well at least it’s tear free 🙂
As we now use upstream libhybris I hope to see distributions to pick up the work and provide a Plasma phone spin. I’d love to see an openSUSE phone or a Fedora phone (or any other distribution). Distributions: you can of course ask us on how to integrate 🙂
Back in 4.x we provided two binaries for KWin: one compiled against OpenGL (kwin) and one compiled against OpenGL ES (kwin_gles). The reason for that is that one can only reasonably link either OpenGL or OpenGL ES and OpenGL ES is only a subset of OpenGL, so one needs to hide the OpenGL calls (especially the OpenGL 1 calls).
With 5.x we were no longer able to provide these two binaries. The reason for that is that OpenGL got “upgraded” in Qt and QtGui itself links either OpenGL or OpenGL ES. To keep the system’s sanity we decided to follow how Qt is compiled. If Qt is compiled with OpenGL support KWin gets compiled with OpenGL support, otherwise with OpenGL ES support.
That’s of course a reasonable design, but it means that it becomes difficult to test the OpenGL ES code paths. One needs a dedicated Qt and all other dependencies compiled against OpenGL ES. Or one needs a nice device like a Nexus 5 with Plasma mobile. This had resulted in breakage already as we were not able to test enough. Such times belong to the past as I have a nice Plasma mobile device to compile and test my KWin on.
But since we introduced OpenGL ES support through a compile time switch, many things have changed. KWin dropped the OpenGL 1 support which means that most of the code which wouldn’t compile with OpenGL ES is just gone. Furthermore we switched to use libepoxy, so we don’t link OpenGL or OpenGL ES at all, but libepoxy which does the right thing for us. With that we are able to remove all the compile time checks. Of course we need runtime checks to ensure that we don’t call functionality which isn’t available on OpenGL ES.
Now with the upcoming 5.5 release we are able to have one binary which serves both OpenGL and OpenGL ES. Note to distributions: the artifact kwinglesutils.so is no longer compiled, please adjust your packaging rules. KWin will use either OpenGL or OpenGL ES depending on what Qt uses.
Given that nowadays it’s also possible to create both an OpenGL and OpenGL ES context at runtime we can also make use of that and introduced a new value for our KWIN_COMPOSE environment variable: O2ES. If that’s specified KWin will use the EGL backend and create an OpenGL ES context. Although it in general is also possible to create an OpenGL ES context through glx we do not support that for simplification. As proof debug output (qdbus org.kde.KWin /KWin supportInformation) from my running KWin instance:
<code>Compositing
===========
Compositing is active
Compositing Type: OpenGL ES 2.0
OpenGL vendor string: Intel Open Source Technology Center
OpenGL renderer string: Mesa DRI Intel(R) Ivybridge Desktop
OpenGL version string: OpenGL ES 3.0 Mesa 10.6.8
OpenGL platform interface: EGL
OpenGL shading language version string: OpenGL ES GLSL ES 3.00
Driver: Intel
GPU class: IvyBridge
OpenGL version: 3.0
GLSL version: 3.0
Mesa version: 10.6.8
X server version: 1.17.3
Linux kernel version: 4.2
Direct rendering: Requires strict binding: no
GLSL shaders: yes
Texture NPOT support: yes
Virtual Machine: no
OpenGL 2 Shaders are used
Painting blocks for vertical retrace: yes
</code>
So in a way we have kwin_gles back, it’s different as it’s no longer a dedicated binary, but it’s runtime switchable. For the moment the only way will be the environment variable. I’m reluctant to add a config option as that sounds like quite some chance for breakage.
September was a busy months in the KDE Wayland world. We have worked hard to bring Plasma closer to a workable system and could cross off some very important milestones.
Transient window positioning
One of the biggest oddities when trying out Plasma on Wayland in the 5.4 release is the fact that menus open at random positions. The reason for this is that KWin applied it’s placing strategy on it and ignored the hints provided by the window. We have now implemented support for the so-called transient windows in KWayland and use the provided placement hint in KWayland. So now all menus open at the correct position. A useable Wayland session is much closer now.
Plasma/KWin specific extensions
Marco did quite some work for the integration of Plasma. KWin provides some Plasma specific extensions like the sliding popups effect, blur and background contrast effect, etc. We have an abstraction through KWindowSystem so that applications do not have to use low level X11 calls directly. Now we extended this abstraction to also Wayland: if the application uses the API it will work on both X11 and Wayland. Granted it will only work with compositors providing the specific protocols, but that is no difference to X11. Also there the compositor needs to implement the custom protocol.
On the client side the protocol is implemented in KWayland and the integration for KWindowSystem is provided through
a plugin provided by the kwayland-integration repository. On KWin side the protocol is also provided in KWayland allowing a very easy to use API. Of course also KWin needed small adjustments in the effects to announce support for the protocol and read the information provided by the windows. Thanks to nice wrapping in KWayland the code is cleaner and simpler than in the X11 case.
Support for multiple X Servers
A change not directly relevant for KWin went into KWindowSystem and will be released with the upcoming 5.15 KDE Frameworks release. KWindowSystem provides an X11 API which looks like it supports multiple X Servers (e.g. one application talking to multiple servers), but that has never worked as it fetched required atoms only on first connection. We refactored the relevant code to no longer have this limitation.
Current Plasma in a nested KWin/Wayland session with grabbed input
Granted normally applications do not talk to multiple X servers, but there is a mode in KWin which uses just that: a nested kwin_wayland on X11. To explain: it needs to talk to the host X server for rendering two (one server) and it starts it’s own Xwayland server (second server). In case you have ever wondered why the nested kwin_wayland window released with Plasma 5.4 neither has a window icon nor a window title: that’s the reason. We couldn’t use Qt’s abstraction (wrong QPA plugin) and also not KWindowSystem as we needed to make sure the atoms get resolved for the Xwayland server. Now with this restriction removed the window has an icon and a title. Even more I added a “grab input mode” as one might know from virtual machines. While it’s easy to implement I didn’t want to implement it without having a way to tell the user what happened and what the current grab state is.
Preparing KWin for the cloud
The most exiting new feature in my opinion was born last Friday based on frustrations about testing KWin Wayland. Last week Marco and I spent quite some time investigating a few regressions (as it turned out due to adding transient window support). The way to test it was just uncomfortable. We had a test case but it meant starting KWin, waiting till it’s started, start the test application, perform some clicks and interpret whether it worked. Once we fixed that issue I started to look at a crash and the process was similar annoying. What I wanted was a way to automate the test condition, that is an autotest which we could even run in our CI system.
So on Friday I decided to dedicate my development time on a virtual framebuffer backend.This backend (to start use kwin_wayland --xwayland --virtual) doesn’t render to any device, but only “simulates” rendering by using a QImage which then isn’t used at all. Well not completely true: there is an environment variable to force the backend to store each rendered frame into a temporary directory.
Why is such a virtual backend so exiting? Well it means we can run KWin anywhere. We are not bound to any hardware restrictions like screen attached or screen resolution. With other words we can run it on servers – in the cloud. The first such instance runs on our CI servers in the form of an automated integration test. And in future there will be much more such tests.
But that is not only interesting for KWin to have it’s autotests, it’s also interesting for all other projects of the workspace as we have now a virtual Wayland server which is functional identical to the one we use. We also have a better virtual X server now as we have Xwayland instead of Xvfb (e.g. support for XRandR extension).
Once I had it implemented ideas came to me for improving our CI system: we could use it for something like OpenQA (or integrate the existing tool) and start a complete Plasma session and screenshot various points (if that sounds like an exiting project: please contact me, also if you want to do that as a Season of KDE project ).
Or integrate a remote rendering solution (e.g. VNC, rdesktop, spice, html5) and run a complete session through the web. That could be a very interesting feature for designers, translators, supporters and many other non-developers. Get the latest state of the code directly tested. We have things like Kubuntu CI which make it easy to test in a virtual machine, but wouldn’t it be even more awesome to just run the latest build of the CI system in the browser?
KWin Tests
With the help of this virtual backend we are now able to start a “complete” KWin in the autotests. This allows us to very precisely test whether a specif feature works as expected. Unit tests are great, but sometimes one wants to test the complete integration and that’s now possible.
Working auto completion tooltip in Kate on KWin/Wayland
The first problem addressed with this new possibility was a bug noticed while writing this blog post. I used Kate on Wayland and the tooltips got keyboard focus. So now we have an autotest which ensures this case works.
On the Plasma workspaces Display Power Management Signaling (DPMS) is handled by the power management daemon (powerdevil). After a configurable idle time it signals the X-Server through the X11 DPMS Extension. The X-Server handles the timeout, switching to a different DPMS level and restoring to enabled after an input event all by itself.
The X11 extension is a one-way channel. One can tell the X-Server to go to DPMS, but it neither notifies when it has done so nor when it restored. This means powerdevil doesn’t know when the screens are off. Granted, there is a call to query the current state, but we cannot perform active polling to figure out whether the state changed. That would be rather bad from a power management perspective.
So overall it’s kind of a black box. Our power management doesn’t know when the screens turn off, Neither does any other application including KWin. For KWin it would be rather important to know about it, because it’s the compositor and there is no point in running the compositor to render for something which will never be visible. The same is obviously true for any other application: it will happily continue to render updated states although it won’t be visible. For a power management feature this is rather disappointing.
Now let’s switch to Wayland – in this case kwin_wayland. Last week I started to implement support for DPMS in KWin. Given that KWin is responsible for DRM, we need to add support for it in KWin. Of course we want to leverage the existing code in powerdevil, so we don’t add the actual power management functionality, but rather an easy to use interface to tell KWin to enter a DPMS state on a given output.
Of course we tried to not repeat the mistake from X11 and thus the interface is bi-directional: it notifies when a given state is entered. Although in a Wayland world that’s not so important any more: KWin enables DPMS and knows that DPMS is enabled and can make internal use of this. So whenever the outputs are put into standby the compositor loop is stopped and doesn’t try to repaint any more. If the compositor is stopped windows are not being repainted and aren’t signaled that the last frame got painted. So all applications properly supporting frame callbacks stop rendering automatically, too. This means on Wayland we are able to properly put the rendering of all applications into an off state when the connected outputs are in standby mode. A huge improvement for power management.
Lately I have been asked a lot about using Vulkan in KWin: in fact almost every blog post in the last few months has questions about it and that seems to me there is something to write about it.
So the quick tldr: I don’t have any plans on adding support for Vulkan. Over the last years I ported to so many new technologies, including at least 3 incompatible OpenGL versions. I am not looking forward to another incompatible OpenGL version (whether it’s called Vulkan or OpenGL doesn’t matter to me on that).
Now let’s look at the strength of Vulkan: going closer to the hardware and doing multi-threaded rendering. KWin still performs the rendering in the main GUI thread. Qt tried to do rendering in an off-thread for QtQuick’s scene-graph, in case of KWin we also do that in the main-gui thread. Reworking our compositor to use threading is a lot of work and would also probably improve the performance with OpenGL. Anyway as long as KWin doesn’t support threaded rendering this improvement by Vulkan is rather moot.
Now let’s look on the closer to hardware: yesterday I did a blog post on how I want to use e.g. KMS Planes to get closer to hardware and bypass the OpenGL compositing. Vulkan will allow us to perform rendering closer to hardware, but if we don’t render at all, we don’t benefit from it. Vulkan is probably more power saving, but not using the rendering will save even more.
So Vulkan can only improve when the scene needs to be rendered: e.g. when you wobble a window or spin the desktop cube. I don’t think that those effects are what we should optimize for and spend our time on and even if: there are low-hanging fruits to optimize using OpenGL, we do not yet use OpenGL 3 or 4 features to improve these effects.
Overall Vulkan looks to me like a lot of work as once again we would have to add a new compositing backend, write a new low level interaction, rewrite all shaders, etc. etc. Going to Vulkan early would mean introducing more complexity to KWin, more different code paths our users might use. It sounds like a mood exercise to do so. But I also doubt that there will be useable Vulkan drivers any time soon.
Vulkan is a great technology which will make graphics much better. But I doubt that an application like KWin is the use-case it was developed for.
Now to something else!
KDE is currently running a fundraiser for the Randa Meetings with the topic “Bring Touch to KDE!”. I personally will not participate in the Randa Meetings, but many other KDE developers will go there and we need your support for it. Please help and make the Randa Meetings possible.
On X11 the (OpenGL) compositor renders into a single buffer through the overlay window. This is needed to get features like translucency, shadows, wobbled windows or a desktop cube as Xorg itself doesn’t have any support for such features. The disadvantage of this approach is that we basically always have to perform a “copy” of what needs to be rendered. Consider VLC is playing a fullscreen video the compositor needs to take VLC’s video pixmap and render it onto the overlay window. The compositor needs to run, evaluate the scene and then render the one window.
Of course the compositors optimize for this case by only repainting what is needed and repainting from top to bottom in the window stack so that only the one fully opaque window gets rendered. Even more they have features like unredirecting fullscreen windows or (in the case of KWin) blocking compositing while specific windows are active. Unredirecting is a nice idea but it doesn’t really cover the use case. If one has just a small window (a tool tip, menu, on-screen volume change display) on top of the VLC video window, we need to start compositing again. Although we still would like to just forward the one VLC window.
On Wayland the situation looks better. Instead of rendering to an overlay window the compositor can directly interact with the hardware through e.g. DRM. This allows to for example take the buffer provided by VLC and pass it directly to DRM and bypass the compositing all together. But modern hardware allows us to do more than Xorg ever allowed us through “layers”. Be it the raspberry pi, Android’s hwcomposer or Linux’s KMS Planes, the idea is always similar: let the hardware compose the final image together.
If we think of Plasma phone we realize that in most cases we just need three layers: top panel, bottom panel and a maximized window in between. There is no need to do complex composition of the scene at all as all that is needed is putting the three visible (and opaque) windows in the three layers.
On a desktop system it’s a little bit more complicated. Our windows are not all maximized, they have translucent window decorations, shadows, there is a panel with blurred background. Especially the latter one needs a composition through OpenGL. Nevertheless there should be situations where we can make use of layers. Especially in the case of a fullscreen or maximized window this should help making the compositor faster.
Using layers will mean a large change on how KWin’s compositor works. The infrastructure for unredirecting of fullscreen windows cannot be reused in a sensible way as it is evaluated only for fullscreen windows and before the compositor runs. That is before the effects are executed. The decision whether to put a window into a layer needs to come at a later point: once we have evaluated whether we need to compose the complete screen (you might be wobbling a window) or can short pass through layers.
What I have in mind is to give this architecture a try with the VC4 hardware of the raspberry pi which supports layers in a decent way. And to first adjust the QPainter based compositor (as it’s the simpler one) to make use of this architecture. The idea is to experiment with it to get a useful generic KWin-internal backend API, so that we can also use it for OpenGL compositor and even more for our DRM and hwcomposer backend.
For the DRM backend I want to use the new atomic modesetting feature which will be introduced in Linux 4.2. Given that we cannot depend on it anytime soon, this will still need some time till it will be implemented. Overall all of this is not going to happen tomorrow – there are still more important things to work on. But if you, dear reader, are interested on working on this, please contact me. I’m happy to walk you through the code and share my ideas.
As this blog post is mostly about optimizing the compositor by making use of new hardware features which we didn’t have on X11, I can already point out that I will do a follow up blog post on Vulkan.
Now to something else!
KDE is currently running a fundraiser for the Randa Meetings with the topic “Bring Touch to KDE!”. I personally will not participate in the Randa Meetings, but many other KDE developers will go there and we need your support for it. Please help and make the Randa Meetings possible.
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish.AcceptRead More
Privacy & Cookies Policy
Privacy Overview
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
