Last week I had been at the 2014 X.Org Developer’s Conference in Bordeaux, France. After Nuremberg two years ago this was my second XDC and I am very glad that I was able to attend and want to thank KDE e.V. for sponsorship. With the X.Org windowing systems (X11 and Wayland) being at the heart of KWin it is good to get an update on the latest development efforts.
Especially the talks about Wayland related topics were quite important. It was interesting to see what the Tizen developers are working on with their IVI-Shell and the discussions around the talk highlighted the need for a careful put out standardization process for the Wayland shell interfaces: if the interface does not contain enough functionality, multiple incompatible interfaces could be the result making it difficult for applications to interact with the system. On the other hand if the interfaces are too broad it will result in compositors not being able to completely implement the interface resulting again in compatability issues.
More directly touching our development were the talks concerning input handling with libinput. I already have some code prepared and expect that KWin/Wayland 5.2 will come with libinput support (more about that in a future blog post when the work is ready). The talks also showed that I need to reach out to the Krita community to get support for tablet devices. I don’t have access to such a device and even if, I would not know or understand the needs for these kind of input devices. There I will need the expertise of developers using tablet devices and I hope to find this within the Krita community.
KWin/Wayland 5.1 gained support for the fullscreen shell interface. My idea when adding this was to not have to implement DRM support in KWin, but (for the time being) leverage Weston. This simplifies development and allows us to move forward on a higher speed. Jason Ekstrand’s talk showed that the fullscreen shell provides more interesting aspects than our use case. The shell can also be used for use cases such as screen sharing: a compositor renders in addition to a fullscreen shell provided by a different compositor which can use it to e.g. capture a video stream or forward an rdp session. Very interesting and quite useful that we already support it and won’t have to add additional support for rdp into each compositor.
Last but not least I want to highlight Steve Dodier-Lazaro and Martin Peres’s talk about security in a Wayland world. They presented a solution for privileged clients which is very important for our development in the Plasma world: there will be interfaces between Plasma and KWin and nobody except Plasma should be allowed to bind to them. The library presented in this talk will solve this problem and many more. Even more the talk touched various interesting security related topics which I had never thought about in that way. For example setting a window fullscreen provides a security risk: it could fake a lock screen interface tricking the user into entering the password. On X11 the Window Manager/Compositor is unable to mitigate this risk. But on Wayland a compositor can e.g. overlay the window and explicitly ask the user whether the window is allowed to go fullscreen – compare to what browsers already do today. I’ll reach out to the VDG to come up with a good solution.
More interesting is the question on how we can trust password dialogs. This is something I already spent some thoughts about in the past. The problem right now is that a malicious application can easily trick the user into entering the password by faking a system dialog (e.g. polkit authentication or kwallet dialog). We need to find a way to show to the user that this is a trusted dialog and that the password is entered for a trusted application. A difficult task. There are many ways we can make the dialog in a way that nobody would be able to fake it: color the decoration, wobble it, put it into the Plasma’s notification area, but none of them are obvious to a user that no other process can fake it. As explained in the talk it needs an out-of-band communication channel to inform the user that this dialog is trusted. An idea which came to my mind is using KDE Connect to send a token to the connected smartphone to indicate that the dialog is secure.
Unfortunately the state of KWin/Wayland has not processed enough compared to Akademy that I could have done a talk about it. After all it was just a few weeks. So instead I decided to give a talk about what we mostly did last year: port KWin to XCB.
With the initial release of Plasma 5.0 behind us I also started to look more in the direction of Wayland again. Now I’m kind of in full flow on Wayland work and kwin_wayland is progressing nicely. Yes, KWin 5.1 will introduce a new binary called kwin_wayland to complement the kwin_x11 binary which got introduced in KWin 5.0.
Now I do not want to list all the changes as you can hardly express them all in a blog post, but I can point to my Akademy talk. I will provide a small overview of the current state, what is new in KWin 5.0, what will be new in KWin 5.1 and where the journey is going.
Of course there is lots of work going on and help is always appreciated. We started to use a public available task tracker on todo.kde.org. Also I have to say that there are still quite some open tasks for kdecoration2. Please help as I cannot split myself and it would be super important to release KWin 5.1 with kdecoration2.
Before heading into the weekend I thought about writing a small update about the KDecoration2 status. Since my last blog post I started integrating KDecoration2 into KWin. This was partially easier and partially more difficult than anticipated. Especially ripping out the old decoration code is rather complex. There are quite some design differences which make the transition complex and especially values inside KWin core are using enums defined in the decoration API – e.g. the maximized state is kept as a KDecorationDefines::MaximizedMode. This will need further work to move the enums and so at the moment the old decoration library is still compiled although the library is no longer in use.
Ah that means there is code? Yes, today I pushed the branch as “graesslin/kdecoration2” into kde:kwin git repository. To give it a proper try you also need the kde:kdecoration (master branch) and kde:breeze also on “graesslin/kdecoration2” branch. The new decoration API is working quite well and I’m rather satisfied. The memory usage of KWin dropped significantly. In a previous report I mentioned that KWin needs around 200 MB, right now my KWin only needs around 40 MB, the number of open windows is a little bit smaller, but still it shows in the right direction. And that’s without any optimisations. There is still some optimization potential in Breeze (check out our todos, all purple tasks are in Breeze) and in KWin (red tasks). Also a nice improvement is that the window decoration no longer flickers when resizing the window. This is a rather big annoyance of KWin 5.0. I expected that the new API would fix this issue, but seeing it confirmed is really nice
Last but not least the restart of KWin got faster which is a nice improvement for KWin developers, for users it’s not that important. The reason for this is that when we enabled/disabled compositing we recreated the window decorations. So when restarting KWin we first created all windows with their decoration before enabling compositing to just destroy them and create them again (yes it would have been possible to improve it, but it doesn’t hit users). Now with the new API there is no need to recreate the decoration. Only a Renderer is exchanged, but the rendering is delayed till it’s needed.
This screen shot does not only show the new Breeze decoration, but also some new features: when quick tiling a window the borders cornering the screen edges are removed. Also in the preview application one can see that the decoration scales which is nice for high-dpi screens. The decoration API follows the approach Plasma is using.
I have been running a KWin with the new decoration API since Monday, which means we are close to start the review process. But there’s of course still work to be done and I’d appreciate any help. The window tab API is not yet implemented, the configuration module needs adjustments to load and render the new deco and most important we have to port the existing decorations like Oxygen, Aurorae, deKorator and QtCurve. For the last three I want to improve the theme experience by allowing the plugin to say that it supports themes directly in the JSON meta data and point to where to find them. So the configuration module can just show all of them. The trick will be to pass the to be used theme to the factory method (we have a nice QVariantList there which can be used). Also the meta data will make it possible to point to GHNS configuration files so that we can not only download Aurorae themes from the configuration module, but also other themes.
Yesterday I blogged about why Breeze is not the default window decoration in KWin 5.0. The blog post touched a little bit the problems with our decoration API. In short: it’s QWidget based and that doesn’t fit our needs any more. It uses a QWidget as an X11 window. At the same time KWin intercepts the rendering and also input handling, redirects it and forwards it. So why use a QWidget at all? Also using a QWidget is quite a memory waste in the Qt5 world. The QWindow behind the QWidget uses a QXcbShmImage with the same size as the window. As explained in yesterdays blog post the window has the size of the managed window plus the decoration. So for a maximized window we hold an image of the size of the complete window while we just need the titlebar strip. We can do better
Our decoration API is also showing it’s age. It’s cumbersome to use, too difficult to use. In fact there is a KDecoration and a KCommonDecoration – the latter trying to make KDecoration easier to use by for example providing buttons. The API got extended several times to support more features which are all optional. The API is difficult to use from KWin side as it’s not stateful and quite often needs to call into the decoration API calling virtual methods the decoration API provider needs to implement. Last but not least it’s difficult to test new decorations as we don’t have a dedicated viewer application to test the interaction and painting. You have to kind of use KWin as the development host. Not the best solution.
The idea for a new API had been in the room for a long time. I opened a bug report for it more than two years ago. Last week I finally started with the implementation and tackled three things at the same time:
- New decoration API
- Viewer application
- Breeze decoration
In the current state it looks like this:
The outer decoration is the existing Aurorae theme. It looks already quite good, but there is of course still lots of work to do. The API is not yet feature complete, it needs implementation in KWin and Breeze needs a pixel perfectionist to get it right (which I am not). And that’s where you can help! Thanks to our sysadmins we have a new todo board and I requested a project for KDecoration2 and prefilled it with very easy tasks. This is a wonderful opportunity to work on easy and new code and helping both KWin and the Visual Design Group achieving an important new step. Without your help we won’t have this in 5.1 and it would be so important for both KWin and the overall design of the Plasma desktop. So please grab the code and start hacking. Grabbing the code, where? The new decoration API can be found in the git repository kde:kdecoration, the viewer application in git repository kde:kdecoration-viewer and the new Breeze window decoration in git repository kde:breeze in branch “graesslin/kdecoration2”.
So what makes KDecoration2 better? Obviously it’s no longer QWidget or QWindow based. Instead it’s a pure QObject based API. It provides a paint method a plugin needs to implement which gets a QPainter passed into. This allows KWin to control the rendering and to render to the best suited backend for the current compositor (e.g. just a QImage or a Shm shared pixmap). Also the input handling will be controlled from the backend side by sending appropriate events to the decoration. The API takes care of all of the handling – activating the buttons forwarding title bar presses, etc. This alone makes the API hopefully much easier to use from plugin side. It also provides the base implementation for all the required buttons, so that the decoration only needs to provide the painting of the buttons. Simplified the API looks like the following:
There’s an additional singleton DecorationSettings which provides common settings for the decoration. Also there’s an additional second private API which must be implemented by a backend. This will allow us to use KDecoration2 in multiple places. I want to see it uses in KWin, but in future I’d also like to provide our decorations for QtWayland. For this I hope that KDecoration plugins can also be a solution.
This week we finally released Plasma 5.0 including KWin 5.0 and also a new design called “Breeze”. While Breeze provides a window decoration, KWin still defaults to Oxygen and that’s for a good reason. As I had been asked quite often why that’s the case and on the other side got lots of feedback from disappointed users using the Breeze decoration I think it’s needed to explain in a blog post the technical background.
I start with explaining how our window decorations work in KWin 4. KWin is a so-called re-parenting window manager. This means the managed X11 window is put into another X11 window providing the windwow frame. In KWin we use a QWidget for the window frame. Thus we are also restricted to what QWidget provides us. Our window decoration API predates the Compositing support and this adds quite some restrictions to it. Our solution is to intercept all paint events on the decoration’s QWidget and suppress it, trigger a repaint of the compositor and in the rendering pass ensure the decoration widget repaints to a temp image which then gets copied into a texture to be rendered by the Compositor.
The Breeze window decoration theme is based on the Aurorae theme engine. As I’m the main author of Aurorae I can bash it in this blog post without feeling sorry about it Aurorae was designed to make it very easy to style a decoration and to make use of the new added translucency features. Being a solution which could be used as a default decoration was never the aim. The idea was to allow users who want the customizability this feature while the majority of the users could use the faster native themes. Aurorae was never fast and will never be fast. Over the time the performance improved, especially thanks to using QML. But also in that case it was only usable with the raster graphics system in KWin 4 times.
Now in KWin 5 the usage of QML is the main problem which makes it difficult to use Aurorae at all. QtQuick uses the SceneGraph and uses QWindow instead of QWidget. That’s quite a bummer for our QWidget based API. We adjusted the internal usage to support QWindow based decorations but that was quite a tough road as there are differences in the behavior of the windows. As it’s no longer QWidget based our interception of paint events is broken and we needed a new solution for it. And this solution is even more ugly than the old one because QtQuick is nowadays rendering through OpenGL. Due to limitations in Qt’s OpenGL implementation (might be addressed in Qt 5.4) we cannot share with the OpenGL context used by QtQuick. But we need to get the content from the Aurorae window into our OpenGL texture. The solution is to render to a QOpenGLFrameBufferObject and read it back into a QImage just to upload back to a texture. Not only is this a huge overhead to copy the content from GPU to RAM and back to GPU it’s also wasting lots of memory. The frame buffer object has the same size as the window and that’s way larger than the actual window decoration. In case of a maximized window it’s not just the title bar area but the complete window. And that overhead exists for each window.
That alone might render Aurorae completely unusable. I’m currently using the Breeze theme and KWin needs more than 200 MB of RAM – not really acceptable. But the situation is even worse. With QWindow we don’t get to know which areas got updated. So whenever e.g. a button gets updated we have to repaint the complete window including the complete copy of the decoration content. Which especially in animation situations is quite a problem.
The last problem to mention is OpenGL. QtQuick is supposed to use a dedicated rendering thread, but Qt disables that for all Mesa drivers and instead uses the main thread. But that’s also the thread KWin’s compositing OpenGL context lives in. That’s quite a performance problem and also introduces lots of instability problems (might also be addressed in Qt 5.4).
Overall the situation is so bad that colleagues recommended to disable Aurorae overall in the 5.0 release. As we released with it you can see that I disagree in that point. There is hardware where it’s working quite fine – on my system it doesn’t matter that it uses lots of RAM and if your driver supports Qt’s threaded rendering the performance problems are mostly gone and also all stability problems are gone. So the situation hasn’t changed: for users who want to customize there is a solution but it might introduce a loss of performance. But as a default theme like Breeze it’s clearly not an option.
So what’s the road forward? I started implementing a new decoration API removing the restriction of the decoration being QWidget based and at the same time I started implementing the Breeze decoration with this new API. I hope that we can introduce this in KWin 5.1. I’ll write another blog post about it soon and also prepare lots of small and easy tasks for new developers to join in this effort and help making a kick-ass decoration solution for 5.1
Now that I have your attention: the binary of KWin/5 just got renamed from “kwin” to “kwin_x11”. For you as a user nothing changes, the startup is adjusted to start kwin_x11 instead of kwin. Nothing else changed. The D-Bus interface is still org.kde.KWin, the config file is still “kwinrc”, etc. etc. Only if you start KWin manually remember to run “kwin_x11 –replace &” instead of “kwin –replace &”.
Say thanks to Hrvoje Senjan for preparing the patches to rename the binary and adjust all the places where the binary name is used.
This week we had the beta release for our upcoming release of the next iterations of our Plasma workspaces. This also includes KWin 5.0 which has the first major transition since the introduction of Compositing back in 2008. The changes in KWin are huge as we ported to Qt 5 and with that also to XCB and QtQuick2. Personally I comare it to having exchanged the engine, the tires, adding new spoilers and getting a new finish for our car. But after such a huge change there will be the one or other screw which needs to be tightened to get the fastest car in the race.
And that’s where we need YOU. We cannot find all the small issues. We need you as a tester to know what to fix. So please give a try to our beta, our daily packages and weekly isos and try as hard as possible to break KWin. I want that KWin 5.0 has at least the same quality as KWin 4.11 and I’m sure you want that, too.
Head over to David’s blog post for general testing instructions. Of course KWin bugs should be reported against product “kwin” on bugzilla.
As you might have read in Aleix’s blog post the kde-workspace repository got split into several repositories. KWin was also affected by this change. As the largest module in the old kde-workspace repository and given the fact that it has always been a rather stand alone part in kde-workspace it got moved into an own repository. If you want to clone it just run:
git clone kde:kwin
This change will hopefully make development easier, especially following all commits should be easier. A nice change is that KWin now does the dependency resolving itself which removes quite some quirks we had in the old setup. All of kde-workspace had X11 and a few other dependencies optional, but for KWin they were mandatory. Now all mandatory dependencies can clearly be stated.
The new KWin repository does not contain all source code as it used to. The Oxygen window decoration got moved together with other parts of Oxygen. But this doesn’t change anything for KWin: Oxygen is still the default window decoration, just the dependency is now at runtime. If Oxygen is not around KWin can fallback to the Plastik window decoration just as it used to be.
Furthermore the documentation for the configuration modules got moved into the new repository, so all the documentation is now also directly in KWin. And also the new kwin-compositing-kcm got imported into the new repository. This got initially developed in an own repository to ease the initial development (which started on Qt 5 when KWin was still on Qt 4), but we had the plan to merge it back into KWin once the repositories are split. Note for distributions: you can drop the package in case you packaged it already. Sorry if that created some extra work for you.
Bug fixes for the 4.11 release still need to happen from the old kde-workspace repository as that’s the one used to make the releases. Commits to this branch will get cherry-picked by me to the new repository.
One of the new features in KWin 5 is a completely rewritten configuration module for our Desktop Effects. In KWin 4 our module was based on KPluginSelector, which is a great widget for a small list of plugins, but it was never a really good solution for the needs of KWin.
Also we noticed that a QWidget based user interface is not flexible enough for what we would like to provide (e.g. preview videos). So when QtQuick came around we had the first experiments with reimplementing the selector with QtQuick, but with the lack of what today is QtQuick Controls it never left the prototype state. But it encouraged us to use one GSoC project on redesigning the control module from scratch and Antonis did a great job there to lay the foundation for what we have now in the upcoming alpha release.
The most noticeable change is that the new control module just focuses on the Desktop Effects. What we learned from our users is that they are only interested in configuring the effects and that the other options exposed in that control module bare the risk of users changing and breaking their system. Thus we decided to give the users what they need and move all other options into another control module.
In order to give the users the possibility to focus on the effects we also did some cleanup in the list and all effects which are not supported by the currently used compositing backend are hidden by default (e.g. OpenGL effects when using XRender). Also all internal or helper effects are hidden by default. These are effects which replace functionality from KWin Core or provide interaction with other elements of the desktop shell. Normally there is no reason for users to change that except if they want to break their system. That’s of course a valid use case and so there is a configuration button to modify the filtering of the list to show also those effects.
Last but not least our effects got extended by information on whether they are mutual exclusive to other effects. For example one would only want to activate the minimize or the magic lamp effect. Both at the same time result in broken animations. For effects in a mutual exclusive group the UI uses radio buttons and manages that only one of the effects can be activated. That’s the change I’m most happy about.
Check out the video to see the new configuration module in action and also see some of the new features I haven’t talked about. Please don’t tell me in the comments about padding issues and rendering problems. We can see those, too, and are quite aware of them. If you want to help iron out issues with Oxygen and QtQuick Controls check have a look to our wiki page.
We are currently approaching the release of the first alpha version of KWin 5. This is of course a significant milestone and we have put lots of work into the transition to ensure that it’s already working quite fine. With such a big migration it is also important to not only work on new cool stuff, but also to do lots of house cleaning.
For example last week the build option for disabling scripting support got removed as we consider scripting as an important and integrated part of KWin. Thus during the house cleaning the build option got removed.
And of course there are areas where we wanted to remove some legacy code for quite some time but never really dared to do. Two years ago I already discussed the costs of supporting the legacy OpenGL 1 compositing backend. Now it was time to re-evaluate the situation.
KWin 5 is a strong user of QtQuick 2. In fact almost all user interface code is written with this new framework – be it the close button in Present Windows, the window switcher or (since today) the outline shown when using quick tiling/maximization. QtQuick 2 uses an OpenGL (ES) 2 powered scene graph thus KWin already has a runtime dependency on OpenGL 2. And in opposite to our Compositor this is a required dependency. While it has always been possible to disable the Compositor (and that won’t change for X11), it is not (yet) possible to disable the OpenGL usage of QtQuick.
Thus we have to ask whether it’s still worth keeping a code path for legacy hardware, if we require newer hardware. Combined with all the problems mentioned in the above linked blog post from two years ago it becomes really questionable to keep the code.
After some preparation to ensure that we have a clean cut I pushed today the changes to remove the legacy OpenGL 1 compositor in order to have this change in Alpha 1. We encourage all users who used to run the OpenGL 1 compositor to try with the OpenGL 2 compositor and also with the still supported XRender based compositor. If you run into any severe issues please report a bug (or check whether one has already been reported) and provide the support information. This will help us to iron out any issues, like ensuring that no effects get loaded which are not suited for your hardware. After all the base OpenGL 2 compositor should not be an issue for most hardware. But some effects are more demanding than the compositor itself and those can be disabled automatically. Also we try to keep the impact low. If our logic recommended OpenGL 1 compositing for a specific set of hardware, it doesn’t enforce OpenGL 2 now, but uses the better suited XRender compositor.