A KDecoration2 update

Before heading into the weekend I thought about writing a small update about the KDecoration2 status. Since my last blog post I started integrating KDecoration2 into KWin. This was partially easier and partially more difficult than anticipated. Especially ripping out the old decoration code is rather complex. There are quite some design differences which make the transition complex and especially values inside KWin core are using enums defined in the decoration API – e.g. the maximized state is kept as a KDecorationDefines::MaximizedMode. This will need further work to move the enums and so at the moment the old decoration library is still compiled although the library is no longer in use.

Ah that means there is code? Yes, today I pushed the branch as “graesslin/kdecoration2” into kde:kwin git repository. To give it a proper try you also need the kde:kdecoration (master branch) and kde:breeze also on “graesslin/kdecoration2” branch. The new decoration API is working quite well and I’m rather satisfied. The memory usage of KWin dropped significantly. In a previous report I mentioned that KWin needs around 200 MB, right now my KWin only needs around 40 MB, the number of open windows is a little bit smaller, but still it shows in the right direction. And that’s without any optimisations. There is still some optimization potential in Breeze (check out our todos, all purple tasks are in Breeze) and in KWin (red tasks). Also a nice improvement is that the window decoration no longer flickers when resizing the window. This is a rather big annoyance of KWin 5.0. I expected that the new API would fix this issue, but seeing it confirmed is really nice 🙂

Last but not least the restart of KWin got faster which is a nice improvement for KWin developers, for users it’s not that important. The reason for this is that when we enabled/disabled compositing we recreated the window decorations. So when restarting KWin we first created all windows with their decoration before enabling compositing to just destroy them and create them again (yes it would have been possible to improve it, but it doesn’t hit users). Now with the new API there is no need to recreate the decoration. Only a Renderer is exchanged, but the rendering is delayed till it’s needed.

This screen shot does not only show the new Breeze decoration, but also some new features: when quick tiling a window the borders cornering the screen edges are removed. Also in the preview application one can see that the decoration scales which is nice for high-dpi screens. The decoration API follows the approach Plasma is using.

I have been running a KWin with the new decoration API since Monday, which means we are close to start the review process. But there’s of course still work to be done and I’d appreciate any help. The window tab API is not yet implemented, the configuration module needs adjustments to load and render the new deco and most important we have to port the existing decorations like Oxygen, Aurorae, deKorator and QtCurve. For the last three I want to improve the theme experience by allowing the plugin to say that it supports themes directly in the JSON meta data and point to where to find them. So the configuration module can just show all of them. The trick will be to pass the to be used theme to the factory method (we have a nice QVariantList there which can be used). Also the meta data will make it possible to point to GHNS configuration files so that we can not only download Aurorae themes from the configuration module, but also other themes.

KDecoration2 – The road ahead

Yesterday I blogged about why Breeze is not the default window decoration in KWin 5.0. The blog post touched a little bit the problems with our decoration API. In short: it’s QWidget based and that doesn’t fit our needs any more. It uses a QWidget as an X11 window. At the same time KWin intercepts the rendering and also input handling, redirects it and forwards it. So why use a QWidget at all? Also using a QWidget is quite a memory waste in the Qt5 world. The QWindow behind the QWidget uses a QXcbShmImage with the same size as the window. As explained in yesterdays blog post the window has the size of the managed window plus the decoration. So for a maximized window we hold an image of the size of the complete window while we just need the titlebar strip. We can do better 🙂

Our decoration API is also showing it’s age. It’s cumbersome to use, too difficult to use. In fact there is a KDecoration and a KCommonDecoration – the latter trying to make KDecoration easier to use by for example providing buttons. The API got extended several times to support more features which are all optional. The API is difficult to use from KWin side as it’s not stateful and quite often needs to call into the decoration API calling virtual methods the decoration API provider needs to implement. Last but not least it’s difficult to test new decorations as we don’t have a dedicated viewer application to test the interaction and painting. You have to kind of use KWin as the development host. Not the best solution.

The idea for a new API had been in the room for a long time. I opened a bug report for it more than two years ago. Last week I finally started with the implementation and tackled three things at the same time:

  • New decoration API
  • Viewer application
  • Breeze decoration

In the current state it looks like this:

The outer decoration is the existing Aurorae theme. It looks already quite good, but there is of course still lots of work to do. The API is not yet feature complete, it needs implementation in KWin and Breeze needs a pixel perfectionist to get it right (which I am not). And that’s where you can help! Thanks to our sysadmins we have a new todo board and I requested a project for KDecoration2 and prefilled it with very easy tasks. This is a wonderful opportunity to work on easy and new code and helping both KWin and the Visual Design Group achieving an important new step. Without your help we won’t have this in 5.1 and it would be so important for both KWin and the overall design of the Plasma desktop. So please grab the code and start hacking. Grabbing the code, where? The new decoration API can be found in the git repository kde:kdecoration, the viewer application in git repository kde:kdecoration-viewer and the new Breeze window decoration in git repository kde:breeze in branch “graesslin/kdecoration2”.

So what makes KDecoration2 better? Obviously it’s no longer QWidget or QWindow based. Instead it’s a pure QObject based API. It provides a paint method a plugin needs to implement which gets a QPainter passed into. This allows KWin to control the rendering and to render to the best suited backend for the current compositor (e.g. just a QImage or a Shm shared pixmap). Also the input handling will be controlled from the backend side by sending appropriate events to the decoration. The API takes care of all of the handling – activating the buttons forwarding title bar presses, etc. This alone makes the API hopefully much easier to use from plugin side. It also provides the base implementation for all the required buttons, so that the decoration only needs to provide the painting of the buttons. Simplified the API looks like the following:

There’s an additional singleton DecorationSettings which provides common settings for the decoration. Also there’s an additional second private API which must be implemented by a backend. This will allow us to use KDecoration2 in multiple places. I want to see it uses in KWin, but in future I’d also like to provide our decorations for QtWayland. For this I hope that KDecoration plugins can also be a solution.

Why Breeze is not the default window decoration

This week we finally released Plasma 5.0 including KWin 5.0 and also a new design called “Breeze”. While Breeze provides a window decoration, KWin still defaults to Oxygen and that’s for a good reason. As I had been asked quite often why that’s the case and on the other side got lots of feedback from disappointed users using the Breeze decoration I think it’s needed to explain in a blog post the technical background.

I start with explaining how our window decorations work in KWin 4. KWin is a so-called re-parenting window manager. This means the managed X11 window is put into another X11 window providing the windwow frame. In KWin we use a QWidget for the window frame. Thus we are also restricted to what QWidget provides us. Our window decoration API predates the Compositing support and this adds quite some restrictions to it. Our solution is to intercept all paint events on the decoration’s QWidget and suppress it, trigger a repaint of the compositor and in the rendering pass ensure the decoration widget repaints to a temp image which then gets copied into a texture to be rendered by the Compositor.

The Breeze window decoration theme is based on the Aurorae theme engine. As I’m the main author of Aurorae I can bash it in this blog post without feeling sorry about it 🙂 Aurorae was designed to make it very easy to style a decoration and to make use of the new added translucency features. Being a solution which could be used as a default decoration was never the aim. The idea was to allow users who want the customizability this feature while the majority of the users could use the faster native themes. Aurorae was never fast and will never be fast. Over the time the performance improved, especially thanks to using QML. But also in that case it was only usable with the raster graphics system in KWin 4 times.

Now in KWin 5 the usage of QML is the main problem which makes it difficult to use Aurorae at all. QtQuick uses the SceneGraph and uses QWindow instead of QWidget. That’s quite a bummer for our QWidget based API. We adjusted the internal usage to support QWindow based decorations but that was quite a tough road as there are differences in the behavior of the windows. As it’s no longer QWidget based our interception of paint events is broken and we needed a new solution for it. And this solution is even more ugly than the old one because QtQuick is nowadays rendering through OpenGL. Due to limitations in Qt’s OpenGL implementation (might be addressed in Qt 5.4) we cannot share with the OpenGL context used by QtQuick. But we need to get the content from the Aurorae window into our OpenGL texture. The solution is to render to a QOpenGLFrameBufferObject and read it back into a QImage just to upload back to a texture. Not only is this a huge overhead to copy the content from GPU to RAM and back to GPU it’s also wasting lots of memory. The frame buffer object has the same size as the window and that’s way larger than the actual window decoration. In case of a maximized window it’s not just the title bar area but the complete window. And that overhead exists for each window.

That alone might render Aurorae completely unusable. I’m currently using the Breeze theme and KWin needs more than 200 MB of RAM – not really acceptable. But the situation is even worse. With QWindow we don’t get to know which areas got updated. So whenever e.g. a button gets updated we have to repaint the complete window including the complete copy of the decoration content. Which especially in animation situations is quite a problem.

The last problem to mention is OpenGL. QtQuick is supposed to use a dedicated rendering thread, but Qt disables that for all Mesa drivers and instead uses the main thread. But that’s also the thread KWin’s compositing OpenGL context lives in. That’s quite a performance problem and also introduces lots of instability problems (might also be addressed in Qt 5.4).

Overall the situation is so bad that colleagues recommended to disable Aurorae overall in the 5.0 release. As we released with it you can see that I disagree in that point. There is hardware where it’s working quite fine – on my system it doesn’t matter that it uses lots of RAM and if your driver supports Qt’s threaded rendering the performance problems are mostly gone and also all stability problems are gone. So the situation hasn’t changed: for users who want to customize there is a solution but it might introduce a loss of performance. But as a default theme like Breeze it’s clearly not an option.

So what’s the road forward? I started implementing a new decoration API removing the restriction of the decoration being QWidget based and at the same time I started implementing the Breeze decoration with this new API. I hope that we can introduce this in KWin 5.1. I’ll write another blog post about it soon and also prepare lots of small and easy tasks for new developers to join in this effort and help making a kick-ass decoration solution for 5.1

Next Generation Klipper

A few weeks ago I contacted Thomas Pfeiffer with the idea to design a new user interface for Klipper in Plasma 5.1. Surprisingly he informed me that a discussion was already started in the KDE Forums. Which is awesome as that means there was already some ideas on how the user interface could look like. Last week the number of new bug reports for KWin get lower so I started to look into Klipper for 5.1.

The old Klipper

The user interface of Klipper is based on a context menu and this restricts the possible interaction patterns. The task of Klipper is to hold a history of clipboard items and allow the user to select an old clipboard item and make it the current clipboard content. For this a context menu is a good solution as it allows the user to just select an item, it gets synced to the clipboard and the menu closes.

But Klipper offers more than just the clipboard history. It can perform various actions on the clipboard content:

  • Edit the clipboard content
  • Show a QR-code for the clipboard content
  • Remove content from history
  • Invoke actions (e.g. open link in browser) on clipboard content

Due to the nature of the context menu as user interface these actions can only be performed on the current clipboard item. Technically there is no reason for this restriction, but the user interface requires it. This in turn makes the usage interaction cumbersome: to show the QR-code of a previous item one needs to open Klipper, select the item – Klipper closes, open Klipper again and select the action to show the QR-code.

Also a problem with the current Klipper is that it doesn’t integrate with the environment. It sits in the notification area, but opens a QWidget styled menu. This makes Klipper an odd item in the system tray behaving different to all other items.

How Klipper should work in Plasma

A clipboard history is of course an important part of a desktop shell and thus should be a first class citizen. The user interface needs to be integrate and this means the interface needs to be provided by a Plasmoid which needs to be added to the notification area. The interface would still show a list and this is best done by providing the data in the form of a QAbstractItemModel.

As there should only be one clipboard history manager, but at the same time perhaps several user interfaces for it (e.g. one panel per screen) the QAbstractItemModel holding the data needs to be provided by a DataEngine. So overall we need to separate the user interface (Plasmoid) from the data storage (DataEngine) and turn the existing Klipper in just being the data storage.

How Klipper works internally

Internally Klipper consists of a History with HistoryItems in a circular double-linked list with a marked first item in the circle. Whenever the History changes the context menu gets repopulated on next show. Whenever the first item in the History changes Klipper syncs this item into the clipboard. This means the complete logic of Klipper interacts only with the History class. Which makes it a nice and clean interface. In a simplified class diagram Klipper looks like the following:

For creating the DataEngine we need access to the History and all items in it and put it into a QAbstractItemModel. There are two possible solutions: connect the QAbstractItemModel to the existing History or use the QAbstractItemModel as the new storage for the History instead of the own double linked list. As the current use of the History is to just mark everything as dirty and then to re-populate connecting the model to the History doesn’t sound like a good suggestion. Instead I investigated in using the model as the storage end. The design is then modified to look like the following:

To achieve this transition I first created a unit test to ensure that my transition doesn’t break the existing code. Then I created the model from scratch as well covered with unit tests and in a third step I hooked it into the existing History. With this transition both the use cases of the existing code base and the new DataEngine can be satisfied.

DataEngine, Service and co

With the model in place it’s time to make it available to Plasma. The tool of choice is the DataEngine which allows to export a model and can provide a Service allowing us to interact with Klipper. Of course it would be possible to add invokables to the model, but that would kind of destroy the design as the model is only supposed to be the data store. A Service which then interacts with our existing Klipper API is way cleaner. The task of the Service is to start ServiceJobs which will in turn interact with Klipper. Overall this looks now like this:

The Plasmoid

The last and maybe most important part of the rework is the user interface in form of a new Plasmoid. But that’s also the part I’m least interested in (perfer to work on backend) and are hoping for help on it 🙂 Given that the current user interface can and should only be considered as a draft. But it’s fully functional and support text and image elements, a filter (note: klipper supported filtering for quite some time, just start typing) and all the actions. But now the actions are available on each item and not just the latest one. In the current state with a vertical panel it looks like this:

Be aware that this is material for Plasma 5.1 – it won’t be part of the upcoming release. Also I should point out that it’s still possible to run the stand-alone Klipper application and that no features are removed. All global shortcuts and actions are still supported.