Aurorae 3: Window Decorations with QtQuick

Once upon a time there was a window decoration which was KDE’s default decoration. But years ago it fell into a deep sleep. The world changed while the decoration slept. Compositing was added, decorations received shadows, Qt introduced the graphics system raster and many many more changes.

Now after years the decoration temporarily woke up, but it is no longer a beauty, but ugly and old. The changing world made the decoration break. The decoration’s bits started to rot and this is visible. The truth is that the decoration will stay in the broken state as there is nobody who would want to fix it or anybody knowing the code.

What I just highlighted here is true for basically all old decorations. Of the decorations we ship only Oxygen and my theme engine Aurorae are maintained. This is a situation which can no longer continue. We have to face the truth and remove all old decorations.

But of course we don’t want to remove decorations without adequate replacement and this leads us to the story of Aurorae.

Aurorae was written to experiment with new features introduced in KWin 4.3. It was based on our common decoration API and by that is rather limited. So for 4.5 I decided to port it to the generic decoration API and reimplement with GraphicsScene instead of QWidgets. Aurorae allowed to download decorations through Get Hot New Stuff.

But given that it is a theme engine the possibilities are still very limited. Only what the theme engine supports can be rendered. Porting our old decorations to the theme engine might not be possible and not to be recommended.

But still we would like to have those decorations on a modern basis without the maintenance costs. So I started to port Aurorae to QML. This does not only give us a more powerful theme engine but also introduces QML bindings for window decorations, allowing everybody to write decorations with QtQuick.

So we can write themes without the limitations of a theme engine ๐Ÿ™‚ This does not only remove 3000 lines of C++ code and replaces it by around 1000 lines of QML code, but also gives us an interactive decoration control module.

Interactive decoration control module

For Aurorae based themes all the buttons are interactive giving our users the possibility to really try the decoration before using it. But that’s not where it will end. I want to have new themes not based on Aurorae but using QML directly. For that I want to use the Plasma Package structure and would love to see integration in Plasmate. Another idea is to support deKorator themes inside QML.

But as always this does not come without disadvantages. Not all features which were possible with Aurorae will be available in Aurorae 3. Some features will also be dropped because hardly any theme makes use of the features, for example special backgrounds for non-composited setups. Aurorae 3 will hardly work without compositing, so it might be that we will make it a hard requirement somehow.

I hope to see many innovative QML based window decorations in 4.9.

What’s new in KWin Scripting?

First of all a happy and successful 2012 to everyone. Let’s work together to make this year a great success for KDE!

One of the big topics in 2012 for my development efforts in KWin will be JavaScript bindings and QML. This is a technology which will allow our users to build their own custom window manager giving even greater power to our static window rules as before. It will also reduce our maintenance need when major parts of our UI can be transitioned to either QML or JavaScript.

The work on this area has already started and the first code set has been merged into master (aka 4.9). The old scripting API had manually crafted bindings for around 30 properties for windows. Now the API is generated and everything that is available for effects is also exported to JavaScript, which means more than 60 properties, more properties than before are writable and many more signals are available. Interestingly I could drop several hundreds lines of code by exporting more properties and the code becomes cleaner and better documented. These are changes I really like ๐Ÿ™‚

The API documentation of KWin scripting is now auto-generated through Doxygen and already imported to techbase together with detailed Update Notes on what changed. If you run master and use KWin scripting be aware that you have to adjust the script. Exactly the same API will be available in QML and also the Effects API is very close – given that the property names are inspired by the Effects API.

One of my favorite changes is that our Plasma Desktop Console is now able to execute KWin Scripts:
Plasma Desktop Console executing a KWin script

To open just use KRunner and type the new keyword “wm console” or start the desktop console and switch using the toolbar button.

This is a very important change for me during development as I can easily test whether my scripts are correct and most important my implementation. But also for users it means that scripts can be tried at runtime and don’t require a KWin restart any more. So adjusting the runtime behavior becomes possible.

So what’s next? I am currently with the progress quite satisfied with the work done so far on scripting and will now concentrate on two other areas first: getting my unmerged kickoff-qml and screenlocker branch into master. Both require still a little bit of work. After that scripting and QML get my number one priority again.

KWin meets QML

Two years ago I started to re-implement the Window Switching capabilities of the KDE Plasma workspaces (also known as Alt+Tab or TabBox). The old implementation was based on a QWidget with custom painting which of course had some drawbacks. The new TabBox evolved into a framework around Qt’s Model-View concept to design your own window switcher.

Before I started to work on that area I did evaluate the existing Window Switching capabilities of the various desktop shells (both free software and proprietary solutions) and I noticed that all of them have drawbacks. None is the solution where I would say: that’s the perfect switcher.

The main issue is that depending on the user’s habits different work flows are required. A switcher optimized for users with few open windows fails for users with many open windows and vice versa. Especially thumbnails become useless for many open windows.

So I realized that we need something flexible enough to give users the possibility to use different layouts or to even define their own layout. This ended in a framework for window switching. The layout is described with an XML file and rendered through custom painting code in a Qt delegate.

When I implemented this framework I wanted to add a GUI to easily configure own layout. Unfortunately I never got around to implement this UI and I think there has never been any other layout except the default shipped once. It is a rather unknown feature and I’m quite unhappy that this great functionality never got implemented.

In the mean time the trolls did a great job to make UI development easier: QML. This makes it possible to do what I wanted to have for window switching: easy to design and exchangeable layouts. So this weekend I implemented the default (informative) layout in QML and plan to completely replace the existing View and Delegate code in 4.8. And this gives us the possibility to let users write their own layouts and to share and exchange them.

But this is only the beginning of usage of QML in KWin. I have many more ideas by making window thumbnails available to QML UIs in a limited way. This will allow us to even integrate the BoxSwitch effect into the normal window switching framework and replace the Window Strip in Plasma Active.

So great UI times ahead in KWin.

New Screen Locker

Some might have already read about it: we are currently working on a new screen locker. The old implementation is mostly a hack around X. There is the lock window which tries really hard to be the top most window. Obviously this does not often work, especially as the whole framework is way older than Compositing.

Too often I have come back to my system or resumed from suspend and seeing my windows shine through. Or seeing some notifications on top of the screen locker. We decided to change that and solve it once and for all in a proper way by putting security first and moving the screen locker into the Compositor.

The Compositor controls the rendering, so it can ensure that nothing gets above the screen locker. When we have the compositor being responsible for the screen locking we also get nice fading from and back to the desktop for free.

My initial idea was to just move the screen locker to KWin but not changing the unlock dialog. But now I started to play around with QML and need your help:

QML unlock dialog

You can see that I am really no expert in doing user interfaces. So if you have some QML experience or just want to help defining a very important area of Plasma, please get in contact with us. I will do all the JavaScript and coding, so that only the QML is needed.

The new way unfortunately conflicts with the existing screen savers. They are almost 20 years old and we have problems to keep support for them. At the moment we decided that if you have a screen saver configured the old implementation will be used with all the security issues. But we are not yet sure what to do in 4.9. As we use QML it would be possible to write new screensavers in QML. This would allow us to keep the improved security and to only need one code path. But we are unsure how our users will react on removing the X Screen Savers. To find that out we set up a poll on forum.kde.org and I would like to ask you to participate. This will help us to define our next steps.

The new screen locker will also be used by the first version of Plasma Active, which will be released in a few days. There we use the same architecture but ship a different QML file which does the unlocking without needing a password (we only want to protect for unwanted touch events). This shows that it is possible to exchange the QML files for custom screen savers.

Moving the screen locker into KWin has also more security advantages in addition to the protected screen content. Of course KWin is very crash resistant. So if the screen locker crashes KWin will just restart and lock the screen again. Even if KWin would completely fail it is more secure than before as without a window manager you can hardly do anything with the system. Also we have finally a real solution for the most often reported KWin crash of all time: the Intel driver crashes KWin when the screen saver is used due to unredirecting the fullscreen saver window. With the locker being a KWin effect it is impossible to trigger this driver crash.

Some users might wonder what that means if they don’t use KWin. Well as the screen locker has been moved to KWin you can no longer lock the screen if not using KWin (the same way you could not lock the screen if not using KRunner before). If you want to lock your screen in Plasma 4.8 but don’t want to use KWin you should get in contact with your Window Manager vendor. Sam from Compiz wanted to add support for it in Compiz. For anything else I don’t know. So why did we do it: we think that the security of most of our users is more important than the need of some users to use a different window manager. Also we had added many integration features to Plasma in the past and if you were not using KWin you lost these features. With the screen locker it is now exactly the same: you lose the integration feature when not using KWin. I also want to remember everybody who wants to complain in the comment section that the KDE Plasma Workspaces are an integrated product consisting of the Plasma Desktop Shell, KWin, KDM, Systemsettings and a few other applications. We develop this as one product, so if you want to remove a part, well that’s your decision, but we have to compromise and we clearly see that other vendors are integrating their window manager even further into the desktop shell than we do.

Power Saving and Desktop Effects

For a long time KDE Plasma offered to turn off Desktop Effects when battery runs low. In the past we changed the default to keep compositing on and now in 4.8 we remove the option completely.

We all care about power saving and so it is important to understand why keeping compositing on will most likely save more power than turning it off. (As with all hardware and driver dependent functionality we can only offer what is best for the majority of users, it is possible that with specific hardware turning compositing off would in fact save more power.)

So let’s have a look at what happens when you turn off compositing:

Turning Compositing Off

Window decorations

Our default window decorations uses a shadow build into the decoration itself. When you turn off compositing the shadow has to be removed. Given the way the shadow is implemented, this means that each window has to be resized and moved back to its position. Furthermore all window decorations have to be completely repainted.

Plasma

Everything that is painted in the desktop shell is a pixmap rendered from SVG. Plasma uses a cache for the recently generated pixmaps. The themes itself come in three forms: one for no-compositing, one for compositing and one for compositing with blur. When turning off compositing the theme has to switch from one of the two composited to the non composited. This means we need to access the hard disk to load the SVG. Then we have to render the SVG in the correct size to a pixmap and access the hard disk again to save the pixmap in the cache. This has to be done for each Plasma styled window: the panel, KRunner, the tooltips, Alt+Tab, Kickoff. Also in later states during using it is likely that the same cycle happens again: e.g. the size changes of KRunner. With compositing it’s most likely in the cache, with non-compositing it isn’t.

KWin itself

When turning off compositing KWin has to be quite busy. All the effects need to be turned down, lot’s of resources have to be released. The OpenGL context needs to be destroyed. The overlay window has to be removed, quite some properties need to be removed from the root window (support for blur, taskbar thumbnails, etc.), the manager selection has to be released. So overall quite some roundtrips to the X Server till everything is teared down correctly.

X Server

With compositing disabled the X Server becomes responsible for rendering the screen again. It no longer has to redirect the windows into offscreen pixmaps and notice KWin through the damage extension about changes. It has to update the screen and perform the clipping of the content. Pretty much the same what KWin did before.

How the Desktop Effects work

As we see overall turning compositing off is a rather heavy operation. It involves many applications, many context switches and lot’s of I/O. All the things you don’t want to do when running low on battery. But later on having compositing turned off must really bring power saving, right? To answer that we have to look on how KWin performs the rendering.

How KWin renders

KWin is notified by the X Server about an update in a window (damage) and triggers the next repaint. The repaint starts with asking all loaded effects whether they are active. In a normal situation only the blur effect will be active, all other effects won’t participate in the rendering. Each participating effect is asked to perform its transformations and additional rendering. In the normal case the blur effect notices that the window does not have to be blurred and does exactly nothing.

The windows are rendered from top to bottom. That is if we have an opaque window on top, the one below won’t be rendered. Only if it is translucent it will be rendered from bottom to top (that is all windows). So in the normal case (area of a window updated) only one window will be updated. The rendering itself is just a very simple 2D texture rendering. Yes it’s 2D, not 3D.

After the window is rendered KWin goes back to sleep to wait for the next damage event, it does not perform constant rendering.

But it’s using OpenGL, that’s expensive on the GPU!

Well think about it. Someone has to bring your updated windows on the screen. This someone is the graphics card. You need it anyway, no matter whether it is KWin or the X Server. KWin uses OpenGL to do the rendering, which means it’s hardware accelerated. I do hope that the X Server uses hardware acceleration to do it’s rendering ๐Ÿ˜‰ In the most modern architectures (like Gallium3D) 2D is just a special case of 3D. I honestly don’t see how using X instead of KWin to perform the 2D rendering should save any power.

But can’t you at least deactivate the effects?

As explained above only active effects will participate in the rendering. So if you don’t use the cube effect, it won’t be used and cannot consume power. If on the other hand you use the effect than it is adding usability features. Imagine a user who switches windows using the Present Windows effect. Of course using this effect actively will affect the battery life. But what is worse? Using the effect once very shortly or searching a minute for the window in the taskbar? What will affect the battery life more? The minute searching (one minute less battery) or the animation?

But at least Blur should be disabled!

It is true that Blur is the most expensive effect we have depending on the hardware you have. That’s why we are optimizing the effect (thanks to Philipp for all the work on it). First of all: if the rendering of the window does not intercept any blurred area, the blur effect won’t do anything. If the rendering intercepts a blurred area the part has to be updated. Thanks to some optimization we only have to update this part and we cache the blurred background. So blurring is no longer that expensive as it was. Now we could nevertheless turn blur off, but that would require to generate new pixmaps (see above) and I doubt that this is better than doing some blur operations.

What about the animations?

Animations are probably the only thing where it might make sense to deactivate when running low on battery. Animations trigger repaints so they are not cheap. But to change that quite some work is required and I am not sure if it is worth it. So if you can do an evaluation on how expensive the fade effect is for the battery life we might consider to add a way to disable the animations.

Where Compositing helps to actively save power

There are operations where compositing is much better than good old X. For example when moving a window only some texture rendering need to be performed. With X the areas where the moved window used to be needs to be repainted. The same is of course true for resizing. Another advantage is that KWin renders with a well defined frame rate. No matter how often a window tries to render, KWin will throttle it. So a window generating damage events each millisecond will be rendered only each 17 msec. And yes there are applications out there trying to render as often as you can. Preferable some old icons in the systray or non-standard toolkits.

Where we enforce Compositing

KWin in 4.8 is the first version which will temporarily enforce compositing. This is currently related to the new screen locker. The new screen locker is implemented in the compositor to ensure that the screen is always blanked (something the old one could not guarantee). When we use the new locker KWin ensures that compositing cannot be turned off! So imagine we would allow turning compositing off when running low on battery.

First scenario: active usage. Compositing is turned off and later on the system goes to suspend and locks the screen with the old insecure implementation. I hope you all see that security is important.

Second scenario: screen locked when battery is running low. The power management functionality will try to turn off compositing, but it will fail and the power management will never notice about it. There is in fact no way how the power management could notice that and it would have to try again and again to turn off compositing. That sounds clearly wrong to me.

But 4.8 will hopefully have another situation where we need compositing: Wayland. When we run a Wayland server we will not allow to turn off compositing, because you would lose your Wayland windows. Currently the code is not yet merged and most likely the distros won’t have packages available anyway but it is something we have to plan for. With Wayland we need compositing, so turning it off doesn’t make sense.

Overall I hope you see that turning compositing off will be worse for the battery than keeping it on. We are concerned about helping the environment and giving our users the best of the limited resources. That’s why we removed the feature: to help you and not use wrong settings just because you assume that 3D is worse than 2D.

Second Strike for KWin Scripting

I already blogged about it: KWin Scripting needs help from the community if it should be available in 4.8. Unfortunately nothing changed about the state, except that I added a build flag to not compile Scripting.

I just wanted to apply a patch and got a compile error because of scripting and I remember that my GSoC student had also problems with scripting during his project.

So this is the second strike for scripting. And you know what will happen when I have to proceed to the third strike ๐Ÿ™ Step up now, or scripting will have to go.

Rendering at 60 Frames

Full credits go to Alex Fiestas for complaining about the performance in a way that I wanted to do something about it, instead of being just annoyed and ignoring the comment!

Alex’s complaints got me wondering why we are not able to render 60 frames/second. Each frame should only take 16.67 msec and knowing that our repaint loop is fine, I could not imagine how a frame could take longer to render than the 16.67 msec. Knowing the KWin’s source code pretty well I suspected two parts of the repaint loop to be slow: the method which actually renders each window and our effect chain. The effect chain calls each effect in turn to transform windows. I had an idea to improve the effect chain for quite some time by calling only the currently active effects. That is currently each effect checks first whether it is active and just does nothing. So you basically call all effects again and again and nothing is actually performed except waisting cycles on checking whether it should do nothing (some effects are heavy there).
Of course I do not just optimize without checking if the code needs to be optimized, so I did an analysis of callgrind output and I was surprised. The effect chain was way more heavy than expected. In fact it’s so heavy that the paint method doesn’t matter in comparison. A deeper analysis of the code showed that there is a small bug, which we will fix in 4.7.2 (too late for 4.7.1), so that we can give the benefits as fast as possible to our users. But the real optimization by only calling the active effects will hit only 4.8. After that change the effect chain is no longer visible in the hot pathes of KWin. Also the change immediatelly helped to identify some expensive checks in some effects which are now ensured that they are not called in each frame (unless the effect is active).
After the optimization of the effect chain we are still not yet there and do not reach 60 frames during the animation, but you can really feel the improvements ๐Ÿ™‚ Now the paint method shows up as one of the most expensive in callgrind (as expected previously) and I will now spend some time and thinking in how to optimize this one further by moving heavy operations out of the repaint loop.
In the long term I hope to be able to move the compositor into an own thread (also something I have in my mind for quite some time), so that heavy operations in window management and the decoration plugin do not slow down the repainting of the compositing. This might be tricky as we can run into dangerous deadlocks (imagine compositor and window manager both blocking the X Server), so we will probably first concentrate on moving e.g. texture loading into threads – something one of our new deelopers in KWin experimented with.
Overall very nice improvements, I’m happy with, but not yet satisfied as we do not yet reach the constant 60 frames/second.

=-=-=-=-=
Powered by Blogilo

Smooth Window Resizing

As Thomas does not blog and this is quite impressive, I had just to do a video and provide it here. Without further ado I present smooth window resizing with KDE Plasma 4.8:

(Direct Link)

Much Kudos to Thomas for fixing the XSync implementation inside KWin!

Guest Post: Crash – Exam – Relaunch

This is a guest post from my Google Summer of Code Student Arthur Arlt.

Isn’t it a funny title for a blog post? But there is a reason for that because it happend a lot in the last weeks…

Crash

Having done my first review requests for my GSoC project, my new (six week old!!!) solid state drive stopped working from one moment to the other. I tried to get it running in other systems, but it was no longer detected by any BIOS. Taking a look at the homepage of Kingston I had to discover, that they found a technical issue of exactly that brand of SSD I had installed to my system. They are strongly recommending to do a firmware upgrade because this failure could lead to a total crash of the drive…
Thanks for the information ๐Ÿ™ Luckily I made my review requests before, so at least the work done for GSoC so far was saved. But everything I did in these six weeks on my laptop is gone…
I got the SSD replaced, but this took almost two month, since the producer had to check by itself that the SSD is broken… So I bought a new one, because it was no fun working with my slow, old HDD.

Exam

I did not have too much time to be angry about the loss of data. I had to study for an exam in “Principles of Dependable Distributed Systems” which took place at Friday before Whitsun. After that, I have spent a lot of time to (re-)set up the build environment during the holidays.

Relaunch

I am now working into Martins hands supporting him to port KWin to Wayland. Especially the class Scene and its subclasses were provided with some signal and slot functionality.
A very important step to Wayland is to remove all X11 specific stuff from class Toplevel and to insert it into a new class XWindow which will then provide the functionality for X11 based windows. This refactoring emerged to be too difficult at that moment and Martin and me decided to do some other refactoring first. The parts I am currently working on is on the one hand to move all Workspace functions implemented in the file composite.cpp to an own class Compositor. On the other hand I am introducing a new class
DecorationPaintRedirector, which handles the composited decorations.

Work done so far

Some functionality of KWin was moved to own classes or even deleted. For example the broken functionality of TopMenu was removed. The functionalities of screen edge handling and Tabbox were moved to its own classes. The same happend to the tiling functionality. I also added some build option to chose whether KWin should be build with or without supporting a specific functionality. These build options are provided for Tabbox, Tiling, DesktopChangeOSD and ScreenEdges. They are automatically deactivated if building KWin for mobile devices is enabled.

Benefits

Thanks to the refactoring work done so far some parts of KWin are better structured and readable as before. One thing which confused me when I was starting coding for KWin was that in one file there could be several classed implemented. That issue was changed for some classes like Tabbox and Tiling by introducing new files or classes and by moving the related functions to the appropriate classes. These changes do not only make the code better readable but also better maintainable.

Desktop Summit

Two weeks ago I attended the Desktop Summit 2011 in Berlin. I was helping as a volunteer for video operation. Since I did not know anyone there, doing the volunteer work was a great decision. I never expected that I will get to know so many nice people. I was talking to other GSoC students and we were telling each other about our projects and the current state. Meeting some developers of KWin and Plasma was a great occasion. It’s kind of funny meeting them in real life if you had only virtual contact before ๐Ÿ™‚ Last but not least the daily social events were the best opportunity to meet new friends. And one can say: Partying with open source people kicks ass!
The days in Berlin were very exhausting, but I had a wonderful time and I don’t want to have missed it.

Outlook

The GSoC project “Modularization of KWin” did not have a specific goal to reach, since we did not know how much work it will be to refactor all this stuff. One can say that there are still lots of things to do, but I am satisfied with my work done so far. The main point for me was to get into the community and I can say that doing a Google Summer of Code is a great opportunity to achieve this. I am really looking forward to go on with my contribution on KWin because it is really fun working with such a great team.

Cheers,
Arthur

KWin turns 12

Today twelve years ago at 23:26 Matthias Ettrich did the first ever commit to KWin:

Say hello to kwin. WARNING: NOT USABLE YET. See README.

Given this nice warning we also have to show the content of the README:

This is the beginning of kwin, kwm next generation.
WARNING: this thing is hardly usable now, neither ICCCM nor KDE
compliant yet!
All it has is a context menu that allows you to switch between two
decoration styles, KDE classic and an experimental style.
Please don’t work on the code, I’ll finish it during my summer
vacations (four weeks from now on).
kwin was only commited to allow people like Mosfet to have a look at
the Client API (and StdClient) to write nifty new themable decorations.
Have fun,

   Matthias

As we can see KWin has its root in KWM from KDE 1 (there are still one or two comments in KWin source tree saying KWM) and it seems like the capitalizations was added in later times ๐Ÿ™‚ 
Matthias seems to have been a little bit too enthusiastic about how much time it needs to write a window manager. It took 8251 commits (1195 translation commits) by 284 contributors (including scripty and cvs/svn migration tools) to get to where we are today. And still we are not finished with KWin. After adding Compositing Support to the Window Manager we are now adressing the second big transition to support more than just X11. So exciting times are ahead of us.
Let’s work together to make KWin rock the next 12 years even more than the last 12. Billions of devices are out there which want to be using KWin as their Wayland Compositor. Let’s work on the future for the free software eco system!

=-=-=-=-=
Powered by Blogilo