Posted on

What a Week

I am just enjoying my last day at Tokamak 5 in Nijmegen and wow that has been a productive week. Personally I have of course mostly worked on the Plasma Compositor and Window Manager but I also helped on getting the Compositor better integrated into the various Plasma shells. My plans were to work on an easier to use Effects API and JavaScript binding but of course I did not have the time to work on it 😉 This task is no a little bit delayed and might be moved to the next release cycle as I would not be able to finish it till the hard freeze anyway.

There was quite some backlog of work from the OpenGL 2/OpenGL ES 2.0 work and I fixed all the issues I knew of. So e.g. cube and sphere effect are working again, there is a config option to enforce legacy OpenGL 1, the logout effect works again and KWin requires at least Mesa 7.10 if you use free drivers.
For Plasma Active I implemented a way to close windows from Present Windows effect in a nice touch friendly way. It will most likely not end in Plasma Active as we want to have something better than the normal Present Windows effect for tablets. KWin no longer requires window decorations which makes it possible to compile KWin completely without window decorations to reduce the package size and KWin won’t just crash if there is no decoration. As a nice side-effect that helps myself during development in case Oxygen evolved and I was not aware of it. If KWin is compiled without window decorations all windows are opened as maximized. In order to reduce the package size we are also looking into limiting the set of effects for Plasma Active and removing most of the window manager related configuration modules.
I finally merged the refactoring of the Outline handling by our new Google Summer of Code student Arthur and added an effect to it, so that it can be nicely styled and looks much better.
There are many more ideas to be implemented especially to further improve the user experience by better integrating the Plasma Compositor with the Plasma Shells.

=-=-=-=-=
Powered by Blogilo

Posted on

About Windows and Widgets

Last week a blog post on planetkde suggested to merge the functionality of Plasma and KWin. I quickly replied with a blog post that this is technically not possible and by the developers not considered. Jos asked me to write a blog post to explain this a little bit further. Who can say "No" to Dutch people and so here it is:

Windows and the Window Manager

First let’s have a look at Windows and the Window Manager. In the KDE Plasma Workspaces KWin is the Window Manager. That means it is responsible for managing windows. Ensuring that the right window has focus, adding window decorations, remembering the order of usage and much much more.

To the Window Manager a window is just a unique Id and completely transparent. The Window Manager has no idea what the window is about, what it looks like and why it is there. The window itself can set some properties which are used by the Window Manager to manage the window. This includes things like maximization, the desktop it is on, the icon to show and many more things mostly defined by the NETWM specification. Now the window system itself (currently X11) does not include all these things. For example the concept of virtual desktops is only provided by the window manager. For KWin a "window" is not a window at all. It is a "Client" to the manager. All windows (including the window manager) are in fact X11 clients. So the window manager does internally not manage windows despite its name but clients.

Many concepts are implemented several times. So a window is able to move itself as well as the window manager is able to move the window. In order to make this all work the window and the window manager must agree to some general protocol. This mostly works quite well, but in the end the window manager will have the final word. There are windows which are misbehaving and the window manager is able to fix their bugs.

KWin is a very advanced window manager and probably the window manager with most available features. This means that KWin gives you quite some candy to customize your personal window management, but nevertheless most just works due to common sense.

Windows and the Compositor

Next to KWin being a superb Window Manager it is also a compositor. The task of the compositor is to render the windows on screen. In legacy setups each window got assigned a part of the framebuffer and the XServer ensured that the areas are clipped correctly. When a compositor is around the window redirects the painting into an off-screen pixmap and the compositor renders the pixmap as a texture on the screen.

As the compositor has access to all window textures and is mostly integrated into the Window Manager we can put transformations on the windows. We can animate the appearance of windows, lay all windows out in a grid and much, much more. Without these "effects" there would not be much difference between the legacy way of rendering and the composited way. It’s just nicer looking with compositing and in general should be way more efficient (which may not be true with X11 but will be true with Wayland).

The compositor has no idea what it renders on the screen. In the end it’s just some texture rendered to the screen. In most cases it is just rendering a 2D texture in a 2D space. Very simple and very primitive OpenGL. The compositor tries to optimize the rendering by ensuring that only changed areas are rendered and that windows obscured by opaque windows are not rendered. In the end the compositor is just optimized to render windows. This is important to remember for the further understanding.

Plasma and Widgets

Plasma is first of all a framework to create desktop shells. We currently have the Plasma Desktop, Plasma Netbook and the Plasma Active shells. The shells can contain different widgets (aka Plasmoids) and lay them out. This layout is very different per shell. In the desktop shell it is possible to fully move the Plasmoids around, you can rotate them, scale them and do whatever you want. In the Netbook Shell on the other hand the Plasmoids are layed out like in a newspaper. It is much more constrained than in the Desktop shell. In the panel the constraints are even further: the Plasmoids are also constrained in size. Most are dropped down to an icon and expand when clicked. All this is achieved by having different containments for different fields of application. It is an important part of the support for Plasma’s device spectrum.

In opposite to the windowing system Plasmoids are completely controlled by the containment they belong to. They cannot move themselves, they cannot always change their size, they cannot start demanding attention or raising themselves above other Plasmoids. The system is quite different if we look at the details.

Plasma and the Compositor

To the compositor the Plasma shells are just a texture like any other as well which is rendered as a whole. As explained above the compositor does not know what is rendered at all and that there are Plasmoids. Currently the Plasma developers are working on libplasma2 which will make us of the new QML Scene Graph to do hardware accelerated rendering of the Plasma stack. This Scene Graph can be fast because it knows about the internals of the QML scene. Something the compositor would never know even if it were in Plasma. No matter how good our compositor would be it will never be on par for rendering plasmoids directly.

Plasmoids in the Window Manager

One of the claims in said blog post was that widgets are just a kind of windows and that therefore they should be known to the window manager. Plasmoids should be accessible through Alt+Tab, should show up in Present Windows and should just behave like normal windows. Now lets think about the implication. Right now I have two open windows and six open widgets in my panel including things like Kickoff and the systray. Who would want kickoff been shown in Alt+Tab? Who wants the latest Dilbert comic strip taking away valuable screen estate in Present Windows? Who wants to focus on the Weather Widget?

Of course there are widgets which are like "apps". But most are not. In general at least on the desktop it would be terrible to have Plasmoids available in the window manager.

Throwing around code bases

Now of course we could put Plasma and KWin into one application and create a monster. But what would be gained? The window manager component still does not know anything about Plasmoids neither does the compositing component, nor can the widgets component start to include windows. Just imagine you would try to allow that: putting LibreOffice Writer into the panel – great idea.

In order to get the window manager component to know about Plasmoids it is required to make Plasmoids windows. If we do that we don’t need to put the source code together – that can be achieved by the existing solutions. But doing that would pull in all the disadvantages of the legacy X11 system. KWin would need to be changed to know "this is a Plasmoid", KWin would need to duplicate all the special handling for the different containments, all other Window Managers would need to do the same or Plasma would only continue to work with KWin. As Plasma is just one big window the Plasmoids can make use of translucency without compositing. If they become windows they would need compositing to function at all. This is quite a change in requirements. Currently developing a new Shell is very fast, but when we start to do that in the window manager it would be complicated work, taking months to achieve what we now can have in days or hours.

About Merging Shells and Compositor

This month sees the first releases of two new Desktop Shells which are merged with the Compositor: GNOME Shell and Unity. I assume this is the motivation for the idea behind "Plasma and KWin should merge". Users might think that this is an advanced solution, but while preparing this blog post I tried to think up some advantages and I could not find even one. If you want to know about advantages you would have to ask GNOME Shell and Unity developers.

Both systems do to my knowledge provide nothing which cannot be achieved with KWin and Plasma. In fact I would go so far and claim that Plasma/KWin has a higher level of integration than Unity/Compiz or GNOME Shell/Mutter. We do that for quite some years now and are pretty good in doing this. And it gives us quite some advantages: You don’t have to use KWin to run Plasma, you don’t need compositing to run Plasma. The integration parts we develop can be used by other applications. Like the sliding animation written for Plasma is reused by Yakuake. The new shadow system is primarily developed to support widget styles and Plasma to set the shadow on windows. Our system is completely developed on the existing frameworks like X11, but is also compatible with Microsoft Windows, Mac OS X or in future Wayland. It is our believe that we can reach the same level of integration by defining a good API than to merge KWin and Plasma and to be fully open to any other window manager to work together with Plasma. Our custom additions are all well documented and mostly implemented at least by Compiz.

Summary

Even if it is not understandable for users there are differences between widgets and windows. In general you would not want the window manager to manage the widgets. To bring in a small comparison: who would want the window manager to manage and composite all browser tabs, in the end Facebook and GMail are just "apps", aren’t they?

KWin’s compositor is damn stupid and optimized for rendering 2D textures in 2D space. The QML scene graph is way better suited for rendering widgets than KWin’s compositor and making KWin render the widgets would be a large step back. From a technical point of view it does not make any sense to merge the two applications.

=-=-=-=-=
Powered by Blogilo

Posted on

KWin and Plasma Active

Plasma Active is an extremely awesome project and I am really looking forward to work on it and have my first own Plasma powered tablet. Of course KWin will be the Compositor and Window Manager in Plasma Active. And this is pretty awesome and very interesting for our future development.

KWin on OpenGL ES 2.0

Plasma Active will probably be the first large installation to make use of our new OpenGL ES 2.0 based compositing backend. In fact OpenGL ES is our primary target because the API is way more advanced than desktop GL and gives better results. Unfortunately the decision whether to use desktop GL or OpenGL ES is a compile time decision. As not all drivers support OpenGL ES on the desktop we still need to default to desktop GL. On Plasma Active we know the driver to be used and therefore can compile against OpenGL ES if supported.

OpenGL ES is far less complex than desktop GL. The desktop stack still carries around the complete outdated and deprecated OpenGL 1.x functionality. It is not surprising that writing a desktop GL driver is difficult. Because of that I expect that the hardware running KWin in Plasma Active will be faster, more energy efficient and magnitudes more stable. It will be the first important step to leave the messy state of OpenGL on Linux behind us.

Different Focus

While Plasma Active still requires a window manager, the superb functionality known from our desktop and netbook shells are not required. By targeting touch screens without keyboards quite some changes to our way of thinking how to manage windows are required. My personal pet Alt+Tab is completely useless on such a device. In order to make it usable we will need to adjust and drop the requirement of holding the "Alt" key. This will of course benefit the desktop users: switching windows could become possible with multi-touch gestures on a touchpad.

Another area of change will occur for window decorations. In fact in Plasma Active there won’t be window decorations – they are just not needed. No, I don’t twist my mind, we won’t introduce client side decorations. We just won’t have decorations. The only relevant button (close) is put into the panel. Also that will need some small tweaking of the KWin source code like many other little advantages we have to make which all will also benefit the desktop.

KWin and the Device Spectrum

Plasma supports the complete Device Spectrum by using custom shells and different QML files for the widgets. Now KWin does not (yet) support the various form factors in such a way. From the window management perspective there is nothing to change. Since the introduction of the netbook shell we know that adjusting KWin just means to change the configuration. KWin already includes all the little tweaks required to have a pleasant experience on systems different to the desktop.

Also in the compositor there is hardly anything to do. We support OpenGL ES and the general compositing doesn’t change just because we run on a gadget. There might be some adjustments required for the drivers as we did not yet have the chance to play with many of them but in general I expect everything to "just work".

The only field which requires adjustments are the effects. Many effects are just not suited for such devices and we will not ship them. In fact I expect that we won’t even allow to customize the window manager and effect system. The danger to harm the system are too high. And some of our effects need adjustments for the use on such a device. Let’s take the Present Windows effect as an example. It allows to close a window when hovering over it. Closing windows from the overview is damn useful (I got the inspiration from Maemo), but hovering is just not possible on a touch device.

In order to properly support both Present Windows on the desktop as well as Present Windows in Plasma Active we need two distinct Present Windows effects. Now writing something like Present Windows is far from trivial. The effect is complex and around 2000 lines of code. We need a better solution for that.

Over the last weeks and months even before I knew about Plasma Active I started to think about how I want to have effects being developed. My aim is that Nuno is not only able to write effects, but that he wants to write the effects. My task is to give him the tools to do so. Over Tokamak next week I will sit down with our QML and JavaScript binding experts to discuss and implement a solution for KWin. A subset of the kwineffects API will be stabilized and be made available for bindings. After lots of thinking I am sure to have found a solution which allows us to write effects in JavaScript and/or QML without spending time in the interpreted part during screen rendering. This will allow us to write effects in a more suited language with the speed of compiled C++ code. The first effect I want to port to the new system will be the Present Windows effect.

In that regard I want to have Plasmate also as a development tool for KWin effects. This will take some time, but I have ideas how we can get previews for the animations and effects outside of KWin. I am both looking forward to implement these parts as well seeing them used.

New Technology for the Desktop

Some of my last blog posts were rather controversial and that was not by chance. I wanted to test out how users will accept the changes which are implied by Wayland. I wanted to see how much acceptance there is for removing features on the Desktop in order to go to a new windowing system. I am very thankful for all the comments I received and that there was some media coverage about these posts. It was very enlightening to read all those articles and comments to them.

My fear that it is not yet acceptable to aim for Wayland was overall confirmed. Especially for KDE it seems that our users do not want us to remove any features. Everything has to be configurable, we "may not go down the GNOME road". For the adoption of Wayland we have a chicken and egg problem: as long as it is not used, applications won’t be ported, as long as applications are not ported it won’t be used. This is similar to what KDE faced in late 2007.

Now Plasma Active gives us new possibilities. Having compositing always enabled is not just a nice thing, it is a requirement which we can fulfill thanks to the advanced drivers compared to the broken state on desktop. We can experiment with moving the screensaver into the compositor without ensuring backwards compatibility. We even do not need to support widgets in screen locker on Plasma Active. This gives us the advantage of first implementing the new functionality and later add the backward compatibility layer for the desktop systems.

And now: Wayland. Wayland is a very interesting technology but it will be difficult to embrace. There is so much depending on X that we have to either port everything and then do the big switch or dare to break functionality. Both is hardly possible on the desktop. Announcements like Mark Shuttleworth’s "Ubuntu 11.10 might use Wayland" do not cause joy in me but fear that it will harm the platform like the too early switch to PulseAudio. Wayland is too important to risk that users think it’s bad because someone just wants to be quick.

Plasma Active on the other hand is a wonderful candidate to embrace Wayland. The stack would significantly benefit from dropping X11 and using Wayland as the windowing system. The driver situation is good: the devices run OpenGL ES 2.0 which is a requirement for Wayland. But most important: we don’t need the window manager. We only need the compositor and effect system – both is basically independent of the X11 stack and can rather easily be made Wayland ready. On Plasma Active we don’t have window decorations, so we don’t need to port them and so on and so on. This gives us a third possible strategy for approaching Wayland: port step by step and use it and do not switch on the desktop before everything is put together. Please don’t over estimate this statement: it is a long road to Wayland and the first release of Plasma Active will not use Wayland. There is lots of work to be done in KWin before we will consider to add support for Wayland. This will most likely not happen in 2011.

How to get involved

KWin can use helping hands. We are only three developers and none of us can work fulltime on KWin. There is enough work to be done for fulltime developers, so we need new blood. Especially after the first support for JavaScript and/or QML is added we need help to get effects written in that language and Plasmate adjusted. These are things I would not want to do. So please get in touch with us either through the mailing list or in one of our IRC channels: #kwin, #plasma or #active are all fine. Please don’t overrun us, we are a small group 😉

=-=-=-=-=
Powered by Blogilo

Posted on

Why Blur Does Not Work in Kubuntu Natty With Intel

Over the last week we received quite some complaints about blur not working after an upgrade to the latest beta of Kubuntu Natty. So far we could not make anything out of it. All users had already been using Plasma Workspaces 4.6.2 in Maverick and were often using the Xorg Edgers drivers. So you would assume that they had more or less the same software versions like before the upgrade.

Furthermore we had not changed anything. We haven’t touched blur effect for some time – a git log does only show changes relevant to our development version. So a very strange happening. Now finally we received a bug report enlightening us. Apparently the Intel driver changed the renderer string in a minor driver release, causing our direct rendering check to disable direct rendering and by that blur cannot be enabled any more.

Here on my notebook running openSUSE 11.4 the renderer string looks like the following: "Mesa DRI Intel(R) Ironlake Mobile GEM 20100330 DEVELOPMENT" Our code to test whether direct rendering is supported looks like this:

// Assume that direct rendering works with DRI2 drivers

const GLubyte *renderer = glGetString(GL_RENDERER);

if (strstr((const char *)renderer, "DRI2"))

return 0;

// The Intel driver doesn’t have DRI2 in the renderer string

if (strstr((const char *)renderer, "GEM"))

return 0;

The code has last been changed on June 13th 2010! The section for Intel driver has been included end of May of last year. So the code is around for nearly a year. Now according to the bug report the Intel driver dropped the "GEM" from the renderer string. With the result that direct rendering does not get enabled anymore.

This change has happened in a minor release of Mesa. And I’m asking: Why? Why change what is not broken? Why introduce such a change in a minor version? Why oh why? I would accept it for a major version. We would have known beforehand and could have adjusted for our next major version. But we cannot change something like that in a minor version. And it would not help the situation. Natty will ship 4.6.2 and our next bug fix release will be after the release of Natty and I won’t accept a patch to that code as I fear that it could break for other Intel users.

I must say that I am very disappointed by this change and very, very sad. After all the trouble of the 4.5 release caused by the broken state of graphics drivers on Linux the driver developers again broke one of their most important downstreams. Maybe the message has not yet come through: nobody will use your drivers on Linux to run fancy games if the basic compositors do not work. So it is completely unacceptable to break Mutter, Compiz or KWin. Those three applications are your most important targets and whatever you do: DON’T BREAK THEM!

The sad state is that the driver developers do not even do the most simple regression test after such a change like starting into Plasma Workspaces and see if everything is working fine. It is also disappointing to see that they do not know what parts of their driver string is parsed by their most important clients. I’m now doing this kind of stuff for more than three years and changing the driver strings seems to be a hobby of the free drivers. The only driver which has in that time not change the pattern is NVIDIA. Apparently they recognize that customers may parse this information and rely on it. But of course as we all are open source evangelists it would be completely unacceptable to recommend the use of the NVIDIA driver or to recommend people to buy NVIDIA cards because their driver is bad, bad, bad just by the fact that it is closed!

Now we all remember the time of the 4.5 release and how the drivers announced support for what they don’t support. At that time the driver developers justified themselves with we should have known better, asked or at least assume that if the drivers say they support version X, that they in fact only support version X-2. Now my question: how should we handle such custom adjustments for broken drivers if the drivers change how they can be recognized?

The last time people complained that we did not communicate with the driver developers, but that I wrote an angry blog post just like this one. So I need to explain: Nobody informed us that the driver string will change. Now I am kind of stupid, I know. Even in my worst nightmares I would not expect that the developers change the version string in minor revision. Still I could talk to them now instead of writing angry blog posts? Sadly I cannot. First of all we were notified about the change too late to do anything. As just explained we were notified after 4.6.2 and no chance to adjust before Natty. Furthermore I was ill last week, are sitting in a plane to the US right now, so I can write blog posts but cannot use Internet and I will stay more or less disconnected throughout the next week.

Luckily KWin is prepared for driver fuckups and there are workarounds to this problem. An affected user can start KWin with "KWIN_DIRECT_GL=1 kwin –replace &" to enforce direct rendering and skip the checks.

A few weeks ago I suggested on our mailing lists to enforce compositing. Overall the response from my fellow developers was that they expected further driver fuckups in the future causing a hard time for our users. I disagreed and pointed out the improved driver situation and that after the 4.5 fiasco the message has got through. How could I have been so naive? It looks like we will never be able to have a decent composited sAetup with this mess of a stack 🙁

=-=-=-=-=
Powered by Blogilo

Posted on

KWin and Plasma Won’t Merge – Not Even in “KDE 5”

Today reading through planetkde gave me quite a WTF moment. The merge of KWin and Plasma was announced for KDE 5. This is quite a WTF if I do not know about that. In fact I do not even know of anything “KDE 5”. So be assured: there are no plans to merge KWin and Plasma – not even in a hypothetical KDE 5. This does not make sense, would bring no benefits and the “advantages” posted in the blog post cannot be achieved by merging the code bases.

It would be nice if bloggers would at least ask one of the maintainers of the software in question if they have any plans in that direction before posting it on planetkde which is picked up by media. We have enough work to do without correcting non-sense or discussing how best to break the desktop.

=-=-=-=-=
Powered by Blogilo

Posted on

Rethinking Screensavers

Screensavers are a relict of the last century. They were required to prevent serious hardware damage on CRT screens. This is even implied by the name: a screensaver saves the screen from damage. Now we live in the second decade of the 21st century and the world has changed. I can’t remember when I last saw a CRT computer screen, they have been completely replaced by LCD screens. LCD screens on the other hand don’t show the danger of burn-ins. They don’t need to be “saved”. Nevertheless we still have screensavers.

Nowadays screen savers are used to indicate that the screen has been locked. It is a way to animate the screenlocker. But is that wise at all? Especially in the current time it is extremly important to save power. This means if a screen is locked the screen should turn itself off, the CPU should go into the lowest power state, the compositor should stop repainting the screen, the GPU should go into the lowest power state and so on and so on. But is that possible if a screen saver is active?
The screensaver implementation in the X11 world is rather dated: XScreenSavers were first published in 1992. To be honest: most screensavers even look like being from the last century. Now if a screensaver is active, the CPU has to calculate the animation which implies that it cannot be in idle state. The compositor needs to update the screen which means the compositing engine cannot turn itself off. This implies again that the GPU cannot turn itself in the sleep mode and can cause really heavy load if the compositor needs to constantly do full repaints. Last but not least if the system DPMS is misconfigured the screen will not turn off.
In summary a system will waste power in a situation where it does not need to use any power. It is even possible that (depending on the used screensaver) the system needs more power in the idle state than under usage. Given the world we live in, I want to do something about it. We need to stop wasting power.
The waste of power is not the only disadvantage of the current screen locker system. Several of the existing screensavers have issues as they pre-date the current infrastructure with compositors by years. We have seen that parts of the screen can be leaked which means there can be privacy issues. Even worse is the situation with OpenGL screensavers. They sometimes use ARGB windows and don’t clear the screen (this used to work fine in the pre-compositing era), all areas not repainted by the screensaver are leaked. In case you have a buggy driver it is even likely that either the screensaver or the compositor will not survive that both are used. According to our bug reports this is a common problem. Now driver bugs should be fixed but reality shows: it is not happening.
For the future there is only one obvious solution: the screen locker has to be moved into the compositor. The compositor can ensure that the screen is blanked correctly and not leaking information. If the locker is part of the compositor, it can ensure that the compositing re-paint loop is stopped as long as nothing has to be shown. Only if the unlock dialog is shown the compositor needs to enable repainting and can ensure that everything else except the dialog is blanked. Another small advantage of having the screen locker in the compositor is that we can finally do a nice fade in/out when locking/unlocking the screen.
Removing the screen saver is nothing which will happen for 4.7 as it requires coordinated development. Even if we move screen locking into the compositor we need to ensure that the old architecture is still in place in case KWin is not used or compositing turned off. There is also the Plasmoids on screen locker implementation which is a valid use case and needs to be ported to any new infrastructure. This can become tricky, but I am positive that we will be able to support it. Altogether this is something I want to have for 4.8 and it is something I want to discuss at Tokamak and will make sense to be included in Plasma Active as we there have the control over the compositor which is used and saving as much power as possible is very important for the targeted devices.

=-=-=-=-=
Powered by Blogilo

Posted on

Welcoming a new OpenGL Compositor

With the release of GNOME 3 the third major Window Manager (Metacity) entered the field of OpenGL based compositing in the incarnation of Mutter. Congratulations! Now I know some will say that Mutter was around before, but with GNOME 3 it’s the first release for a non-niche product like Moblin and Unity (10.10) and is targeting desktop computers as well. I hope to find the time to play around with GNOME Shell a little bit. I wanted to try some weeks ago but even on my Intel system I only got the fallback mode 🙁

Personally I am very interested in some of the aspects of Mutter and I am eager to see how Mutter will perform under the new conditions it will be put through by the users. While KWin and Compiz are very similar using C++ and OpenGL directly, Mutter is very different. As the name implies it uses Clutter as a scene graph. So instead of directly transforming the scene with OpenGL, Mutter uses an abstraction library wich does not directly map the OpenGL semantics.
Internally KWin also has it’s own scene graph, which allows us to write Effects without any knowledge of OpenGL. So for example Present Windows uses only the built-in functionality to translate and scale windows and the same code base works with all our compositing backends: XRender, OpenGL 1.x, OpenGL 2.x/OpenGL ES 2.0. The rendering itself is done using our own custom OpenGL code. Compiz is from the architecture very similar which allows us to share ideas and sometimes even code (Compiz and KWin are much closer as you might think, for example we have the same perspective projection matrix). Sharing more than ideas with Mutter/GNOME Shell seems to be impossible, unfortunatelly.
Given my own experience I know that introducing a new OpenGL compositor can be difficult. It takes time till you know all the required quirks for all the various hardware and drivers and it takes many bug reports to learn about them. Here I am very interested to see how Mutter will perform as the compositing is based on an existing OpenGL abstraction layer. Will they hit the same problems as we did and if yes how will they be able to deal with them given that they cannot change the rendering code directly? KWin has learned a lot about drivers during the last year and nowadays we can go down to disable specific features based on hardware chips, drivers and versions. This allows us to tackle severe issues even in minor revisions.
Another point I’m interested in is the usage of a scene graph library in general. KWin’s internal scene graph is tamed to the needs of an OpenGL compositor. On the other hand Clutter is a general purpose scene graph not only developed for compositors. Of course you would expect that Clutter’s OpenGL is better given that the developers are only working on OpenGL and do not need to maintain a window manager. But a compositor is very different to all other applications. For us the most important part is the texture from pixmap operation, which is hardly needed by non-compositors. But advanced OpenGL functionality is hardly required in an compositor: KWin for example does not use a z-buffer or stencil buffers.
As indicated in my last blog post I would like to be able to always enable compositing for all users. Here I want to thank all people who commented on the post – this is highly appreciated and the feedback will be evaluated. Now Mutter or in fact GNOME Shell is a step ahead: they require compositing. But as I also outlined in my last post there are situations where you don’t want to have compositing – and I know what I’m talking about: my primary system does not wake up from suspend if compositing is enabled. I am really interested in seeing how Mutter handles such situations where you actually don’t want compositing and I hope to learn from them in these aspects. I am really looking forward to talk about such issues at Desktop Summit with the Mutter developers.
The last important point I will study with the advance of Mutter (and also Unity) is the tight integration of Compositor and Desktop Shell. In the KDE Plasma Workspaces Compositor and Desktop Shell are two separate processes which allows to use KWin with any Desktop Shell (excluding GNOME Shell and Unity) and Plasma with any Window Manager (with or without compositing support). Given the new development of missing interoperability between desktop shells, keeping the option to use a different window manager becomes less important. In fact with Wayland it might even be stupid to not have the desktop shell (and many other parts) in the compositor and switching compositors might be impossible at all. I really want to know about the advantages of having Desktop Shell and Compositor in one process and one rendering graph. There are many aspects in Plasma which can be done better in the compositor and we already make use of it – like the sliding popus, but there is much more. This is btw. a wonderful way to get involved with KDE development 🙂

=-=-=-=-=
Powered by Blogilo

Posted on

Turning Compositing off in the Right Way

I have a dream: a dream of an always composited desktop. There are use cases for using compositing and there are usecases for not using compositing. For the user it is very difficult to know what he currently needs and can do a very bad job at deciding himself. A good example for that is turning compositing off to save some more minutes of battery. I personally doubt that turning compositing off will save battery, but will cause a further drain. How is that possible, you might ask? When compositing is turned off, Plasma starts to change the backgrounds of all SVGs causing in the worst case a re-rendering of all of them. This is costly and will most likely drain more battery than using compositing. 

Another example are fullscreen applications. Currently KWin supports unredirecting of fullscreen windows. This means the screen is no longer composited, but the resources, that is the OpenGL context and the effect system is still running. For applications like a web browser or an office suite it is completely useless, while for OpenGL applications like games just unredirction might not be enough. If you have one of those awesome drivers not supporting two OpenGL contexts at the same time, you might see either artefacts or a crashing KWin. For games there is only one proper solution: turn off compositing. The same is true for watching Full-HD videos: often graphics cards are not powerfull enough to do both compositing and GPU accelerated decoding.
Now KWin supports the solution for this: suspending of compositing. Just press Alt+Shift+F12 (or for the more technical users: use a script to change the state through DBus) to suspend compositing. The OpenGL context is removed, the effect system turned down and you have the plain old X desktop. Of course we cannot demand from our users that they know about it and can handle it. Therefore we need a bettter system.
This is what Thomas has been working on lately. First of all he removed the difference between disabled effects and suspended effects. If you disable effects it will just suspend them and the effect system will be in suspended state after a restart. This should make everything more clear to the user. The second part Thomas has been working on is allowing applications to block compositing. This is pretty awesome. Let’s say we want to watch a video in VLC. As soon as you would switch to fullscreen, VLC would set an X property to tell KWin “now please don’t composite”. KWin will suspend compositing and keep the state untill no window blocks compositing. So the GPU is completely free for VLC. Now what is the difference to the unredirection, you may ask? Imagine a user notification would pop up. With unredirection compositing would be started again, causing an ugly flickering and taking away important resources from VLC. With the new solution a notification would not cause a restart of compositing. Everything is still with VLC. On the other hand a web browser would not block compositing as that is not what the user wants. Here we need the complete advantage of a composited system as it’s not a single task such as watching a video or playing a 3D shooter. I really hope that video players, games and Wine pick up our new property and we will also recommend it as an additon to the NETWM specification.
Though the last piece might turn out difficult. While Plasma completely supports being without compositing, the world looks different with the two new Desktop Shells to be released this month. In my humble opinion both are fundamentally flawed concerning the fallback to no compositing. My biggest concern towards GNOME Shell has been from the beginning that it requires OpenGL (I talked about that part with Owen Taylor at GCDS). There is no fallback except GNOME Panel. Which at least looks for me unacceptable to switch the desktop shell just because you want to watch a Full-HD video. Now Canonical did the same fundamental mistake with Unity. It also requires an OpenGL compositor (in that case Compiz). While Compiz nowadays supports non-OpenGL I do not know how good this is and whether Unity supports it. Currently the fallback is also GNOME Panel, in future maybe Unity2D? Plasma on the other hand just needs to switch the rendering of SVGs (which is part of the style) and will lose some functionality, such as thumbnails in taskbar tooltips, Present Windows effect or Desktop Grid. The basic working with the system remains the same. Another big thumbs up for getting the right abstraction layers.
Now with Thomas work I am confident that we will be able to remove the UI to turn on/off compositing in maybe 4.8. It will just not be needed any more. The applications will take care of providing the right user experience to the users. When compositing is needed, it is on, when compositing is bad at the moment, it is off. The users won’t have to care about the state anymore and an ugly hack like unredirection of fullscreen windows can be removed. I am looking forward to such improved ways of desktop compositing 🙂

=-=-=-=-=
Powered by Blogilo