Posted on

Turning the world upside down

This blog post is a rather important one for me. Not only is it the first blog post which I write in a nearby cafe sitting in the sun and enjoying spring, it is also the first blog post written in a Plasma session running inside kwin_wayland.

This marks an important step in the process of getting Plasma and KWin ready for Wayland as we have reached a state where I can dogfood kwin_wayland on one of my systems. It means that I’m confident enough to write a blog post without having to fear that the session crashes every few seconds (granted I press save a lot nevertheless).

So what do I mean when saying that I’m running a kwin_wayland session? Does it mean that everything is already using Wayland? No, unfortunately not, rather the opposite: all running applications are still using X11, but we use a rootless Xwayland server. The only Wayland application in this setup is Xwayland and KWin itself.

Nevertheless it’s a noteworthy achievement as we now have the architecture pulled upside down. No longer does KWin connect to either an X-server or a Wayland-server which does the rendering to the screen, it does it now all by itself. Over the last week I worked on the DRM (Direct Rendering Manager) backend for kwin_wayland. The DRM backend will become the primary backend for KWin, doing mode-setting, creating buffers for rendering and performing page flips of those buffers on the connected outputs and rendering the mouse cursor on the outputs. In the current state it’s able to get all connected outputs and perform mode setting on them. In addition it can power both compositors: OpenGL and QPainter. The QPainter one is the rather trivial: all we need are two buffers which we map into a QImage to render on. With each rendered frame the buffers are swapped and the last rendering buffer gets presented.

The OpenGL compositor is slightly more difficult. Here we need to introduce another component: GBM (generic buffer management). Gbm allows us to create an EGLDisplay for our DRM device, create an EGLSurface for each of the outputs and get the buffer from the surface to be passed to DRM for presenting. Unfortunately GBM is currently only available on Mesa and given NVIDIA’s presentation at last years XDC it looks like NVIDIA will come up with a different solution. My hope was that this would have settled at the time when we start implementing the solution, but unfortunately that’s not the case yet. So for the foreseeable future it looks like we will first have a Mesa specific backend, then later on can add a new NVIDIA-specific backend and hope that at some future point in time Mesa also supports NVIDIA’s solution (after all it was presented as a generic solution) and we can drop the GBM backend. The situation is none I like, but at the moment we can only ignore the proprietary drivers.

Another noteworthy detail is how KWin opens the DRM device. KWin runs as the normal non-privileged user – thus is not allowed to open the device. Just like libinput integration the DRM backend uses logind to open the device file. Given that we already have support for that in KWin it was a straight forward, elegant and secure way to implement it. I know, that not everybody will agree with this solution. But of course it’s just a DBus interface and anybody can implement it.

There is still quite some work needed before this backend, which got merged into master today, will be fully functional. For example all outputs are set to (0/0) that is render overlapped sections. Here we will in one way or another integrate with kscreen to get sane defaults from the start. I’m so looking forward to being responsible for screen layouts. If I look at all the terrible things one has to do to keep properly updated on XRandR… Now we won’t render with a wrong viewport because different parts of the stack are stuck in different states and no, we won’t freeze just because applications start and call an XRandR call. There are so many things which “just work” and which have been painful on X11. From my first tests on hardware I’m confident that we will finally have working multi-screen handling which is adequate for 2015.

Of course this is a wonderful time to start working on KWin. There are lots of small tasks to work on. I am constantly adding new tasks to our todo list. And if you ask, I can always provide some more small tasks.

Posted on

There is no KEndorsements?

You might have noticed that my yesterday’s blog post was an April Fool. With this post I want to explain a little bit about the idea of the article.

First of all I want to thank the VDG, especially Jens and Ken, for their support by providing the great mockups. That was kind of the first element to notice that it’s an April Fool: normally I do only integrate screenshots by myself and I would not have been able to show a drawing in Krita.

The idea for this April Fool is based on the reports about the inclusion of an adware tool called superfish on Lenovo notebooks. Especially the initial response by Lenovo claiming that this malicious software got installed “to improve the shopping experience”.

Nevertheless the article contains truth, half truth and thing we would never do. So let’s look at some of the ideas presented and evaluate them. The first idea presented was share buttons integrated in the browser’s window decoration. This is an idea which already got introduced similarily through Share-Like-Connect. Personally I think it’s a good idea if browsers would provide such global buttons and in return filter out all the user tracking share buttons on the web sites. A global area for share buttons directly integrated in the browser could increase the security of users a lot.

The next idea is about monitioring the usage behavior of the user. For this I must say that in a Wayland world KWin would have enough information to do that, but we don’t look at the information at all. We do not care what an application is: we don’t know that a given window is “Krita”, we don’t know that it should be used with a drawing tablet, we don’t analyze any input and just pass it through to the application. KWin is completely ignorant on what the application does with input events passed to them and as ignorant to what a window content looks like. To us a window is just a texture we render to the screen. We don’t know that there is a “drawing area” or a “video” element, it’s just one texture. Similarly we do not know (and care) about input events. All we do is determine which window should get the event and pass it to it. We are quite aware that by passing all input events through KWin we could do evil things. The good thing is that our software is open source and you can see what we do (relevant code file is called input.cpp, events come in from the files under the libinput/ directory).

Last but not least I want to point out that we as a window manager will not integrate advertisement into the window management. We do not care whether you watch a video, we might not even know it, because as explained: we are ignorant about what you do. I consider applications which integrate advertisement without the user’s explicit acknowledgement as malware.

The bad part of my blog post is that I expect these things to happen. And if we look around us we can see that it already happened: adware is unfortunately also available in FLOSS. And in the walled garden’s of the proprietary world we will see more attempts to gain more information about the user’s to provide even better matching advertisement. It will blur more and more.

Posted on

Enhancing user experience through the Compositor

This article is an April Fool. For an explanation please see the update.

Some time ago Ken introduced the conecept of dynamic window decorations (DWD) and during the last Plasma sprint there was already some work on experimenting with an implementation. DWD are extremely promising to enhance the user experience by morphing the application and window manager scope together.

The basic concept of DWD is to get content of the application inside the scope of the window manager. But what works one way works also the other way: we can enhance the user experience by providing additional information relevant for the window inside the window decoration.

An example for this could be a volume applet for music and audio players. But of course it doesn’t stop there, today is all about the user content and integrating the web. For web browsers we can integrate a global “Like It on Facebook” button. And by analyzing the user behavior we can provide even more help. We can notice if you struggle using Krita and provide a link to where to get a better drawing tablet.

If you use digiKam you might be interested in some books about photogrophy. We know which music you listen to on Amarok and show you that the new album of your favorite music group just got released. Not every application integrates stores in a proper way. This is a shame as it does not provide the best possible user experience and KWin as a window manager can help there to add missing store functionality where it is needed.

Of course this also allows to show you better alternatives. If we see that you struggle with wine emulated applications we can provide the Linux alternative. We can show that new updates are available for your currently used application. If you use sub-par free software applications we can show you were to get a proper (slightly more expensive) application.

But DWD are not the only place where KWin can help to increase the user experience by offering useful content. We always went a great way to make the desktop more useable by for example allowing to click on a splash screen to hide it. I think we can all agree that splash screens are something really useless: look at a stupid window for a few seconds and having to wait till it hides again. Meh.

We can do better: we know which application starts and what it is used for. And as a compositor we can exchange the content. Wouldn’t it be way more awesome to show a short video giving you advice where to buy the latest tips on C++11 book when launching kdevelop?

And there is so much more room to integrate useful content. When starting a video player we can overlay the window with a nice trailer of the latest movie (of course that will be skipable after 5 sec). This is a feature which might also be handy in Present Windows and Alt+Tab. Of course such videos would be perfectly localized as we know your geolocation and we would take great efforts to only offer trailers which might interest you: after all we have baloo to scan your video data and can also look at your Facebook profile. Now trailers are only one part, of course also other areas could be covered. If you have many pictures of cars you might be rather interested in spots about the latest car of your favorite brand instead of move trailers. Of course we will make it possible to configure which areas you are interested in. As we all know KDE is all about configurability.