On Friday QtCon starts and there will be of course an update about the current state of Wayland support in Plasma. See you during the lightning talk session on Friday between 17:30 and 18:30 for my lightning talk “We are in Wayland!”
Panels on shared screen edges
Plasma 5.8 will bring an improvement fixing a bug reported more than a decade ago. Back then Plasma did not even exist, the bug is reported against an early KDE 3 version. The addressed problem is the handling of panels on multi-screen setups.
This is if one has multiple screens and tries to put a panel between two screens – on the shared edge – the panel does not have a “strut” set and thus windows maximize below it:
[------------][------------] | ||P | | 1 ||P 2 | | ||P | [------------][------------]
In this illustrated setup the panel is “P” and windows on screen 2 ignore the panel. What might be surprising here is that this was not just a bug, but deliberate behavior. There is code making sure that the panel on the shared edge gets ignored. Now one doesn’t write code to explicit break useful features, there’s obviously a good reason for that.
And to understand that we must look at how panels and there struts work. First let’s look at Wayland. Wayland doesn’t have a concept for panels or struts by default. KWin provides the PlasmaSurface interface which Plasma can use to give a window the role “Panel” and to describe how the panel is used: whether it’s always on top, or whether windows can cover it or go below. KWin can use that to decide whether the panel should have a strut or not. Thus on Wayland KWin was able to support the setup shown above since it supports panels.
On X11, though, we have the NETWM spec which describes how to set a partial strut:
The purpose of struts is to reserve space at the borders of the desktop. This is very useful for a docking area, a taskbar or a panel, for instance. The Window Manager should take this reserved area into account when constraining window positions – maximized windows, for example, should not cover that area.
The start and end values associated with each strut allow areas to be reserved which do not span the entire width or height of the screen. Struts MUST be specified in root window coordinates, that is, they are not relative to the edges of any view port or Xinerama monitor.
Now here we see already the problem: it’s not multi screen aware. The strut is specified in root window coordinates. So in our case above we would need to set a strut for the left edge which spans the complete height. So far so good. But the width of the strut must be specified in root window coordinates which includes the complete screen 1. If Plasma would set this, we would have a problem.
In Plasma 5.7 KWin’s strut handling code got slightly reworked to perform sanity checks on the struts and to ignore struts affecting other screens. Basically KWin broke the implementation of given spec and in multi-screen setups only allows struts which make sense.
Now at least KWin could handle this situation properly, but Plasma still has the check to not set a strut on shared edges. For Plasma 5.8 we now changed the condition: if the window manager is KWin we allow such struts. For any other window manager we still go with the previous solution. We still think that we cannot just set a strut which would in the worst case exclude a complete screen. As that’s how the spec is written, we need to assume the window manager is standard compliant. For KWin we know that it is not standard compliant any more and support such struts, so Plasma can make use of it.
This change hopefully improves the multi-screen experience for our Plasma users who use KWin as a window manager.
Creating a Photo-Box with the help of KWin
For a family celebration I wanted to create a “Photo-Box” or “Selfie-Box”: a place where the guests can trigger a photo of themselves without having to use tools like a selfie-stick.
The requirements for the setup were:
- Trigger should be remote controlled
- The remote control should not be visible or at max hardly visible
- The guests should see themselves before taking the photo
- All already taken photos should be presented in a slide show to the guests
The camera in question supported some but not all of the requirements. Especially the last two were tricky. While it supported showing a slide show of all taken photos, the slide show ended as soon as a new photo was taken. But the camera also has an usb-connector so the whole power of a computer could be taken in.
A short investigation showed that gphoto2 could be the solution. It allows to completely remote control the camera and download photos. With that all requirements can be fulfilled. But by using a computer a new requirement got added: the screen should be locked.
This requirement created a challenge. As the maintainer of Plasma’s lock screen infrastructure I know what it is good at and that is blocking input and preventing other applications to be visible. Thus we cannot just use e.g. Digikam with gphoto2 integration to take the photo – the lock screen would prevent the Digikam window to be visible. Also there is no way to have a slide show in the lock screen.
Which means all requirements must be fulfilled through the lock screen infrastructure. A result of that is that I spent some time on adding wallpaper plugin support to the lock screen. This allowed to reuse Plasma’s wallpaper plugins and thus also the slide show plugin. One problem solved and all Plasma users can benefit from it.
But how to trigger gphoto2? The idea I came up with is using KWin/Wayland. KWin has full control over the input stack (even before the lock screen intercepts) and also knows which input devices it is interacting with. As a remote control I decided to use a Logitech Presenter and accept any clicked button on that device as the trigger. The code looks like the following:
class PhotoBoxFilter : public InputEventFilter {
public:
PhotoBoxFilter()
: InputEventFilter()
, m_tetheredProcess(new Process)
{
m_tetheredProcess->setProcessChannelMode(QProcess::ForwardedErrorChannel);
m_tetheredProcess->setWorkingDirectory(QStringLiteral("/path/to/storage"));
m_tetheredProcess->setArguments(QStringList{
QStringLiteral("--set-config"),
QStringLiteral("output=TFT"),
QStringLiteral("--capture-image-and-download"),
QStringLiteral("--filename"),
QStringLiteral("%Y%m%d-%H%M%S.jpg"),
QStringLiteral("-I"),
QStringLiteral("-1"),
QStringLiteral("--reset-interval")
});
m_tetheredProcess->setProgram(QStringLiteral("gphoto2"));
}
virtual ~PhotoBoxFilter() {
m_tetheredProcess->terminate();
m_tetheredProcess->waitForFinished();
}
bool keyEvent(QKeyEvent *event) override {
if (!waylandServer()->isScreenLocked()) {
return false;
}
auto key = static_cast(event);
if (key->device() && key->device()->vendor() == 1133u && key->device()->product() == 50453u) {
if (event->type() == QEvent::KeyRelease) {
if (m_tetheredProcess->state() != QProcess::Running) {
m_tetheredProcess->start();
m_tetheredProcess->waitForStarted(5000);
} else {
::kill(m_tetheredProcess->pid(), SIGUSR1);
}
}
return true;
}
return false;
}
private:
QScopedPointer m_tetheredProcess;
};
And in addition the method InputRedirection::setupInputFilters needed an adjustment to install this new InputFilter just before installing the LockScreenFilter.
The final setup:
- Camera on tripod
- Connected to an external screen showing the live capture
- Connected to a notebook through USB
- Notebook connected to an external TV
- Notebook locked and lock screen configured to show slide show of photos
- Logitech Presenter used as remote control
The last detail which needed adjustments was on the lock screen theme. The text input is destroying the experience of the slide show. Thus a small hack to the QML code was needed to hide it and reveal again after pointer motion.
What I want to show with this blog post is one of the advantage of open source software: you can adjust the software to your needs and turn it into something completely different which fits your needs.
Modifier only shortcuts available in Plasma 5.8
Today I’m happy to announce that for the first time we were able to backport a new feature from our Wayland offerings to X11: modifier only shortcut support. This is one of the most often requested features in Plasma and I’m very happy that we finally have an implementation for it.
Modifier only shortcuts mean that an action is triggered if one clicks a modifier key without any other key. By default the Meta (also known as super and windows) key triggers the main application launcher of your Plasma session. But the implementation is way more flexible and allows to use any modifier (ctrl, alt or shift) as a trigger for an action (currently configuration is only possible through manual modification of kwinrc).
The feature was initially implemented for Wayland inside KWin and has been available for a year already. The tricky part is to recognize when only a modifier is pressed. On Wayland KWin gets all key events and passes them through xkbcommon. Given that KWin knows when a key is pressed and knows when only the modifier is pressed. Thus it was possible to implement modifier only shortcuts.
On X11 our global shortcut system does not have such a deep knowledge about all key states. It only registers to the shortcuts one has configured and in general it triggers on the press event, for modifier only we need trigger on the release event. As our X11 shortcut system relies on key grabbing we couldn’t use it for modifier only support: the chances to break applications are too high.
The implementation we have now on X11 reuses the infrastructure setup for Wayland. It uses XInput 2 to listen to all raw key events (which are also delivered if an application grabs the device) and sends those events through our xkbcommon infrastructure just like on Wayland to recognize the pressed keys.
In the past my main concern was that we mis-detect the state and that we trigger the shortcut when that is not wanted, for the wrong key, etc. etc. Also with the new implementation I have these concerns, although I’m very confident that it works correctly. Nevertheless I want to ask you to give a try to this feature and report if you notice any problems. I’m especially interested in setups which change the meta-modifier position through xmodmap and layout changes. If anything is not working as expected please tell us so that we can have an awesome modifier only experience in Plasma 5.8.
In case you use KSuperKey I recommend to disable that. Otherwise it might happen that shortcuts are triggered twice. At that point I want to thank KSuperKey for having filled this gap in our default feature set and providing a solution for our users who wanted to use this feature.
OpenGL changes in KWin compositing
In Plasma 5.8 we will see a few changes in the OpenGL compositor which are worth a blog post.
Debug information
The new KWin debug console gained a new tab for OpenGL information. This shows information comparable to what we know from glxinfo/es2_info. It shows the information gathered from the OpenGL library (version, renderer, etc.) and all the available extensions. This is intended to be useful when for whatever reason glxinfo is not working and one needs to know exactly which extensions are available.
As you can see in the screenshot: KWin also learned to add decorations around “internal” windows on Wayland. Which means the debug console can now be freely resized and moved around.
Support for llvmpipe
With Plasma 5.8 we finally allow OpenGL compositing through the llvmpipe driver. So far KWin fall back to XRender based compositing when the OpenGL driver uses software emulation. We thought that with XRender based compositing one gets better results than with software emulation. In this release we re-evaluated the situation and came to the conclusion that it doesn’t make sense to continue blocking llvmpipe driver. Due to QtQuick KWin will render parts of its user interface through llvmpipe anyway and also Plasma will render completely through llvmpipe. Thus most of the system is going through that driver. If the system is usable enough for Plasma, it will also be sufficient for KWin’s compositing.
Related to that many of the reasons to not use llvmpipe by default are going away. For example we didn’t want virtual machines to render through llvmpipe, but today also KVM can do accelerated OpenGL through virgl driver. Similar the embedded systems also provide working drivers nowadays, if we think of raspberry pi or odroid – they all can do OpenGL and we don’t have the risk of them going on llvmpipe which would result in a very bad experience.
Last but not least there is the question whether XRender or QPainter compositing is a better solution than llvmpipe. And there I’m not so sure anymore. For XRender it’s possible that the xorg modesetting driver is used in combination with glamor. In that case it would also use llvmpipe. So nothing gained by forcing to XRender.
But still it’s possible that using llvmpipe compositing will result in high CPU usage if KWin has to use it. The solution to this problem is to deactivate what we know to be expensive – all these effects which are not available on XRender because we know it to be too expensive. We don’t want blur effect on software emulation and neither wobbly windows. So we are now working on disabling the effects if we are on software emulation. For this we introduced a way to disable all animations in KWin and a lot of effects are already adjusted. This might also be a handy new features for users who don’t want any animations at all, but still want the compositor enabled.
Last but not least using llvmpipe also exposes performance problems which we normally don’t see with a fast GPU. And as a result we were already able to improve various parts of our rendering stack which will benefit all users – independently of whether llvmpipe is used or not.
Removal of GLX/EGL selection
A change which also got backported to a 5.7 bugfix release is the removal of the selection of GLX or EGL in the compositor settings. Unfortunately EGL is still not a good enough option on X11. Way too often we saw bug reports about rendering being broken and it was caused by using EGL. We decided that it doesn’t make sense to expose a config option which in most cases will result in the users systems being broken. So on X11 the default will stay GLX, while on Wayland the only option is EGL. Of course on X11 the option can still be set manually be editing the config file or by specifying the environment variable to pick EGL or to pick OpenGL ES 2.
Removal of unredirect fullscreen windows
Unredirection of fullscreen windows has been a kind of blue-headed step child in KWin’s compositing infrastructure for a long time. It’s a feature not loved by the developers, not properly integrated, but you have to support it. For those not knowing the feature: it excludes an area from compositing and let’s the fullscreen window be rendered the normal way in X11 (unredirect). The idea is that you get slightly better performance if you bypass the compositor.
The functionality was never fully integrated into the compositor. It was way too easy to break out of the condition (e.g. a tooltip), but at the same time effects which should break it, had no way to do it (e.g. Present Windows should either not activate or end it). The weirdest oddity of the feature is that we had to hard disable it for all Intel drivers due to crashes. We don’t know whether this is still the case but after having had such a bad experience with it in the past, we decided to never turn it on again. Which means it’s a feature not even supported by all devices.
We developers did not spent much time on the feature as we think it doesn’t make much sense as KWin has a better infrastructure in place: blocking compositing. Applications are allowed to specify that compositing should be blocked. This results in KWin shutting down the compositor, freeing all resource related to it (e.g. destroying the OpenGL context), so all power to the running game. As the compositor is shutting down, you don’t have weird interactions like tooltips jumping out or effects not working properly.
There is a standardized way for applications to request this and we see that many games and applications (e.g. Kodi) make use of it. This is the preferred way in our opinion. Given that this mode is fully supported, we decided to remove unredirect fullscreen windows from KWin’s compositor. This streamlines our implementation and gives us one feature to concentrate on and make sure that it works exactly as our users need it. On Wayland the architecture looks different: there is no such thing like unredirect fullscreen, but we can ideally just pass the buffer to the DRM device. The idea is that we do the best optimized way whenever possible.
Support for render devices
Our virtual rendering platform in KWin/Wayland gained a new feature. Like the normal DRM device it now uses udev to find all possible devices and will default to a render device (e.g. /dev/dri/renderD128) if present. If not present it will look for a card device created through the vgem driver. This is a change intended mostly for the use on build.kde.org so that we can properly test OpenGL compositing, but is also a handy feature for running KWin in the cloud on systems with “real” GPUs. If no GPU is present one can easily make KWin pick the virtual device.
Support for restarting the compositor on Wayland
Initially KWin did not support restarting the compositor or even switching the backend on Wayland. But of course we need to support that as well. The OpenGL driver might do a graphics reset after which our infrastructure wants to reinit the complete setup. Thus KWin needed to learn how to restart the OpenGL compositor on Wayland, which will be fully supported in Plasma 5.8.