Creating a Photo-Box with the help of KWin

For a family celebration I wanted to create a “Photo-Box” or “Selfie-Box”: a place where the guests can trigger a photo of themselves without having to use tools like a selfie-stick.

The requirements for the setup were:

  • Trigger should be remote controlled
  • The remote control should not be visible or at max hardly visible
  • The guests should see themselves before taking the photo
  • All already taken photos should be presented in a slide show to the guests

The camera in question supported some but not all of the requirements. Especially the last two were tricky. While it supported showing a slide show of all taken photos, the slide show ended as soon as a new photo was taken. But the camera also has an usb-connector so the whole power of a computer could be taken in.

A short investigation showed that gphoto2 could be the solution. It allows to completely remote control the camera and download photos. With that all requirements can be fulfilled. But by using a computer a new requirement got added: the screen should be locked.

This requirement created a challenge. As the maintainer of Plasma’s lock screen infrastructure I know what it is good at and that is blocking input and preventing other applications to be visible. Thus we cannot just use e.g. Digikam with gphoto2 integration to take the photo – the lock screen would prevent the Digikam window to be visible. Also there is no way to have a slide show in the lock screen.

Which means all requirements must be fulfilled through the lock screen infrastructure. A result of that is that I spent some time on adding wallpaper plugin support to the lock screen. This allowed to reuse Plasma’s wallpaper plugins and thus also the slide show plugin. One problem solved and all Plasma users can benefit from it.

But how to trigger gphoto2? The idea I came up with is using KWin/Wayland. KWin has full control over the input stack (even before the lock screen intercepts) and also knows which input devices it is interacting with. As a remote control I decided to use a Logitech Presenter and accept any clicked button on that device as the trigger. The code looks like the following:

class PhotoBoxFilter : public InputEventFilter {
public:
    PhotoBoxFilter()
        : InputEventFilter()
        , m_tetheredProcess(new Process)
    {
        m_tetheredProcess->setProcessChannelMode(QProcess::ForwardedErrorChannel);
        m_tetheredProcess->setWorkingDirectory(QStringLiteral("/path/to/storage"));
        m_tetheredProcess->setArguments(QStringList{
                                                   QStringLiteral("--set-config"),
                                                   QStringLiteral("output=TFT"),
                                                   QStringLiteral("--capture-image-and-download"),
                                                   QStringLiteral("--filename"),
                                                   QStringLiteral("%Y%m%d-%H%M%S.jpg"),
                                                   QStringLiteral("-I"),
                                                   QStringLiteral("-1"),
                                                   QStringLiteral("--reset-interval")
        });
        m_tetheredProcess->setProgram(QStringLiteral("gphoto2"));
    }
    virtual ~PhotoBoxFilter() {
        m_tetheredProcess->terminate();
        m_tetheredProcess->waitForFinished();
    }
    bool keyEvent(QKeyEvent *event) override {
        if (!waylandServer()->isScreenLocked()) {
            return false;
        }
        auto key = static_cast(event);
        if (key->device() && key->device()->vendor() == 1133u && key->device()->product() == 50453u) {
            if (event->type() == QEvent::KeyRelease) {
                if (m_tetheredProcess->state() != QProcess::Running) {
                    m_tetheredProcess->start();
                    m_tetheredProcess->waitForStarted(5000);
                } else {
                    ::kill(m_tetheredProcess->pid(), SIGUSR1);
                }
            }
            return true;
        }
        return false;
    }

private:
    QScopedPointer m_tetheredProcess;
};

And in addition the method InputRedirection::setupInputFilters needed an adjustment to install this new InputFilter just before installing the LockScreenFilter.

photobox

The final setup:

  • Camera on tripod
  • Connected to an external screen showing the live capture
  • Connected to a notebook through USB
  • Notebook connected to an external TV
  • Notebook locked and lock screen configured to show slide show of photos
  • Logitech Presenter used as remote control

The last detail which needed adjustments was on the lock screen theme. The text input is destroying the experience of the slide show. Thus a small hack to the QML code was needed to hide it and reveal again after pointer motion.

What I want to show with this blog post is one of the advantage of open source software: you can adjust the software to your needs and turn it into something completely different which fits your needs.

Modifier only shortcuts available in Plasma 5.8

Today I’m happy to announce that for the first time we were able to backport a new feature from our Wayland offerings to X11: modifier only shortcut support. This is one of the most often requested features in Plasma and I’m very happy that we finally have an implementation for it.

Modifier only shortcuts mean that an action is triggered if one clicks a modifier key without any other key. By default the Meta (also known as super and windows) key triggers the main application launcher of your Plasma session. But the implementation is way more flexible and allows to use any modifier (ctrl, alt or shift) as a trigger for an action (currently configuration is only possible through manual modification of kwinrc).

The feature was initially implemented for Wayland inside KWin and has been available for a year already. The tricky part is to recognize when only a modifier is pressed. On Wayland KWin gets all key events and passes them through xkbcommon. Given that KWin knows when a key is pressed and knows when only the modifier is pressed. Thus it was possible to implement modifier only shortcuts.

On X11 our global shortcut system does not have such a deep knowledge about all key states. It only registers to the shortcuts one has configured and in general it triggers on the press event, for modifier only we need trigger on the release event. As our X11 shortcut system relies on key grabbing we couldn’t use it for modifier only support: the chances to break applications are too high.

The implementation we have now on X11 reuses the infrastructure setup for Wayland. It uses XInput 2 to listen to all raw key events (which are also delivered if an application grabs the device) and sends those events through our xkbcommon infrastructure just like on Wayland to recognize the pressed keys.

In the past my main concern was that we mis-detect the state and that we trigger the shortcut when that is not wanted, for the wrong key, etc. etc. Also with the new implementation I have these concerns, although I’m very confident that it works correctly. Nevertheless I want to ask you to give a try to this feature and report if you notice any problems. I’m especially interested in setups which change the meta-modifier position through xmodmap and layout changes. If anything is not working as expected please tell us so that we can have an awesome modifier only experience in Plasma 5.8.

In case you use KSuperKey I recommend to disable that. Otherwise it might happen that shortcuts are triggered twice. At that point I want to thank KSuperKey for having filled this gap in our default feature set and providing a solution for our users who wanted to use this feature.

OpenGL changes in KWin compositing

In Plasma 5.8 we will see a few changes in the OpenGL compositor which are worth a blog post.

Debug information

The new KWin debug console gained a new tab for OpenGL information. This shows information comparable to what we know from glxinfo/es2_info. It shows the information gathered from the OpenGL library (version, renderer, etc.) and all the available extensions. This is intended to be useful when for whatever reason glxinfo is not working and one needs to know exactly which extensions are available.

OpenGL information in Debug Console

As you can see in the screenshot: KWin also learned to add decorations around “internal” windows on Wayland. Which means the debug console can now be freely resized and moved around.

Support for llvmpipe

With Plasma 5.8 we finally allow OpenGL compositing through the llvmpipe driver. So far KWin fall back to XRender based compositing when the OpenGL driver uses software emulation. We thought that with XRender based compositing one gets better results than with software emulation. In this release we re-evaluated the situation and came to the conclusion that it doesn’t make sense to continue blocking llvmpipe driver. Due to QtQuick KWin will render parts of its user interface through llvmpipe anyway and also Plasma will render completely through llvmpipe. Thus most of the system is going through that driver. If the system is usable enough for Plasma, it will also be sufficient for KWin’s compositing.

Related to that many of the reasons to not use llvmpipe by default are going away. For example we didn’t want virtual machines to render through llvmpipe, but today also KVM can do accelerated OpenGL through virgl driver. Similar the embedded systems also provide working drivers nowadays, if we think of raspberry pi or odroid – they all can do OpenGL and we don’t have the risk of them going on llvmpipe which would result in a very bad experience.

Last but not least there is the question whether XRender or QPainter compositing is a better solution than llvmpipe. And there I’m not so sure anymore. For XRender it’s possible that the xorg modesetting driver is used in combination with glamor. In that case it would also use llvmpipe. So nothing gained by forcing to XRender.

But still it’s possible that using llvmpipe compositing will result in high CPU usage if KWin has to use it. The solution to this problem is to deactivate what we know to be expensive – all these effects which are not available on XRender because we know it to be too expensive. We don’t want blur effect on software emulation and neither wobbly windows. So we are now working on disabling the effects if we are on software emulation. For this we introduced a way to disable all animations in KWin and a lot of effects are already adjusted. This might also be a handy new features for users who don’t want any animations at all, but still want the compositor enabled.

Last but not least using llvmpipe also exposes performance problems which we normally don’t see with a fast GPU. And as a result we were already able to improve various parts of our rendering stack which will benefit all users – independently of whether llvmpipe is used or not.

Removal of GLX/EGL selection

A change which also got backported to a 5.7 bugfix release is the removal of the selection of GLX or EGL in the compositor settings. Unfortunately EGL is still not a good enough option on X11. Way too often we saw bug reports about rendering being broken and it was caused by using EGL. We decided that it doesn’t make sense to expose a config option which in most cases will result in the users systems being broken. So on X11 the default will stay GLX, while on Wayland the only option is EGL. Of course on X11 the option can still be set manually be editing the config file or by specifying the environment variable to pick EGL or to pick OpenGL ES 2.

Removal of unredirect fullscreen windows

Unredirection of fullscreen windows has been a kind of blue-headed step child in KWin’s compositing infrastructure for a long time. It’s a feature not loved by the developers, not properly integrated, but you have to support it. For those not knowing the feature: it excludes an area from compositing and let’s the fullscreen window be rendered the normal way in X11 (unredirect). The idea is that you get slightly better performance if you bypass the compositor.

The functionality was never fully integrated into the compositor. It was way too easy to break out of the condition (e.g. a tooltip), but at the same time effects which should break it, had no way to do it (e.g. Present Windows should either not activate or end it). The weirdest oddity of the feature is that we had to hard disable it for all Intel drivers due to crashes. We don’t know whether this is still the case but after having had such a bad experience with it in the past, we decided to never turn it on again. Which means it’s a feature not even supported by all devices.

We developers did not spent much time on the feature as we think it doesn’t make much sense as KWin has a better infrastructure in place: blocking compositing. Applications are allowed to specify that compositing should be blocked. This results in KWin shutting down the compositor, freeing all resource related to it (e.g. destroying the OpenGL context), so all power to the running game. As the compositor is shutting down, you don’t have weird interactions like tooltips jumping out or effects not working properly.

There is a standardized way for applications to request this and we see that many games and applications (e.g. Kodi) make use of it. This is the preferred way in our opinion. Given that this mode is fully supported, we decided to remove unredirect fullscreen windows from KWin’s compositor. This streamlines our implementation and gives us one feature to concentrate on and make sure that it works exactly as our users need it. On Wayland the architecture looks different: there is no such thing like unredirect fullscreen, but we can ideally just pass the buffer to the DRM device. The idea is that we do the best optimized way whenever possible.

Support for render devices

Our virtual rendering platform in KWin/Wayland gained a new feature. Like the normal DRM device it now uses udev to find all possible devices and will default to a render device (e.g. /dev/dri/renderD128) if present. If not present it will look for a card device created through the vgem driver. This is a change intended mostly for the use on build.kde.org so that we can properly test OpenGL compositing, but is also a handy feature for running KWin in the cloud on systems with “real” GPUs. If no GPU is present one can easily make KWin pick the virtual device.

Support for restarting the compositor on Wayland

Initially KWin did not support restarting the compositor or even switching the backend on Wayland. But of course we need to support that as well. The OpenGL driver might do a graphics reset after which our infrastructure wants to reinit the complete setup. Thus KWin needed to learn how to restart the OpenGL compositor on Wayland, which will be fully supported in Plasma 5.8.

Why does kwin_wayland not start?

From time to time I get contacted because kwin_wayland or startplasmacompositor doesn’t work. With this blog post I want to show some of the most common problems and how to diagnose correctly what’s going wrong.

First test nested setup

If you want to try Wayland please always first try the nested setup. This is less complex and if things go wrong easier to diagnose than a maybe frozen tty. So start your normal X session and run a nested KWin:
export $(dbus-launch)
kwin_wayland --xwayland

This should create a black window. If it works you can send windows there, e.g.:
kwrite --platform wayland

When things go wrong

Xwayland missing

KWin terminates and you get the error message:

FATAL ERROR: failed to start Xwayland

This means you don’t have Xwayland installed. It’s a runtime dependency of KWin. Please get in contact with your distribution, they need to fix the packaging 😉

Application doesn’t start

KWin starts fine but the application doesn’t show because of an error message like:

This application failed to start because it could not find or load the Qt platform plugin “wayland”in “”.

Available platform plugins are: wayland-org.kde.kwin.qpa, eglfs, linuxfb, minimal, minimalegl, offscreen, xcb.

Reinstalling the application may fix this problem.

This means QtWayland is not installed. Please install it and try again.

Platform plugin missing

KWin terminates and you get the error message:

FATAL ERROR: could not find a backend

This means that the platform/backend plugin is not installed. KWin supports multiple platforms and distributions put them in multiple packages. For X11 you need e.g. on Debian based systems the package kwin-wayland-backend-x11. For the “real thing” you need: kwin-wayland-backend-drm.

XDG_RUNTIME_DIR not set

KWin terminates and you get the error message:

FATAL ERROR: could not create Wayland server

This means that KWin failed to create the Wayland server socket. The most common reason for this is that your environment does not contain the XDG_RUNTIME_DIR environment variable. This should be created and set by your login system. Please get in contact with your distribution. The debug output should also say something about XDG_RUNTIME_DIR.

Platform fails to load

KWin terminates and you get the error message:

FATAL ERROR: could not instantiate a backend

You hit the jackpot. Something somewhere went horribly wrong. Please activate all KWin related debug categories, run it again and report a bug against KWin – ideally for the platform you used. The debug output hopefully contains more information on why it failed. If not we need to try to look into your specific setup.

Trying on the tty

KWin works fine in the nested setup: awesome. In most cases this means that KWin will also work on the DRM device. Before trying: make sure that you don’t use the NVIDIA proprietary driver, if you do: sorry that’s not yet supported. If you are on Mesa drivers everything should be fine.

Log in to a tty and setup similar to nested setup – I recommend the exit-with-session command line option to have a nice defined setup to exit again:

export $(dbus-launch)
export QT_QPA_PLATFORM=wayland
kwin_wayland --xwayland --exit-with-session=kwrite

Ideally this should turn your screen black, put the mouse cursor into the center of the screen and after a short time show kwrite. You should be able to interact with it and when closing kwrite KWin should terminate. If that all works you are ready to run startplasmacompositor. It will work.

But what if not. This is the tricky situation. The most often problem I have heard is that KWin freezes at this point and gdb shows KWin is in the main event loop. This most likely means that KWin tries to interact with logind DBus API, but your system does not provide this DBus interface. Please get in contact with your distribution on how to get the logind DBus interface installed (this does neither require using systemd nor logind). I would like to handle this better, but I don’t have a system without logind to test. Patches welcome.

Running KWin on the weird systems

Yes one can go crazy and try running KWin on devices like the Nexus 5, Virtual Machines or NVIDIA based systems. Here the experience differs and I myself don’t know exactly what is supported on which hardware and in which setting combination. Best get in contact with us to check what works and if you are interested: please help in adding support for it.

Animating auto-hiding panels

With Plasma 5 a change regarding auto-hiding panels was introduced. The complete interaction was moved from Plasma to KWin. This was an idea which we had in mind for a long time. The main idea is to have only one process to reserve the interaction with the screen edges (which is needed to show the panel when hidden) and to prevent conflicts there. Also it allowed to have only one place for providing the hint that there is the panel.

On an implementation level that uses an x11-property protocol between KWin and Plasma to indicate when the panel should be hidden. KWin then does the interaction to hide it and to show on screen edge activation.

Unfortunately from a visual perspective this created a regression. In Plasma 4 the auto-hiding panel was animated with our Sliding-Popups effect, but in Plasma 5 this doesn’t work any more. The reason for that is that our effect system can only animate when a window gets mapped and unmapped. With the new protocol the window doesn’t get unmapped when hidden and not mapped when shown, so our effects are not able to react on it. Technically our Client object is kept instead of being released on unmapped.

Thanks to Wayland this will be working again starting with Plasma 5.8. For Wayland windows KWin keeps the object around when a window gets unmapped and uses the same object when it gets shown again. KWin keeps more state for Wayland windows than for X11 windows. But this also means that our animations are not working for Wayland windows. Last week I addressed this and extended the internal effects API to also support hiding/showing again of Wayland windows. As the effects API does not differentiate between Wayland and X11 windows this change also enables the auto-hiding panel animation. All that was needed was emitting two signals at the right place.

Here Wayland shows another strength: not only does it help to bring features to X11 it also allows us to automate the testing. With a pure X11 system we would have had a hard time to properly auto-test this. But for Wayland we have a nice isolated integration test-framework which allowed to add a test-case for auto-hiding panels.

Multi-screen woes in Plasma 5.7

With Plasma 5.7 we promised improved multi-screen support. While we achieved that, some users are still experiencing issues. This is unfortunate and our users have all the reasons to be disappointed with us. We are working very hard to fix the issues which have been reported to us since the release.

But there are many situations where users blame us for issues not under our control. With this blog post I want to describe some of the problems we got reported and explain them.

Display hardware key not working

In Bug 365147 a user reported that the notebook hardware key does not work. This is rather surprising as our code of course supports the key and it is working on the systems we developers have. We asked the user to run xev and report back what it prints when pressing the key. It turns out it sends Windows+P which is used since Windows 7. That is a rather unfortunate situation and I can only conclude that the hardware is broken. More information on this topic by Matthew Garrett. Although this problem had been described a few years back, we just had not been aware of it. We now fixed it by adding support for this shortcut. I’m not happy about that as it “steals” a shortcut, but that’s the best we can do in the case that a monopolist dictates a shortcut and hardware vendors follow.

Windows shown on incorrect screen

Matlab splash screen shown in non-visible area

One issue which got reported to me by a fellow KDE developer on Thursday was Matlab’s splash screen partially being positioned in an offscreen area in a multi-screen setup. To me as KWin maintainer and knowing how we position windows I knew that it is not our fault. Nevertheless we investigate.

First of all Matlab does not set the window type splash on it’s splash window. If it did KWin would position it on the center of the screen and also would support hiding it if one clicks on it. By setting the wrong window type KWin does not know that it’s a splash screen window and cannot apply the proper handling.

But there are more issues with that window. In addition the window contains a placement hint. This is described in the Inter-Client Communication Conventions Manual Section 4.1.2.3. In this specific case the window indicated that it got positioned by a user. To quote: “which allow a window manager to know that the user specifically asked where the window should be placed”. Of course the user does not specify the position of a splash screen outside the visual area. But Matlab tells us so, it’s clearly wrong, but KWin cannot know that. Instead it follows what the window requests: if the window indicates that it has a position, KWin honors that. The assumption is that the application knows what it does – apparently we are wrong here.

Skype notification window outside visible are

I also got a report about skype notifications being shown outside the visual area or in general on the wrong screen. This is something which we have even less control over. Let’s have a look at the xwininfo of a skype notification:

xwininfo: Window id: 0x1000129 "skype"

Absolute upper-left X: 1011
Absolute upper-left Y: 962
Relative upper-left X: 1011
Relative upper-left Y: 962
Width: 268
Height: 61
Depth: 24
Visual: 0x27
Visual Class: TrueColor
Border width: 0
Class: InputOutput
Colormap: 0x26 (installed)
Bit Gravity State: NorthWestGravity
Window Gravity State: NorthWestGravity
Backing Store State: NotUseful
Save Under State: yes
Map State: IsViewable
Override Redirect State: yes
Corners: +1011+962 -1921+962 -1921-57 +1011-57
-geometry 268x61+1011-57

The important part is “Override Redirect State: yes“. This means that the window actively disabled the window manager on it. Of course the application should be doing the right thing, but sometimes they aren’t. In the case of Skype there is of course nothing we can do about it as it’s a proprietary application.

Of course the best way to fix this from Skype site would be to use the freedesktop specification for notifications. That way they don’t need to care about positioning windows correctly. It would go to experts.

Comboboxes/Menus shown on wrong screen

A few users reported that the comboboxes and menus of applications show on the wrong screen.
Menus placed on wrong screen

This problem is similar to the Skype notification issue. The windows in question are override redirect windows. The window manager is not controlling them. It’s the task of the toolkit to handle this correctly. Unfortunately Qt seems to have problems with it. To make it quite clear: this is a bug the Qt developers have to fix. It’s affecting all Qt applications on X11 and has nothing to do with Plasma. Just because Plasma is also using Qt does not mean that Plasma developers have to fix it. This is a bug the Qt developers have to fix. It’s embarrassing, not just for us, but for all applications built upon Qt – be it free software or proprietary. A combobox or menu opening on the wrong screen is an absolute no go. In my opinion that’s a showstopper bug. Unfortunately in such cases KWin cannot help to fix it.

Newly opened windows not placed on the primary screen

Many users expect that new windows open on the primary screen. Unfortunately primary screen does not imply that, it’s only a hint for the desktop shell where to put it’s panels, but does not have any meaning for normal windows.

Of course windows should be placed on a proper location. If a window opens on a turned off external TV something is broken. And KWin wouldn’t do so. KWin places new windows on the “active screen”. The active screen is the one having the active window or the mouse cursor (depending on configuration setting). Unless, unless the window adds a positioning hint. Unfortunately it looks like windows started to position themselves to incorrect values and I started to think about ignoring these hints in future. If applications are not able to place themselves correctly, we might need to do something about it.

Of course KWin allows the user to override it. With windowing specific rules one can ignore the requested geometry.

Please report your bugs

We saw many users complaining on social networks about the state of multi-screen. Please report the issues to us. It’s really important that we get to know the setups which don’t work. Yes of course it’s not acceptable if a hardware key doesn’t work, but if we don’t know about it there is nothing we can do to fix it.

We are also a little bit disappointed that users started to complain after the 5.7 release instead of testing our beta. It’s not difficult to try a live image to see whether an issue is resolved and then reporting a bug. It might even be less work than ranting on social media.

Why is multi-screen so difficult?

As a last point I want to discuss why multi-screen is difficult. It’s not like we KDE devs don’t have multi-screen setups (typing on one right now, though on Wayland) and don’t test our code. At the openSUSE conference a part of my talk was about the difficulty to test multi-screen. I think it’s valuable to share this here as well.

Lack of XRandR in virtual X servers

On X11 one has the XRandR extension to configure multi-screen. In addition there is also the legacy Xinerama and even more legacy neither-nor. For XRandR there are multiple protocol versions with different versions being incompatible to each other. Nowadays we can luckily assume that users have version 1.4 of XRandR – this is already quite an improvement compared to the past.

Unfortunately virtual X servers such as Xvfb lack proper XRandR support. Until recently Xvfb did not have the XRandR extension at all. So with Xvfb it was not possible to setup a test environment to test multi-screen code. Nowadays that’s supported but it’s still rather useless. Let’s have a look at the output of xrandr:

xrandr: Failed to get size of gamma for output screen
Screen 0: minimum 1 x 1, current 1280 x 1024, maximum 1280 x 1024
screen connected 1280x1024+0+0 0mm x 0mm
1280x1024 0.00*

We see that we only have one resolution on the screen, which does not have a physical size. This makes it impossible to test things like DPI (would require physical size), changing of modes, etc. Also hotplugging/unplugging of additional screens cannot be simulated that way. The situation is similar for other solutions like Xephyr and even Xwayland:

Screen 0: minimum 320 x 200, current 3200 x 1080, maximum 8192 x 8192
XWAYLAND0 connected 1280x1024+0+0 340mm x 270mm
1280x1024 59.89*+
XWAYLAND1 connected 1920x1080+1280+0 520mm x 292mm
1920x1080 59.88*+

The Wayland display reports the following information for the same setup:

interface: 'wl_output', version: 2, name: 16
x: 1920, y: 0, scale: 1,
physical_width: 340 mm, physical_height: 270 mm,
make: 'GSM', model: 'L1953T/308042',
subpixel_orientation: unknown, output_transform: normal,
mode:
width: 1152 px, height: 864 px, refresh: 75.000 Hz
mode:
width: 1024 px, height: 768 px, refresh: 75.029 Hz
mode:
width: 1024 px, height: 768 px, refresh: 60.004 Hz
mode:
width: 832 px, height: 624 px, refresh: 74.551 Hz
mode:
width: 800 px, height: 600 px, refresh: 75.000 Hz
mode:
width: 800 px, height: 600 px, refresh: 60.317 Hz
mode:
width: 640 px, height: 480 px, refresh: 75.000 Hz
mode:
width: 640 px, height: 480 px, refresh: 59.940 Hz
mode:
width: 720 px, height: 400 px, refresh: 70.082 Hz
mode:
width: 1280 px, height: 1024 px, refresh: 60.020 Hz,
flags: current preferred
interface: 'wl_output', version: 2, name: 18
x: 0, y: 0, scale: 1,
physical_width: 520 mm, physical_height: 292 mm,
make: 'SAM', model: 'SyncMaster/1263088180',
subpixel_orientation: unknown, output_transform: normal,
mode:
width: 1920 px, height: 1080 px, refresh: 60.000 Hz
mode:
width: 1920 px, height: 1080 px, refresh: 50.000 Hz
mode:
width: 1600 px, height: 1200 px, refresh: 60.000 Hz
mode:
width: 1680 px, height: 1050 px, refresh: 59.883 Hz
mode:
width: 1280 px, height: 1024 px, refresh: 60.020 Hz
mode:
width: 1440 px, height: 900 px, refresh: 59.901 Hz
mode:
width: 1280 px, height: 960 px, refresh: 60.000 Hz
mode:
width: 1280 px, height: 800 px, refresh: 59.910 Hz
mode:
width: 1280 px, height: 720 px, refresh: 60.000 Hz
mode:
width: 1280 px, height: 720 px, refresh: 59.940 Hz
mode:
width: 1280 px, height: 720 px, refresh: 50.000 Hz
mode:
width: 1024 px, height: 768 px, refresh: 60.004 Hz
mode:
width: 800 px, height: 600 px, refresh: 60.317 Hz
mode:
width: 800 px, height: 600 px, refresh: 56.250 Hz
mode:
width: 720 px, height: 576 px, refresh: 50.000 Hz
mode:
width: 720 px, height: 480 px, refresh: 60.000 Hz
mode:
width: 720 px, height: 480 px, refresh: 59.940 Hz
mode:
width: 640 px, height: 480 px, refresh: 60.000 Hz
mode:
width: 640 px, height: 480 px, refresh: 59.940 Hz
mode:
width: 1920 px, height: 1080 px, refresh: 59.940 Hz,
flags: current

The broken Xorg Intel drivers

Without the possibility to properly perform automatic testing in a controlled setup one has to resolve to manual testing. And here things get tricky when the underlying stack is broken. I’m now going to share my experience of the last half year with the Intel Xorg driver for my Ivybridge system. Over the last half year I experienced random breakage whenever I updated the xorg-intel driver. I rebooted my system and something broke. The stack on top did not change – that is Qt and Plasma were the same as on the last (working) usage, but the driver updated. The issues I saw ranged from completely losing one of my outputs to outputs being removed from XRandr when going to DPMS powersaving, incorrect modes, etc. It is quite annoying. And from what users reported I saw that this is a common thing: random breakage in the xorg Intel driver. I have a second system – same distribution, same software stack with Intel Sandybridge instead of Intel Ivybridge and there I never hit those problems. The main issue here – in my opinion – is that Intel stopped releasing the Xorg driver. It hasn’t seen a release for more than two years. Of course distributions need to support something newer, so it happened what had to happen. Basically all distributions are now performing rolling releases from the master branch. Of course that means that broken and untested code gets directly delivered to the users – no blame to distros here. They don’t have much choice. The situation is unfortunate and we cannot really do something about it. Distros have to combine untested code released after our releases. If it breaks, it breaks.

Overall that means that testing is really difficult for us. If you have a problem with your setup it doesn’t mean that any dev has ever run into it. We need your bug reports with very detailed descriptions of your setup to be able to understand what went wrong in your specific case. Then we can fix it. Of course we would like to have this supported better, but without proper ways to test it, it’s tricky. Wayland allows us to test much better in that regard.

QScreen

A big problem for our software is the quality of Qt’s QScreen implementation on X11. This has been a problem since it got introduced. Basically the idea is that a QWindow is attached to a QScreen. On X11 a QScreen is mapped to an XRandR output, which means that a QWindow is mapped to an XRandR output. This is IMHO completely wrong, because an X11 window is positioned on the virtual X11 screen. In addition they changed the behavior and documentation of QWindow::screen to allow null values. After that applications started to crash all over the place. We got the following useful hint in the relevant Qt bug report: “Also, KDE developers should make sure not to 1) assume that QScreen* returned from any Qt function is guaranteed to be non-null 2) hold any QScreen pointers.”

This is a nice suggestion, if our code were crashing. We got crash reports for this constantly (over 50 duplicates just for KWin), just the crash has never been on our side of the world. It always crashed directly in Qt. So to conclude: “Also, Qt developers should make sure not to 1) assume that QScreen* returned from any Qt function is guaranteed to be non-null 2) hold any QScreen pointers.”

This situation continued for a few years with Qt trying to catch up and fix their incorrect assumptions. Unfortunately that is still broken in many distributions. I still constantly get crash reports from users on distributions with Qt 5.5. The situation only improved with Qt 5.6 when Qt finally got the idea to introduce a dummy QScreen when there are no outputs. So on X11 we are back to it’s never null. On Wayland the last time I tried Qt crashed and I concluded that I have to make sure from KWin side that there will always be an output.

With Qt 5.5.1 it also looks like most QScreen related crashers are resolved, but that caused other problems, like a lockscreen bypass. Created thanks to the change in behavior of screen handling between Qt 4 and Qt 5. Up until Qt 5.5.1 it never happened, because the lockscreen would crash and be restarted – that was a situation well handled. It’s an issue I never saw on my own system as I never had Qt 5.5.1.

With Qt 5.6 the situation improved thanks to the dummy QScreen. But that still causes weird issues because the QWindows change their position. Especially in combination with issues like I mentioned above that screens disappear in power saving.

The situation for us and our users is very bad. We are very unhappy with the quality of Qt with regard to QScreen – especially with the random crashers we saw in the past. Of course it doesn’t help to point fingers. We have bugs in our code as well, but it’s difficult to investigate them if your code crashes deep down in Qt when testing the changes. You don’t even see the problems. Thus it happens that we now see new problems appear. Given that the Qt issues are resolved now, we hope to be able to fix those issues in a timely manner. Thanks for understanding and thanks for reporting the issues.

Animations on lock screen – Plasma Wallpaper support

With Plasma 5.7 released I’m allowed to blog about new features in Plasma 5.8. One of the features missed by many users in the Plasma 5 series was the lack of animations in the lock screen architecture. With Plasma 5 we dropped support for the old XScreenSaver and went QtQuick only. Now technically it was always possible to have animations on the lock screen. Our lock screen architecture loads the QtQuick files through the lookandfeel package mechanism, which means that one could provide an animation in a lookandfeel package.

But so far, nobody did that. Which means we don’t have animations on the lock screen. Today I’m happy to announce that this situation is going to change with Plasma 5.8. Our lock screen architecture gained support for Plasma Wallpaper plugins. All the wallpaper plugins available for the desktop are now also available for the lock screen. By default it uses the image wallpaper set to the default wallpaper, but one can also have slide shows, or fancier animations. In fact one could even try to provide a Plasma Wallpaper plugin for the old XScreenSavers.

The support for wallpaper plugins got added as a scratch-your-own-itch solution. For a personal project I must be able to show a slide show on a locked screen and looked at possible solutions for this. Pretty quickly I came to the conclusion that adding Plasma wallpaper support is the best solution and also provides something for all Plasma users. But it also makes me wonder why nobody did that before.

Synchronizing the X11 and Wayland clipboard

One of the greatest annoyances after switching my work systems to Wayland is the lack of clipboard synchronization between the X11 and Wayland windowing system. X11 clipboard is only available to X11 windows and the Wayland clipboard is only available to Wayland windows. It’s the useability problem of the two X11 clipboard (primary and secondary) just exposed on a much higher and more annoying way. Instead of “just” remembering whether you copied or selected text, you need to remember which windowing system you copied from and which windowing system the current window is in. And if they are not in the same, there is just no way.

So getting this problem resolved got quite a priority. It was something I had never really spent any thought on and incorrectly assumed it’s something Xwayland would take care of. But after thinking about the problem I realized that the X11 clipboard is part of ICCCM and not a core feature of the X-Server. It’s a communication protocol between clients.

This means that KWin had to learn to communicate with the X11 clipboard in order to provide the synchronization. Not an easy task as ICCCM is quite complex on this topic and implementing an X11 clipboard is nothing I’m looking forward to. Luckily I remembered that we have a clipboard manager for X11 (klipper) and that this one works without having X11 specific code thanks to Qt’s abstraction.

And this makes the whole thing way easier: by using Qt’s X11 clipboard implementation we only need to interact with Wayland’s clipboard which is thanks to KWayland quite straight forward. The solution involves a small helper binary which gets started by KWin. It’s forced to the xcb windowing system to be able to use Qt’s X11 clipboard and manually connects with KWin through KWayland. The task of the binary is quite simple: whenever the X11 clipboard changes it gets the data and sets it on the Wayland clipboard. Similar whenever the Wayland clipboard changes it gets set on the X11 clipboard.

Unlike the X11 clipboard on Wayland clients are normally not informed about clipboard changes. Only the application currently having keyboard focus gets informed by the Wayland server about what kind of data is available on the clipboard. This means we don’t need to inform all Wayland clients about changes in the X11 clipboard and similar we don’t need to inform X11 about clipboard changes in Wayland all the time. Instead KWin helps the process: when an X11 window gains keyboard focus our helper process gets informed about the current Wayland clipboard content, so that it can be available for the focused X11 window and while X11 windows are focused clipboard changes from the helper application are forwarded to Wayland. So when the next Wayland window gets focused it has the clipboard content set on X11. A small and simple solution which works quite well in practice. At the moment only the “real” clipboard is synchronized. There is not yet the concept of the selection based clipboard on Wayland. A solution might be to just synchronize it to the normal clipboard. But this would again introduce the useability problems of this non-explicit clipboard. So I’m not yet convinced what’s the proper way.

In my last blog post I pointed to our current donation campaign to support the Randa developer sprint. A lot of readers donated after reading this. I want to thank everybody who donated. It’s so important for a free software community to meet in person and we need donations for getting developers to sprints and conferences. Last week I was at openSUSE conference and had a talk about how Wayland improves the quality of Plasma. These are important things to talk about at a conference, but it’s only possible if we have the donations to send people there. Please help to make such developer sprints possible. Thank you!

Wayland in Plasma 5.7

Last week we released the beta version of Plasma 5.7 which means we know what this release will have for better Wayland support. First of all I need to mention what didn’t make it: unfortunately I missed the freeze of Frameworks 5.23 to land support for xdg-shell. I have a working implementation, but was not yet satisfied with the API. This is a difficult interface to provide an API for due to the unstable nature of the interface. Due to lack of xdg-shell support GTK applications are still going to use X11 on Wayland (like the Firefox window I’m just typing this blog post in).

In the past I already blogged about a few new features in 5.7 like the improved task manager for Wayland, the virtual keyboard support, sub-surface support and improved input device support. So in this blog post I want to focus on a different topic: quality.

For Plasma 5.7 my aim was to get the Plasma session into a state that I can use it as my primary system. And since last week I have not started into an X11 session any more. This means that we needed to get the whole system stable enough to have neither KWin nor applications crash due to Wayland. Given that our Wayland code is quite a fair amount of new code, changing lots of assumptions there are of course bugs to be expected. We still have code which calls into X11 unconditionally, we have things which are not implemented correctly and of course we do stupid mistakes. So for Plasma 5.7 the task was to find these issues and fix one for one.

For Wayland it’s much easier for us to test. KWayland – our framework for Wayland support – is developed in a test driven approach making it possible to create test cases for every problem. They expose the problems and verify that they are fixed and as regression tests ensure that they won’t hit us again. Over the last release cycle we added several thousand lines of test code in KWayland alone.

Finding those issues is not always easy. If KWin crashes we don’t have DrKonqi like normally, it doesn’t work for Wayland (tries to connect to a display server, but that just crashed). What I saw on my Wayland test device was that KWin sometimes randomly crashed – more often when I interacted with X11 windows. But when attaching gdb to KWin it didn’t crash. But once I caught it: it turned out to be an error in KWin in the handling of Xwayland windows. There are two possible code paths it can take and one was with a mistake. Due to running through a debugger it was more likely to take the correct one. So yeah it’s not always easy.

With that problem gone we were able to find a few more and fix also some bugs which caused windows to quit. Unfortunately some of this fixes had to go into KWayland after the 5.23 release. This means the frameworks version used with Plasma 5.7 is not going to have all fixes. If you want to give Plasma/Wayland a try I recommend to not just wait for Plasma 5.7 but also for frameworks 5.24.

This week I will be at the openSUSE conference in Nuremberg, where I will also give a talk about how Wayland helps us to improve our quality and our workflows. I’ll do another blog post about the content of that presentation – don’t want to spoil 😉 Though if you follow our development you are already aware of it. Thanks to openSUSE to give me the possibility to present at the conference and thanks to KDE e.V. for the support to go there.

I can hear you asking now the question of all questions: “When will it be ready?” I think that I am not objective enough to answer the question or to say that it is ready. I’m too close to the code and might just omit important problems because I don’t see them. Thus I cannot say that it is ready. It depends on your workflow and whether that workflow is already fully implemented. This is something only you can know.

Last week KDE had a very important developer sprint (where I did not participate) and is currently running a fundraising campaign for this sprint. We need the money to send our developers to such meetings or to a conference like openSUSE conf where I will be this week. At the moment just 107 people have participated and donated. This is something which makes me sad. I see the statistics for my blog posts and know that this one will have at least 1000 direct hits. In addition there are people reading planetkde and not directly my blog. We are trying to raise 24000 EUR and please do the math yourself to see how close we would be to it if everybody would donate just 10 EUR.

Why planetkde needs to have political posts

Johnathan’s blog post about the EU triggered quite some noise about that there should not be political posts on planetkde. I think it’s important that we have political posts on planetkde. KDE is a political movement, let’s look at our vision:

A world in which everyone has control over their digital life and enjoys freedom and privacy.

This is a political statement. We want to change the world! We are political! We need to be able to discuss political topics like the Snowden-documents, we need to be able to lobby on planetkde for Free Software in our governments, we need to be able to report about such efforts.

This means politics belong to KDE and belong to planetkde.

This means we need to also tolerate political opinion posts on topics which are not directly KDE related, like Johnathan’s. If we start to remove such posts or start to censor us, we run the risk of not having the relevant posts on planetkde anymore because people don’t post political articles at all anymore.

Please remember that planetkde is for the KDE community by the KDE community. Any political opinion expressed on planetkde does not represent the opinion of KDE or KDE e.V.