Virtual Keyboard on Lock Screen

As part of our efforts to improve out of box experience for touch screens I’m pleased to announce that Plasma 5.10 will provide integration for virtual keyboard.

Plasma 5.10 finally integrates qtvirtualkeyboard directly in the lockscreen component of the look and feel package. Support for Qt’s virtual keyboard is not new in Plasma in general, it is already fully integrated into the Plasma Wayland session. This change, though, can be used on both Wayland and X11.

Greeter with new Virtual Keyboard button

The lockscreen got extended by a new button next to the keyboard layout switcher: “Virtual Keyboard”. When clicking this button the virtual keyboard gets enabled and one can use it (with both touch and mouse) to enter the password and unlock the screen.

Virtual Keyboard in the lock screen

As a nice side effect this finally enables the possibility to enter east Asian languages in the lock screen. So far we had to actively disable input methods on the lockscreen to prevent that the screen cannot be unlocked. As a side effect this meant that only latin characters can be used in the password. I hope that the new virtual keyboard can help here by providing a better experience.

As a note to our touch screen users. It’s possible to use Qt’s virtual keyboard for all Qt application. One just needs to specify the env variable QT_IM_MODULE=qtvirtualkeyboard. When entering text Qt pops up the virtual keyboard as an additional window. This is only needed in an X11 session. In a Wayland session KWin provides the integration and Qt apps (and other apps) get the virtual keyboard through the Wayland text input protocol.

Virtual keyboard support in KWin/Wayland 5.7

Over the last weeks I worked on improved input device support in KWin/Wayland and support for virtual keyboard. KWin 5.7 will integrate the new QtVirtualKeyboard module which is now available under GPLv3. For us this means that we have access to a high quality QML based keyboard. For Qt it means that the virtual keyboard is exposed to more users and thanks to the open source nature it means that we can upstream fixes.

The virtual keyboard is integrated into KWin/Wayland making it part of our platform. To support it we implemented the text input interface in KWayland. If an application supports it, then the virtual keyboard can input text from the compositor side.

Video on youtube:

As you can see in the video the virtual keyboard is only enabled by default if no hardware keyboard is available. If a hardware keyboard gets plugged in the virtual keyboard gets disabled automatically. In addition there is a Status Notifier Item registered which allows to enable/disable the virtual keyboard independently of the hardware keyboard state. So on a notebook with a touch screen it’s possible to get the virtual keyboard even if there is (of course) a hardware keyboard.

Implementing support for the virtual keyboard was an interesting journey which required many unexpected changes and improvements all over the place. First of all wl_text_input interface needed to be implemented as that’s what is used in QtWayland 5.6. At one point I hit a wall and saw that the implementation in QtWayland is incomplete. So I checked how it looks like in QtWayland 5.7 and alas the complete implementation changed, it’s using zwp_text_input_unstable_v2, which is not even in wayland-protocols. I’m not happy about QtWayland using not standardized interfaces as I think that’s very bad for compatibility. Nevertheless I also added support for it in KWayland as otherwise we would not be able to communicate with applications using Qt 5.7. As the QtVirtualKeyboard will only be available with Qt 5.7, we need to support Qt 5.7’s interface as well. In KWayland this is implemented in a transparent way and KWin announces support for both interfaces.

Another area which needed lots of work is the support for input devices in KWin. We need to properly detect whether a real keyboard is available. We needed to add further support for touch events in KWin. So thanks to the integration of virtual keyboard KWin now supports touch much better on Wayland.

The improvements for input devices are also useful in other areas. E.g. today I landed a change to not show the cursor image if no pointer device (mouse or touchpad) is connected.