Last week I concentrated most of my development work on screenshot support through spectacle in a KWin Wayland session. Now I am happy to announce that we merged support for capturing a screenshot of a window with the help of an external application like spectacle.
To explain why this is a great achievement we first need to look at X11. On X11 taking a screenshot of a window is easy. It’s part of the X protocol to read the pixmap data of the root window and you get the position and size of each window. Thus one is able to cut out the window and have it as a screenshot. That’s the most simple variant to do it, spectacle and previously ksnapshot do it differently. More on that later on.
This is the way to screenshot the active window. If one wants to screenshot any window the user needs to select it. Also for that the X protocol contains everything one needs: grab the mouse cursor, get the click and query the x window tree to figure out which window got clicked. Afterwards screenshot it the same way as explained.
For bringing that to Wayland we see some “problems”. Wayland is designed in a sane and secure way matching the security requirements of 2016 and not the security requirements of the 1980s. An application taking a screenshot of another window or of the complete system is nowadays not acceptable any more. And there is no built in way to take a screenshot – neither of fullscreen nor of a window.
And even if there were, it wouldn’t help much. Information about other windows are not available. One cannot get the information of which is the active window or what is the window under the current mouse position. Also an application is not able to grab the mouse and get the click anywhere on the screen. Mouse is always only available on the current window.
So overall quite some obstacles to take a screenshot and we see that it will need support from the Wayland compositor:
- Selection of the window
- Taking the screenshot
- Communication protocol with the application
Luckily KWin is partially already prepared for it. Even on X11 KWin provides a screenshot functionally to spectacle. A few years ago we wanted to have something better than the standard X screenshots. We wanted to have a window completely taken without overlapping windows and the decoration with the shadows included. Shadows are in case of KWin rendered by the compositor and not part of the X windows. So in order to screenshot it needed support from the compositor, just like on Wayland.
Unfortunately we didn’t think of something like “a successor for X11” back then and designed the interaction more in a way suited for KSnapshot than for not X. We used a DBus API which passed the window id of the window to screenshot as argument and as a result sent an XPixmap to ksnapshot.
Overall not suited for Wayland, but a very good starting point as we already have the screenshot functionality available. So what we needed is a an X free DBus protocol and a way to select the window from within KWin.
Just like spectacle also KWin has a functionality to select a window through mouse interaction: the kill window functionality triggered through Ctrl+Alt+Escape. So far this functionality was only available for X11 and X11 windows, we were not able to do the same on Wayland.
For taking the screenshot I wondered whether we could use this functionality in a more generic way: a feature to interactively select a window. This required a slight refactoring. Of course the X11 way to select a window doesn’t help much, but the ideas we have there. The X11 specific code got moved into the X11-standalone Platform plugin and is now invoked through the internal Platform API. It doesn’t directly kill the window any more, but only returns the window which got selected.
A similar interaction code got added for Wayland and now the kill window functionality can be triggered on both X11 and Wayland and can kill both X11 and Wayland windows.
Now all we needed was making this functionality aware to the screenshot functionality. And with that we could trigger an interactive way to trigger a screenshot. This keeps the user in control of the process: the user is informed that a screenshot is being taken and informed how to cancel this. This addresses the security concerns we had for taking screenshots. By making the user perform an explicit action we know that the user agreed to taking the screenshot.
Now all that was needed is adjusting spectacle. Spectacle is a rather new and modern application which had multiple windowing systems in mind when the implementation started. So far I had not done any work on spectacle and the code base was new to me. Nevertheless in about an hour I had the screenshot selection working:
Not everything is supported though. Fullscreen or screen area screenshots are not yet supported. But given that the primary problem is solved now, this will also be addressed soonish.
If you like the work we are doing for Wayland consider participating in our end of year fundraiser.
I can not take a screenshot of the widget open programs in X11 session. Is this a possibility in new version with wayland session?
Sorry I don’t understand your question.
I guess he means the start menu.
AFAIK this is only possible using the “delay” option.
But even with delay and “window under cursor” option, screenshot is somehow bugged:
http://i.imgur.com/EmwfuCe.png (this is original shot by spectacle, no editing)
Also what about screen recording?
Screen recording needs to be done in a very different way – completely unrelated to screen shot
Nice post – as I understand, KWin is then aware of being in a screenshot mode. Maybe it would be a nice addition to then allow moving the mouse cursor with the keyboard arrows pixel-wise. This is a very useful feature in the `greenshot` app that is used widely on Windows, since it allows to fine tune the selection rectangle.
That’s implemented 😉
Thumbs up!
Thanks for your hard work! 🙂 Donation sent 🙂
Is there some sort of list what is missing for distros to switch to wayland by default? Like it just happened for Fedora 25 with Gnome Shell. I couldn’t find anything on phabricator.
Do you know how far the plasma wayland part is?
We have: https://phabricator.kde.org/project/view/99/
In general for an end-user distribution I don’t think QtWayland is up to the task yet. For neon dev-unstable we want to switch, but that needs for example Xwayland 1.19 in neon (which is in Fedora for example).
Please make a standard protocol for this so that we get interoperability between Wayland compositors and clients.
As always, very interesting blog post. I think the VDG should have a look at the visuals and the message. The current grey box in the middle of the screen is not up to the current high visual standards of Plasma IMHO. Also, as KWin is in charge of the window selection, I think a screen darkening effect, with only the window below the cursor being normal, would bring a visual feedback and improve the user experience. But this is polishing, you made a great job !
Reading your blog posts, I understood that menus where considered separate windows than their parent windo. With this is implementation, is it possible to take a screenshot of an application with a menu item opened (like for a tutorial, showing Settings > Item 4 > Item 2) or would you need to use the future fullscreen feature ?
I just tried it: you need the fullscreen feature. The problem is that KWin sends a pointer leave event as soon as the interaction starts and at least Qt closes menus then.
Any opinion on Gnome supporting EGLStreams in Fedora 25? How about a new blog post about that topic to make KDE’s position clear?
Please see: https://blog.martin-graesslin.com/blog/2016/09/to-eglstream-or-not/
Would Plasma with Mutter instead of KWin work under Wayland? It works fine under X11.
No, it does not work fine on X11. You don’t get blur, you don’t get sliding popups and many more KWin specific features are missing.
Hi Martin, your blog posts are as always very interesting. I got curious after this statement:
“Information about other windows are not available. One cannot get the information of which is the active window or what is the window under the current mouse position. Also an application is not able to grab the mouse and get the click anywhere on the screen. Mouse is always only available on the current window.”
I understand this is needed to enforce security. I was wondering if, instead of embedding part of the screenshot functionality in kwin to get around security restrictions, it was ever considered at some point to instead have some kind of “sudo” operation that, after explicit action, grants the application access to all other applications windows and mouse information. Could this solve the screen grabbing problem and potentially support any other possible use case (admittedly from “weird” applications) without having to work on it explicitely on kwin?
I was wondering if this was ever debated in wayland or kde world.
no. that wouldn’t help. How would root know that the compositor transformed the screen?
Can’t any malicious application fake that user interaction (confirmation), and thus “fool” KWin into providing screenshot information?
No.
Thanks for your great work, Martin.
And dont forget screen resolution change support please.
Any plans for screen recordig (video)?
Screen recording is a very different problem to screen shot, so it’s completely unrelated to this blog post.
Screen recording is asked often because so many of us need it very often (in addition to screen capture). We cannot really move to Wayland as long as it is not solved (would be a big hindrance).
Currently, I am using Kazam on Plasma 5 / X. We need something at least as good/flexible on Wayland for me to consider migrating my work machine.
It would also be very nice to have standard Wayland protocols for both screen recording and screen capturing so that current applications like Kazam can be migrated to Wayland and they work on every Wayland compositor.
It’s like I always say: don’t switch to Wayland if your use case is not implemented yet. I don’t need screen capture in my workflows, so I can switch. If you need it: stay with X.
Rather the question is, would screen recording require involvement of the compositor? Since security would prevent applications to access screen data related to other processes.
You just answered it yourself. If only the compositer has access to the screen content it obviously needs support from the compositor.
So anyone writing a screen recording application for Wayland would need to implement various backends to work with different compositors, or Wayland somehow will provide a standard way to do it to avoid reinventing the wheel?
I’m not familiar with the state of matters in this, just wondering where things are likely to progress.
I assume a standard will evolve
What’s your thought of LibWSM?
It’s not provided by any distribution yet, thus it’s not yet up for “thoughts”. I’m very pragmatic on such things, if I cannot use it, it’s nothing I really look into.
Also for screenshot it wouldn’t help. The screenshot functionality uses DBus, not Wayland.
KSnapshot remembered the selection rectangle between successive screenshots during the same session (when using the Rectangular Region mode).
This made it easy to take multiple screenshots of the same screen region, and I used that a lot.
Will this feature return in spectacle?
I’m sorry, but I’m not the spectacle maintainer. Given that your comment is on the wrong place here.
Hello,
Thank you for taking the time to polish KDE.
About screenshot, could it be possible to use the clipboard instead of spectacle?
For instance, when selecting in a window, you can now copy the selection in the clipboard.
You could as well add the option to “take a screenshot of the window” or “copy the pixels of the selection” or “copy the colour of the selection”. These actions would move the corresponding objects to the clipboard where they could be grabbed by another program. This would require some additions to the clipboard protocol.
As this is an action by the user, the security would be the same than with the normal copy/paste paradigm.
Could this be easily implemented?
We used to have this functionality directly in KWin, it wasn’t used by our users. In fact it had been broken for years without being reported.
Given that experience says: it’s not needed.
I think the prettiest way to secure scrinshotmaking is to notify user “was made a screenshot” that’s it. It is enough
Maybe more secure – writing down the program/proces that was taking a screenshot.
Imagine if “wrong” program will make screenshot 3 times per second.. User can just reboot a PC, nothing more
Sorry for my not native english