More Software Compositing

One of the most often repeated misconceptions about Wayland is that it requires hardware acceleration. I would have thought that this issues would have been resolved once the reference compositor, Weston, supported rendering through Pixman. The reason for this misconception is most likely that the earlier versions of Weston required hardware acceleration.

Now KWin in 4.11 introduces the same mistake. Our experimental Wayland support is only available in the OpenGL compositor – even OpenGL ES is not supported (something is broken on my system I cannot start it). I think this is a bad situation. One of the huge advantages of KWin is the exchangeable compositor allowing to switch to software based XRender in case there are no proper drivers available. In fact KWin switches automatically to XRender if it detects a driver which recommends the XRender backend (e.g. software rasterizer).

So last week I started to work on some new Wayland related features and extended the XRender compositor in a way that it renders to a Wayland surface. It’s a pretty simple setup. Our XRender compositor uses an off-screen pixmap for rendering. The adjusted backend uses a pixmap created through XShm, so we get the frame rendered into a shared memory segment. From there we just copy it into a shared memory Wayland buffer to share it with the Wayland compositor. It would be better if we could create the shared Wayland buffer directly from the XShm segment, but I did not see a solution on how to achieve this.

In the end it’s also not really relevant. After all we want to use Wayland and the last thing we want to do on Wayland is using an X11 rendering API for compositing. So we can consider this more as a proof-of-concept and as a task so that I can get more familiar with Wayland ;-) It helped me to improve the code, so it was worth the effort. Even if it is a more academic solution it serves a concrete use-case: KWin uses XRender as a fallback and we can expect that the backend works. Given that we have to enforce compositing in future, it’s good to know that all backends work. The current state of this work can be found in branch kwin/wayland-backend of my clone (note: the link might change in case I force push).

Video on YouTube

If XRender is an academic solution, we are still facing the problem that we lost our software compositor. So I thought I need to also do something about this problem and started to work on a new compositing backend based on QPainter. It’s a quite simple compositor which uses a QImage as the rendering target. The QImage is created directly from the Wayland buffer, so in opposite to the XRender backend we don’t have the needless memory copy from one to another buffer. This is a very interesting compositor as it’s the first part of KWin which has been written from scratch only with Wayland in mind. It doesn’t support X11 (at the moment). This is the first new compositor written in years, so I found some things to improve and which are already under review. And as it was much easier to write a new compositing backend than I expected, I decided to document it in our wiki. It would be totally awesome if someone could have a look at it and write a dedicated compositor for the RaspberryPi ;-) The new compositor can be found in branch kwin/qpainter-scene.

The switch to Wayland does not only affect our software compositors, but compositing in general. So far KWin has always had the possibility to not require compositing. This doesn’t make any sense in a Wayland world. So I needed to teach KWin that we might need to enforce compositing. Of course on X11 we still want to have the option to disable compositing, so the changes only affect the Wayland backend. When starting KWin with the Wayland display environment variable set, KWin will require compositing. Even if it is disabled in the config options it will be enabled. Even more it ignores the shortcut to suspend compositing and options like unredirection of fullscreen windows. But still the backend might fail and in this case KWin is just exiting. There is no need to continue execution if one cannot see the output. This work can be found in branch kwin/enforce-compositing (which also contains all the other mentioned branches).

33 thoughts on “More Software Compositing

    1. the question cannot be answered. On Wayland there is no “no compositing”. How should I compare to it?

      1. How does it compare to the current X11-based backend?

        If a game or video player currently depends on the “suspend compositing” or “unredirect fullscreen window” feature to run fluently, how well will it run on KWin+Wayland?

        1. The motto of Wayland is “every frame perfect”. Think about it :-)

          And get away from comparing to the X world. It’s not X, so just because on X you needed suspend compositing just doesn’t mean anything for Wayland.

          1. It doesn’t mean anything for Wayland, so it also does not necessarily mean that *all* problems of X automagically go away in Wayland or that new problems cannot arise. If in X you have to suspend compositing for games or videos, then that means that there is a speed problem in those cases (which was not solved in X). This means that in Wayland either the problem persists (even if Wayland is implemented in a totally different way than X) or is solved. So the question was valid. To make my point more clear, consider the following example: MS Word documents are still not handled perfectly in LibreOffice (I recently had a major data-loss when editing a docx-document), Calligra Words has a different implementation than LibreOffice, but the question “is editing docx in Calligra as sucky as in LibreOffice?” is still valid.

            If the motto is “every frame perfect”, this means that Wayland will take all the time it needs to produce that perfect frame, so I conclude that speed is not guaranteed at all. Let’s hope there is some fallacy in my reasoning ;-)

            1. No, I think your example is plain wrong. You know that it is a different implementation so the negative view on the world doesn’t make sense. In fact you even know that LibreOffice and Calligra are so different that you are comparing apples to oranges. Yes, both allow to write text but that’s it. You wouldn’t compare a car to a motorbike either.

              So if we can leave the world of trying to compare things which cannot be compared behind us, that would be totally awesome. If you want to have some benchmarks or comparisons: just wait, I’m sure Phoronix will setup a benchmark. It won’t make any sense and will be completely stupid and wrong, but who cares? There are numbers one can put on bars!

              1. WTF? I cannot compare two software on the same feature because the implementation is different? It does make sense to ask whether will Wayland also from performance issues in situations where X + compositing did. Probably nobody knows the answer yet but it does not mean that the question is wrong.

                1. One can compare, but one has to properly compare. If you start from false assumptions (foo sucks, ergo bar must suck), your comparison will be wrong. If you end up comparing apple to oranges you have not achieved anything. If you want to compare apple to oranges, just wait for Phoronix to do the benchmarks.

                  1. I think the correct question would be if the compositor keeps doing the compositing operations, or if it just points to the app’s surface, when an app is fullscreen.

                    1. Well yes that is doable (a good idea in fact, I had not thought about this problem yet). It of course requires support for Wayland clients in KWin first ;-)

  1. Yes that’s what I meant, sorry.
    If the software rendering overhead in Wayland is small or non-existing compared to X11 with no compositing it would mean that even ancient hardware with no acceleration could run it fine leaving no more arguments to Wayland detractors.
    It would be also interesting to know on which hardware the screencast has been made by the way.

    As for the second part of your post, as far as I know the compositing in Wayland is so efficient compared to X11 that such hacks wouldn’t be needed in Wayland, and that fullscreen buffers would automatically be unredirected anyways.

    1. even X11 needs to render a window to the screen – whether compositing is used or not. The window doesn’t end on the screen by pure magic. To some degree also X can be considered as a “compositor”. So these comparisons with X in the non-composited case just doesn’t make any sense.

      1. As “X without compositing” also does the composition, probably in software, you can probably compare X’s compositor and your software compositor. And X’s might be faster, as it totally ignores any eye candy that you might want to support in your sw compositor… Still, it’s not guaranteed, as all the advantages of Wayland protocol over X11 might make it faster.

        And it’s probably just an academic exercise, as both types of composition can be qualified as “instant”.

        1. ok, the point is: I’m running Weston nested on X. So I cannot compare anything

      2. > even X11 needs to render a window to the screen – whether compositing is used or not.

        This is a bit misleading. If you disable compositing on X11, graphics is rendered directly to the screen’s memory by the driver. With compositing enabled, it is usually first rendered into an off-screen buffer, then copied to the screen’s memory. I said “usually”, because a very early compositing manager (I think it was just called xcmpmgr) was able to allow X11 clients to render directly to the screen for uncovered windows, too. Modern compositors avoid this complexity.

        1. Still X needs to do things like clipping etc. But I have to admit that I don’t know X rendering internals that well

  2. Thanks for all the hard work on KWin and for keeping the users updated!
    I have the following question: currently KDE is very good for gaming because one can disable compositing for that specific purpose at any time, or even set a rule that does this automatically, to maximize the resourses available to the game. And, indeed, there is a very noticeable difference in FPS in games between the cases of enabled and disabled compositing. Could you comment on this issue in relation to the plans to have the compositing non-disableable? I understand that this is a necessary step due to Wayland architecture, but how will this affect gaming performance?

    1. I do not believe in these benchmarks ;-) And the current state would hardly do well with games – only llvmpipe is working for glx.

      1. Yes i tested Xmir on Mir running Unity and i think llvmpipe is used yes. Performance is not that great that is true but Compiz never did work great on llvmpipe hence:

        “Personally i have yet to find full blown DE that will work decent when “software composting” is used..”

        ;)

        But it’s nice to be able to test it and i wouldn’t mind to do the same with Wayland + KDE on llvmpipe hence:

        “More Software Compositing”

        This is great indeed. :D

        1. Well KWin on Wayland is native – we don’t need llvmpipe, we get proper OpenGL if the OpenGL Compositor is used.

          1. Thank you for providing non-opengl option to kwin. Me and my laptops battery life give you many many thanks.

  3. One “request” could you write a blog post how you see GLX vs. EGL situation. Or if the isn’t much to say just answer it here in few sentences.

    Personal i am interested in two aspects:

    Will EGL make life easier for proprietary/FOSS drivers compared to GLX.
    Will EGL remove/reduce the need for constantly adding new support for the next Mir or Wayland based display server.

    1. I think that EGL is the better solution. For games it should not change much. Most should already support EGL if they have an OpenGL ES port (e.g. for Android, etc.). The APIs are fairly similar in most cases there are even exactly the same extensions. With GLX you really notice the age of X and the special cases it supports – like direct and indirect rendering.

      The second question I do not really understand.

        1. Yes I do know :-) The answer is NO! As a matter of fact KWin+Wayland requires EGL but does not work with OpenGL ES.

      1. About the second question yes it might not be related directly to EGL but i was wondering if this will change in any meaningful way in the future:

        Xorg breaks proprietary driver compatibility quite often and when we get new Xorg we have to wait to get AMD blob supporting it. Will Mir or some Wayland based display server solve this differently or it will this stay the same.

        Thanks.

        1. I read a post by a wayland dev in phoronix forums saying that Wayland API will not change, but that the ABI can, as X.

          An other post said that AMD was late to support new X ABIs because their schedules says that they release new features only after a long time to have the time to test.

          So it appears unlikely the problems we face with support of new X versions will not happen with Wayland.

          1. Yes i was thinking in this direction. Thanks for both answers.

            Interesting times ahead indeed!

  4. well from my understanding EGL is more agnostic since it doesnt depend on X or wayland, so following that logic wayland will only break blobs if there is a hard requirement of a future revision of EGL[lets say egl 2.1] and blobs only support the old one[lets say 1.3].

    but as far as blobs provide proper egl context wayland should not even care which version is on or not

Comments are closed.