On Benchmarks

A well knwon Linux website published a “benchmark” about Plasma Wayland vs Plasma Xorg vs Gnome Shell (Wayland and Xorg). Before anybody tries to draw any conclusion: this is not a proper benchmark. It shows no statistical relevance as it was only tested on one hardware and only on one distribution. It shows numbers, that’s it. The numbers might be nice or not, I don’t know. I am not able to draw any conclusions from these numbers.

This has been a general problem with those benchmarks for years. The setup is IMHO utterly stupid and not helpful for development. Rather the opposite as we have to spend time on explaining why the benchmark has a useless setup. There are two possible ways to address those benchmarks: ignore or volkswagen. On X11 we do to a certain degree volkswagen those benchmarks as we turn the compositor off if the games used in the benchmark request it. But we don’t do the full story yet, we don’t check for any of the benchmark applications and do not yet adjust our rendering to get better numbers. If anyone is interested: most of it should be easily achievable through KWin scripts. But yeah, if the numbers show better results for X11 than for Wayland it might be due to KWin cheating on X11.