Unit Testing a Window Manager

11 Replies to “Unit Testing a Window Manager”

1. Considering the fact that this mostly revolves around X.org will the things change with Wayland?

   1. we don’t need the interaction with the X-Server any more, but I don’t expect to see an X free KWin any time soon. But of course for the Bachelor Thesis we discussed how to ensure that the tests will also work with Wayland.

2. I had a similar problem while writing tests for Nepomuk. We required a separate instance of the database to be running and a separate dbus session. You case is however a lot more complex.

   You can look over my code if you want. It’s fairly simplisitc, but it helped me find a couple of bugs – http://vhanda.in/blog/2012/03/nepomuk-test-framework/

   Right now, it starts a separate dbus session, kde session and initializes the virtuoso db for first run. It then starts with the tests.

3. Isn’t what you describe a system test?

   1. According to Wikipedia System Testing is blackbox testing, while what I describe would still be whitebox testing.

4. We’re actually doing something similar (but rather simple) in Gentoo, we allow the test phase to start up its own framebuffer x server. See http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/eclass/virtualx.eclass?view=markup …

   1. that’s something to look at. Thanks for the link

5. I don’t think an X-server is really necessary for you.
   You can write a simple dummy library with the same interface provided by X11, and load that library instead of X11 when doing the unit test.

   1. If you give me that library, then yes. But I doubt anyone wants to write a mock to X11

6. Have you considered what is done to test the XMonad window manager. It is way lighter than KWin, hence easier to test. However, the fact that Haskell separates clearly side effecty code and pure code makes it easy to extract the ‘logics’ of the window manager and test it using their regular testing framework (QuickCheck).

   TLDR XMonad’s programmers separate the logic from the interface with X (which is tiny) and test the latter in a regular way with QuickCheck.

7. We tried having a full mock X Server when we were looking at how to test compiz. The idea was that we need something that behaves like an X Server to get an accurate idea of how compiz is behaving. So lp:xig was born, which is an emulator of the X protocol with an internal model of what X’s state should look like. Xig allowed us to verify that certain requests were made in certain operations and that certain events were posted back.

   What we found is that 90% of our time was spent fixing problems in Xig, to do with how it communicated certain requests, especially the tricky ones like property notifications and property updates.

   Integration testing is important, but for now I’ve decided a higher priority is to get things under a sensible unit test story. A large fallcy which I’ve been trying to “correct” in Canonical is that if you have some external system dependency X, then you can’t test without X, because you don’t know what the behaviour looks like.

   In reality, I’ve found that this is not true. If you have good code design and good mocking frameworks like Google Mock, it is easy to get almost anything under test. Recently I refactored our pixmap update code, which was dependent on XComposite to get it 100% under test. (have a look: https://code.launchpad.net/~compiz-team/compiz/compiz.fix_1002602_tests/+merge/108448). If you code to interfaces and use dependency injection sensibly, you can get the trickiest of interwoven complex dependencies under test.

   Lets say for example that you have some code that’s dependent on XGetWindowAttributes and XConfigureWindow. The first thing I’d always do is hide those two functions behind an interface, for example:

   class X11WindowInterface
   {
   public:
   virtual ~X11WindowInterface () {}

   virtual ConfigureWindow (const XWindowChanges &, unsigned int mask) = 0;
   virtual GetAttributes (XWindowAttributes &attrib) = 0;
   };

   Now I make my base window class implement this interface

   class MyBaseWindow :
   public X11WindowInterface
   {
   …
   };

   (lets say for example, that MyBaseWindow has actual references to the window id and Display *).

   Now lets see some function in MyBaseWindow that I want to get under test:

   MyBaseWindow::doSomethingWithWindow ()
   {
   XWindowAttributes attrib;

   if (XGetWindowAttributes (dpy (), win (), &attrib))
   {
   if (attrib.blah)
   bar ();
   if (attrib.foo)
   baz ();
   …

   XConfigureWindow (dpy (), win (), &xwc, mask);
   }
   }

   That code is pretty terrible for dependencies, lets refactor a bit:

   void doSomething (Display *dpy, Window win)
   {
   XWindowAttributes attrib;

   if (XGetWindowAttributes (dpy, win, &attrib))
   {
   if (attrib.blah)
   bar ();
   if (attrib.foo)
   baz ();
   …

   XConfigureWindow (dpy, win, &xwc, mask);
   }
   }

   Almost there – we still depent on bar () and baz (), so lets hide them:

   class BazzyBarCallInterface
   {
   public:

   virtual ~BazzyBarCallInterface () {}
   virtual bar () = 0;
   virtual baz () = 0;
   };

   And MyBaseWindow inherits that too, and forwards to the real foo () and baz () in our program.

   Now we can express the code like this:

   void doSomething (X11WindowInterface *x11, BazzyBarCallInterface *call)
   {
   XWindowAttributes attrib;

   if (x11->GetWindowAttributes (attrib))
   {
   if (attrib.blah)
   call->bar ();
   if (attrib.foo)
   call->baz ();
   …

   x11->ConfigureWindow (xwc, mask);
   }
   }

   Now our code only depends on some interfaces. Lets say we want to test it.

   Google Mock is amazing at this. Have a look:

   #include
   #include

   using ::testing::_;
   using ::testing::Return;
   using ::testing::Invoke;

   Ok, so we want to test that when XGetWindowAttributes returns true, and attrib.blah is set but attrib.foo is not set, that baz () is called and ConfigureWindow is called with 1, 2, 3, 4 and mask is 0

   So first we create “mock” classes which implement those interfaces:

   class MockX11Window :
   public X11WindowInterface
   {
   public:

   MOCK_METHOD1 (GetWindowAttributes, int (XWindowAttributes &));
   …
   };

   (ditto for MockBazzyBar)

   Now we create a test fixture.

   class DoSomethingTest :
   public ::testing::Test;
   {
   }

   We also need a fake X11Window which returns the attrib we want.

   FakeX11WindowAttribReturn :
   public X11WindowInterface
   {
   public:
   FakeX11WindowAttribReturn (XWindowAttributes &a) : mAttrib (a) {}

   bool GetWindowAttributes (XWindowAttributes &at)
   {
   at = mAttrib;
   return true;
   }

   prrivate:
   XWindowAttributes mAttrib;
   };

   TEST(DoSomethingTest, TestBazCalledOnAttribBlah)
   {
   XWindowAttributes attrib;
   MockX11Window mx;
   MockBazzyBar mb;

   attrib.foo = true;

   FakeX11WindowAttribReturn far (attrib);

   XWindowChanges xwc;

   xwc.x = 1;
   xwc.y = 2;
   xwc.width = 3;
   xwc.height = 4;

   unsigned int mask = CWX | CWY | CWWidth | CWHeight;

   EXPECT_CALL (mx, GetAttributes (_)).WillOnce (Invoke (&far, &FakeX11WindowAttributesReturn::GetWindowAttributes));
   EXPECT_CALL (mb, baz ());
   EXPECT_CALL (mx, ConfigureWindow (xwc, mask)); // Matchers

   doSomething (&mx, &mb);
   }

   Now your doSomething is under test!

   In terms of the argument to do with the fact that you expect certain events on certain requests, the reality is that the X Server is just a black box. You can test the event-handling and request-making code independently. After all, all you care about is the particular small unit you’re testing. So you can synthesize the events by making your event handler call certain functions in the actual “event reactor” classes. Same idea basically.

   Hope that helps 🙂 We have about 130 unit tests in compiz and that number is growing now thanks to this strategy.

Comments are closed.