This weekend I was looking a little bit at the bug statistics provided by Bugzilla. I was a slightly surprised by the fact that before 4.0 the numbers of open and fixed bugs hardly changed and started to skyrocket around the 4.0 release. During the last three years almost as many bugs have been fixed as in the eight years before. The number of duplicate reports even trippled, nevertheless the number of open bugs is constantly increasing. I will address the implications of this “bug problem” in a separate blog post.
As I joined the KWin development team around the 4.0 release I was interested in seeing how the commit rate around the time was. Did the development speed change, has there been a more severe impact to the codebase than at different times? Thanks to git I have the complete history around, so I did a useful formatting, wrote a small program to get the data I’m interested in and created diagrams using LibreOffice Calc[1].
As we can see around before 2008 KWin development clearly did not stagnate. Since 2006 the number of commits per year and developers who did at least one commit per year got a significant boost. This might be related to the porting to KDE 4 and the work on the compositor, but even after 2008 the commit rate was more or less the same. Till 2006 the avarage yearly commit rate was 360 commits, from 2007 till 2010 around 890 commits per year. Please don’t misinterpret the last data paint as a reduced activity. In fact the opposite is the case. At the end of 2010 I worked outside our SVN repository and the commits hit git in 2011 which explains that we already have half of the commits of 2010 in the (not yet) three month of 2011.
This brings me to the next point. How much does KWin depend on the development of a small group of developers? We can see that in average about 45 inidividual people committed to the KWin source code per year. Altogether 275 individual developers contributed to KWin and 10 developers have committed more than 100 changes, and a further eight have committed more than 50 changes. The top ten contributed 68% of all commits, Lubos alone contributed 28 % of all commits. Matthias Ettrich is with 4 % of all commits still in the top five. What really surprised me is that I already contributed 11 % of all commits and Hugo only working on the Oxygen Window decoration is third most active KWin developer – over the complete history.
Now the data is not perfectly accurate. KDE switched the version control system twice and the different VCS bring different patterns of work with them. Due to the bad branching feature of SVN large parts used to be developed outside of SVN and committed as a large bump. This is not the case any more with git: you have all commits out in the open. As at least I have been using git-svn bridge for quite some time, it’s not that surprising that I already have a large number of commits.
Now I had my tool to parse a git history into a statistic diagram and I thought what would be more fun than to compare the history to the one of another big window manager and compositor? So I dared to run the script on the Compiz history. The history in general is not comparable or more to say: it’s just numbers, it’s not possible to draw any conclusions from the history for comparing Compiz and KWin. Here are a few of the reasons:
- The Compiz history only dates back to February 2006. According to Wikipedia Compiz was first released in January 2006. So some history is definitely missing.
- Compiz had a vivid history. It was forked to Beryl, re-merged to Compiz Fusion, re-merged to Compiz, rewritten as Compiz++. I have no idea if the complete history of the forks is present in the repository.
- Compiz has many, many plugins not living in the main repository. It is a question whether those have to be honored. KWin in comparison has all (with a minor exception of two or three effects) effects in the main repository.
- KWin includes complex window decorations (c.f. the number of commits by Hugo) while Compiz includes wrappers around KWin’s and Metacity’s decoration. The theme engine for Compiz (Emerald) seems not to be part of the main repository.
- Compiz code repository is split into several repositories. Here again the question which should be included into a comparison.
- KWin is part of the KDE Plasma Workspaces, which gives us a slightly different development focus and the helping hands of many developers correcting common mistakes (e.g. i18n).
Given the constraints I decided to combine the history of Compiz core and of all the plugins in plugins-main. First again a look at the years:
We see that Compiz had a really strong start. In 2006 and 2007 Compiz had each time around twice as many commits as KWin, the difference is fading out lately. In 2009 KWin had more commits, in 2010 it’s very close and during the first three months of 2011 KWin has clearly more commits. What looks really bad in the Compiz graph is the loss of developers. In 2007 54 developers contributed to Compiz, in 2009 just 14. This illustrates the challenges Compiz faced in the beginning of 2009. At the end of 2008 the number of commiters per month dropped below 5!
Altogether Compiz had 92 developers. Three (David Reveman, Danny Baumann and Dennis Kasprzyk) have more than 1000 commits and make up more than 63 % of all commits. Compiz has only 7 developers with more than 100 commits and those 7 developers contributed more than 88 % of the complete code base. Only four more developers contributed more than 50 commits. Adding those we come to more than 92 %. Now to illustrate the problem I see here: David Reveman who is the initial author and holds 25 % of all commits did his last commit to the core repository in December 2007, Danny Baumann (number 2) in November 2010 (but seems to be still a regular contributor) and Dennis Kasprzyk in April 2010. Of the last 500 commits 392 have been done by Sam Spilsbury. This looks like a bus factor of 1 🙁 For KWin I have 265 of the last 500 commits, though this includes the GLES port with more than 100 commits for itself.
Now I have one more graph I want to show: it includes the commits and authers per month and the accumulated number of commits.
The start Compiz had is really impressive. In a very short time they had almost as many commits as KWin had collected in the many years before. The graph is unfortunately difficult to read as it has a great variance between the months. Nevertheless it highlights the strong momentum Compiz had after the launch. In a few months Compiz had more active contributers than KWin and significantly more commits. In August 2007 Compiz reached a maximum of 415 commits! Unfortunately for the development Compiz failed to keep the momentum and both number of commits and developers seems to fade out.
[1]: All KWin commits are without scripty commits. For formatting the git log I used:
git log –format=format:”%ad %h %an” –date=short –no-merges
My application parses the output and counts the commits per months, the developers per month and the commits per developer. All shown diagrams have a logarithmic scale in y direction.
=-=-=-=-=
Powered by Blogilo
Wouldn’t “lines of code” be a more accurate measure for comparing projects than “number of commits”?
it’s not a comparison of projects, but of activity. The activity is not measurable with sloc
Interesting graphs. What they tell me:
*) You need a certain amount of commits to create a fully functioning compositing WM. If you can’t get there in time (and that time is short these days), you will fail. So against the successfull WMs you’re comparing, there may be quite a few failed projects along the road side.
*) There are enough people out there interested in/capable of coding a windowmanager. Maintaining however is perceived to be boring. Arguably, both kwin and definitely compiz are slowly moving towards maintaining mode – which is where kwin3 basically was before. Where are those people now? Ubuntu?
*) A push towards mobile/tablets could perhaps be done in a similar way: Gathering up momentum and get it over with may be an easier way to achieve it than to assume a constant number of devs and get it done slowly as a side project.
KWin is not moving towards maintaining mode. The development is very active and we have quite some new stuff going on. It’s just behind the scenes, such as the new compositor.
Sorry – bad phrasing of mine, that was pretty much what I meant.
I guess it is just cooler for many people to work on something that generates a visible revolution (e.g. the initial compositing effects) than dealing with the gory details of optimizing speed and stability for an ever widening range of configurations. All the more respect to you and the other kwin devs for doing that – while also working on the long term plan.
Why do we even have to compare activity and commits histories?
If a project/application is more or less complete, only the minimum maintenance is required thus it may become less active over time.
Interesting! If you are looking for a measure for “number of
developers” which also takes into account the contributions of each
developer the concept “effective number” might be interesting. The effective
number is the inverse of the Herfindahl index (
http://en.wikipedia.org/wiki/Herfindahl_index ). We use the “effective
number” at work to get an understanding of the granularity of a loan
portfolio (although we do _not_ limit us to 50 which is mentioned in
the Wikipedia article).
You are surprised that after a few years you already contributed 11% of the commits and the guy who only works on the Oxygen window decoration is the third most active KWin developer. This didn’t surprise me, because I have already realized the hard truth about FLOSS: our community is full of users happily using our software and liking the principles of FLOSS (which is of course a good thing), but when it comes to actually writing code, it is very hard to find developers that want to help. I noticed this for my own software, ok my software is not the most crucial software on the desktop, but when you see that important apps such as Konqueror, Ark, … see little improvements during the last 2 years and that for other software such as KWin, Gwenview, Digikam, Skrooge, … only one guy is blogging about improvements, then you realize that you are not the only one suffering from this problem. And then I saw the announcement of KDE SC 4.6.0 and I saw that only a small amount of apps (in comparison to previous releases) had improvements (the software that did improve, improved greatly, which shows that supposed laziness of the developers is not the problem, but the lack of developers), that’s when I started thinking about what is going on and then I realized that the hard truth not only hits my own software, but also KDE. My wild guess is that the hard truth also hit Nokia with Qt: they realized that if you want to have an ecosystem of software for your mobile phones, then you have to do it yourself and pay developers to do it, or you let Microsoft do the work for you (guess which is easiest if you’re not a software company).
I would go even further and say that nearly any FOSS project lacks developers.
A kind of “famous” example is GIMP lately.
I am not sure why that is this way, yet universities could change a lot themselves, if they cared. Most students have to create ridiculous “programs” which won’t be used by anyone. At the same time group works tend to suck, since there are some people motivated while others aren’t.
Working on a real project could change that. Not only would be bad code rejected but good code would be accepted and would actually be used by _users_. This way not only the knowledge of the students would increase, but also the number of contributors. Some of those would maybe also stick around.
In that regard I think it is quite cool that these French (forgot the name of the university) students work on KDevelop.