How Code flows – about Upstream and Downstream

The FLOSS ecosystem consists of a large amount of independent projects which develop their software. In the end this software should be used by a user and we have the “distributions” to provide the software to them. As the name implies it’s about distributing the software. A distribution takes software from a large amount of independent projects and integrates them to provide a working set applications. A huge and impressive task and that’s the main task of a distribution: they are software integrators.

Various software products depend on each other and there is the chance of conflicts. The distributions need to ensure that this all works. The best matching metaphor for how this works is a river. It flows from the spring to the sea and takes on the water of many other rivers. The code flows from one project to the other to reach at the end the user.

upstream-downstream

Although that looks rather linear, it is not. In truth each independent project has many upstream and many downstream projects. It’s an n:m relationship in each position of the stack. For example for KWin it looks something like that:
upstream-downstream1

For each project it’s important to remember that they are just one of the many, many downstreams of their upstream project. They are not the most important piece of software around, but just one of many. This is important for a working relationship. This also influences how code should flow. Code should flow up the stream. Nobody is helped if each of a downstream of a given project fixes the same bug in their code base. It should be fixed in the project upstream of them. To put it simple: I should not work around a bug in Qt, but fix it in Qt directly.

Nevertheless not all code should be upstreamed. If the code is needed for the downstream integration task it needs to be kept downstream. For example openSUSE used to have a small geeko in the Oxygen window decoration. This should not be upstreamed, because it would cause other downstreams (e.g. Kubuntu) to remove it again. Or it would bring other downstreams to the idea to have their branding feature integrated, too. So we as the upstream would have to accept the Debian logo, the Kubuntu logo, the Gentoo logo – you get the idea. The basic idea is that an upstream project should not try to integrate with their downstream as it’s the downstream’s task to do the integration. This is true for any step of the graph, not just the last one. Circular dependencies are not a good idea.

There are exceptions to the circular dependency rule, but this is very rare. An example is the relationship between the KDE and Qt project which are both upstream and downstream to each other. But the integration inside Qt with KDE is done through plugins allowing to have this integration code been kept downstream.

Also problems can arise if a project starts to become it’s own upstream replacing some of their upstreams. They might want to see their new code being exposed to more projects but other projects on the same level of them might not pick it up as they think it’s specific to this one project. Also it might harm the relationship to other upstream projects. They might think that this downstream is no longer interested and fear that their other downstreams might start to replace them, too. This is not in the interest of the user in the end.

An example of how this can look when it goes wrong could be observed this year with Cinnamon and Cinnarch. Cinnamon is too close to the Linux Mint project, in fact it started as part of Mint and depended on the GNOME version shipped with Mint. This made it impossible for Cinnarch to provide the latest of Arch and Cinnamon. It resulted in Cinnarch dropping support for Cinnamon, but also in Cinnamon trying to get more independent from Linux Mint. Whether this step came in time will be seen in the future.

So in summary: downstreams integrate their upstreams and not the other way around.

Thoughts about the Open Source Tea Party

I have been thinking about whether I should write a blog post about the following topic for a long time. After all Jono asked us to calm down and not put more oil into the fire. But on the other hand I had asked Jono to make sure that there are no personal insults and attacks against my person several times and unfortunately on the Ubuntu Planet there is still a blog post which attacks me personally without any sign that this will change. As I had been attacked by the Ubuntu community quite a lot over the last half year and I had to ask Jonathan to tell Jono that I’m not the scape goat for Ubuntu, I think it is important that I stand up against this and point out the abusive behavior we get from the Ubuntu community.

First of all I want to verbatim quote the Ubuntu Code of Conduct:

Be respectful

Disagreement is no excuse for poor manners. We work together to resolve conflict, assume good intentions and do our best to act in an empathic fashion. We don’t allow frustration to turn into a personal attack. A community where people feel uncomfortable or threatened is not a productive one.

And now I’m going to quote verbatim what Mark Shuttleworth wrote:

Mir is really important work. When lots of competitors attack a project on purely political grounds, you have to wonder what THEIR agenda is. At least we know now who belongs to the Open Source Tea Party 😉 And to put all the hue and cry into context: Mir is relevant for approximately 1% of all developers, just those who think about shell development. Every app developer will consume Mir through their toolkit. By contrast, those same outraged individuals have NIH’d just about every important piece of the stack they can get their hands on… most notably SystemD, which is hugely invasive and hardly justified. What closely to see how competitors to Canonical torture the English language in their efforts to justify how those toolkits should support Windows but not Mir.

Mark took care to write it so generic that it would fit Intel, Wayland, KDE, GNOME, Enlightment, Red Hat, systemd and everybody else who criticized the Mir decision. Nevertheless I’m convinced that the primary recipient of that attack is the KDE community and especially me personally. This is something I derive from a comment Mark put below his blog post:

When a project says “we will not accept a patch to enable support for Mir” they are saying you should not have the option. When that’s typically a project which goes to great lengths to give its users every option, again, I suggest there is a political motive.

If we combine all of it, it’s getting clear that he addresses the KDE community. Who else has support for Windows and is known for lots of options? Of all the communities, projects and companies listed above only KDE offers Windows components (well Intel as well, but I assume that Mark is not going to blame Intel for that). Thus I’m assuming that Mark intended those comments only against the KDE community. I asked him in a comment to his blog post to clarify, unfortunately Mark has at the time of this writing not yet replied and the comment is still awaiting moderation. I also copied the same comment to Google+ and included Mark and Jono, but still no clarification.

Now people could say that it’s not that bad what Mark wrote. But his claims are factually wrong and need to be corrected. After all we don’t want that his followers repeat the false claims over and over again to attack the KDE community. I’m now going to reply to the claims without going down to the level of personal attacks but just showing that all those claims are factually wrong if they are intended against the KDE community, KWin and me in person.

So let’s look at the claims one by one.

When lots of competitors attack a project on purely political grounds

together with

When a project says “we will not accept a patch to enable support for Mir”

I said that I will not accept a patch for Mir, but this is not a political decision, but a pure technical one. I’m now going to quote myself from my very first blog post on the subject of Mir:

Will KWin support Mir? No! Mir is currently a one distribution only solution and any adjustments would be distro specific. We do not accept patches to support one downstream. If there are downstream specific patches they should be applied downstream. This means at the current time there is no way to add support and even if someone would implement support for KWin on Ubuntu I would veto the patches as we don’t accept distro-specific code.

Maybe Mark thinks that this is a political decision. But not for me: this is a pure technical decision as we would not be able to maintain the code. And Mark should know about the costs of maintaining code. After all at the podium discussion about CLA at Desktop Summit 2011 Mark told us that the CLA is needed because of the maintenance costs.

Furthermore I had dedicated a complete blog post on the technical reasons on why we do not want to and cannot support Mir. Mark should have been aware of this blog post given that Jonathan re-blogged it to Planet Ubuntu. In summary I cannot understand how Mark could think that these are political decisions given that I clearly outlined the technical reasons.

So let’s look at the next part:

you have to wonder what THEIR agenda is

Well yes, one has. As I showed above I gave a technical reason in less than 24 hours after the Mir announcement. I wonder how Mark can seriously think that we could have come up with an agenda against a product we didn’t know of before or that we are that fast. So to make it clear: there is no agenda. My only agenda is to correct false claims as in this blog post.

Personally I’m wondering what Canonical’s agenda is with the strong lobbying for us to support Mir and these constant attacks against my person. Mark is not the first one to directly attack me since the announcement of Mir.

The next part would be the NIH part. I do not know how that would fit in with KDE as I’m not aware of anything we NIH’ed recently. Also Lennart already commented on that. I think there is nothing more to add to what Lennart wrote.

And last but not least there is:

What closely to see how competitors to Canonical torture the English language in their efforts to justify how those toolkits should support Windows but not Mir.

I would be very interested in seeing where anybody from the KDE community justifies the Windows support in favor of Mir. This just doesn’t make any sense. So let’s look at it in more detail. As Mark states himself most of the applications do not have to care about Mir at all as the toolkit (in our case Qt) takes care of that. That’s exactly the reason why KDE can offer Windows ports of applications. It’s more or less just a recompile and it will work. In some cases X11 dependencies had to be abstracted and exactly that will ensure that the applications will also work on Mir. So to say thanks to the Windows port the applications will work on Mir (and on Wayland). Side note: as Aaron explains on Google+ of course Mark is wrong in saying that applications do not have to care, of course the technological split affects all applications.

As Mark also states what will need adjustments are the desktop shell programs. In case of KDE that would be mostly KWin. I’m now quoting the “mission statement” for the KWin development:

KWin is an easy to use, but flexible, composited Window Manger for Xorg windowing systems on Linux.

As one can see we do not consider Windows as a target for our development. It even goes so far to exclude non-Linux based unix systems. I’m quite known for thinking that support for anything except a standardized Linux stack (this includes systemd) is a waste of time. One can find my opinion to that on blog posts, mailing list threads, review requests or just talking to people from the KDE community who know my opinion about that.

There is an additional interesting twist in this claim about Windows vs. Ubuntu. KWin as explained is currently working on Kubuntu and not on Windows and this will stay so as long as Kubuntu is able to offer either Xorg or Wayland packages. If the Kubuntu community would no longer be able to offer such packages it would be due to changes in the underlying stack introduced by Ubuntu. So it can only be Ubuntu to remove support for KWin, not KWin removing support for Kubuntu. Furthermore it’s of course the task of a distribution to integrate software and not our task to integrate with a distribution.

Even more some years ago one was able to use KWin in Ubuntu. But then Canonical decided to introduce Unity and implement it as a plugin to Compiz. Since then it is no longer possible to run KWin in Ubuntu. A decision made by Canonical. I’m not blaming them for that, don’t get that wrong. I’m just pointing out to show how wrong it is to try to blame us for not supporting Ubuntu. It was Ubuntu which decided to no longer offer the possibility to run our software in Ubuntu. This behavior over the time made me think that I’m being made a scape goat, that Canonical tries to blame me for them moving away from the rest of Linux.

In summary we can see all the claims put up by Mark to attack the KDE community are false.

Last but not least I want to say something about a very common claim: I do neither hate Mir nor Canonical. I can hardly give prove to it, but I just point out that I attended the German Ubuntu community conference last weekend and also last year. If I were in general against Canonical I wouldn’t do something like that, wouldn’t I?

Porting a KControl Module to KF5

Over the last days I ported a few KControl Modules (KCM) to frameworks 5. As it’s a rather simple task I decided to document the needed steps. Yesterday I ported over KInfoCenter with all it’s modules:

Aus 2013-10-15

First some preparation tasks. I highly recommend to configure kdevelop to not stop on the first compile error. This helps to find pattern in the errors during the porting and also allows to start with the most easy tasks and not have to start with a difficult one, which might turn out to be a non-issue once the other errors are corrected. I also recommend to have the KDE5PORTING.html from kdelibs-frameworks open in your browser.

Adjust CMakeLists.txt

First of all you need to re-enable the directory containing the KCM in CMakeLists.txt. Then one should do small adjustments in the KCM’s CMakeList.txt.

Drop any kde4_no_enable_final line – that got dropped:

kde4_no_enable_final(foo)

Remove a few definitions to get less compile errors – especially you don’t want a compile error for each cast from const char* to QString:

remove_definitions(-DQT_NO_CAST_FROM_ASCII -DQT_STRICT_ITERATORS -DQT_NO_CAST_FROM_BYTEARRAY -DQT_NO_KEYWORDS)

Search for the target_link_libraries of the KCM and remove all variables with Qt4 and KDE4. Instead just add a few frameworks you can be sure to need:

target_link_libraries(kcm_foo
    KF5::KCMUtils
    KF5::KI18n
    ${KDE4Support_LIBRARIES}
)

This should be enough for getting most of a KCM to compile. Further frameworks should be added once you hit linker errors.

Adjust desktop file

The next step is rather simple. Look for the desktop file of the kcm. Most often called foo.desktop and look for the Exec line and change from kcmshell4 to kcmshell5:

 [Desktop Entry]
Exec=kcmshell5 foo

Common Compile Problems

Now it’s time to start the compile, fix loop till you hit linker errors. I just want to present the most common errors I have hit so far and how to fix them.

Remove QtGui/ from includes

/home/martin/src/kf5/kde-workspace/kcontrol/keys/kglobalshortcutseditor.cpp:37:32: fatal error: QtGui/QStackedWidget: No such file or directory
 #include <QtGui/QStackedWidget>
                                ^
compilation terminated.

This one is rather simple: just remove all QtGui/ or QtCore/ from the #include. There’s also a small helper application in the Qt source tree, but for a small code base it might be easier to just fix manually. Remember to use block selection mode if the includes are all nicely one below the other.

Q_SLOTS

A common problem is that the code uses slots and signals and that should be Q_SLOTS or Q_SIGNALS. A compile error looks like this:

/home/martin/src/kf5/kde-workspace/kcontrol/keys/select_scheme_dialog.h:38:9: error: expected ‘:’ before ‘slots’
 private slots:
         ^
/home/martin/src/kf5/kde-workspace/kcontrol/keys/select_scheme_dialog.h:38:9: error: ‘slots’ does not name a type

Easy to fix: just replace by Q_SLOTS. I recommend to directly recompile after one of those errors as they cause many compile issues.

i18n

In KDELibs4 the include of KLocale was needed for i18n, in KF5 it’s KLocalizedString. So this is bound to fail:

/home/martin/src/kf5/kde-workspace/kcontrol/keys/globalshortcuts.cpp:67:89: error: ‘i18n’ was not declared in this scope
                     i18n("You are about to reset all shortcuts to their default values."),
                                                                                         ^

The fix is really simple. Look for

#include <KLocale>

and replace by

#include <KLocalizedString>

In the uncommon situation that something from KLocale is used you of course need to keep it and add ${KDE4Attic_LIBRARIES} to the target_link_libraries in the CMakeLists.txt. Another possibility is to directly port to QLocale.

KGlobal::config()

In case you get a compile error about missing KGlobal when using KGlobal::config(), do not just add the missing include, but port over to the new way:

KSharedConfig::openConfig();

KComponentData

Each KCM I ported so far failed with the following error in the ctor:

/home/martin/src/kf5/kde-workspace/kcontrol/keys/globalshortcuts.cpp: In constructor ‘GlobalShortcutsModule::GlobalShortcutsModule(QWidget*, const QVariantList&)’:
/home/martin/src/kf5/kde-workspace/kcontrol/keys/globalshortcuts.cpp:37:13: error: ‘componentData’ is not a member of ‘GlobalShortcutsModuleFactory’
  : KCModule(GlobalShortcutsModuleFactory::componentData(), parent, args),
             ^

The solution is again very simple: just drop the call to componentData():

  : KCModule(parent, args)

KAboutData

KAboutData changed in frameworks with the old one being moved to K4AboutData. Luckily the changes are rather simple and a pattern can be used:

  • ki18n -> i18n
  • KLocalizedString() -> QString()
  • 0 -> QString()
  • wrap normal string in QStringLiteral()
    • As an example the old code:

           KAboutData *about =
               new KAboutData(I18N_NOOP("kcmfoo"), 0,
                              ki18n("KDE Foo Module"),
                              0, KLocalizedString(), KAboutData::License_GPL,
                              ki18n("(c) 2013 Bar FooBar, Konqui"));
      
          about->addAuthor(ki18n("Bar FooBar"), KLocalizedString(), "foobar@kde.org");
          about->addAuthor(ki18n("Konqui"), KLocalizedString(), "konqui@kde.org");
      

      becomes:

           KAboutData *about =
              new KAboutData(I18N_NOOP("kcmfoo"), QString(),
                             i18n("KDE Foo Module"),
                             QString(), QString(), KAboutData::License_GPL,
                             i18n("(c) 2013 Bar FooBar, Konqui"));
      
          about->addAuthor(i18n("Bar FooBar"), QString(), QStringLiteral("foobar@kde.org"));
          about->addAuthor(i18n("Konqui"), QString(), QStringLiteral("konqui@kde.org"));
      

      Common other problems

      Usages of KUrl can in most cases just be switched to QUrl and it will work as intended. The same is true for KAction, though if it sets global shortcuts you need to properly port following the steps in the porting guide. A usage of KFileDialog can be changed to QFileDialog – be aware that the order of arguments is different in QFileDialog.

      Linker errors

      If everything went fine you should reach a point where you hit linker errors. Now we need to add the required frameworks to the target link libraries. I try to locate the header file of a class which threw a linker error to get the framework. E.g.

      $ locate kiconloader.h
      /home/martin/src/kf5/kdelibs-frameworks/tier3/kiconthemes/src/kiconloader.h
      

      So the file is part of KIconThemes and thus one needs to add KF5::KIconThemes to target_link_libraries.

      Testing

      Once all linker errors are fixed, install the kcm, and run in your KF5 environment:

      $ kbuildsycoca5
      $ kcmshell5 foo
      

      kcmshell5 is part of kde-runtime. In case you don’t have that one installed you can also use systemsettings for a KCM that is listed there. Otherwise just build kde-runtime.

      And that’s it. Not much work and hardly any code to change.