How Code flows – about Upstream and Downstream

The FLOSS ecosystem consists of a large amount of independent projects which develop their software. In the end this software should be used by a user and we have the “distributions” to provide the software to them. As the name implies it’s about distributing the software. A distribution takes software from a large amount of independent projects and integrates them to provide a working set applications. A huge and impressive task and that’s the main task of a distribution: they are software integrators.

Various software products depend on each other and there is the chance of conflicts. The distributions need to ensure that this all works. The best matching metaphor for how this works is a river. It flows from the spring to the sea and takes on the water of many other rivers. The code flows from one project to the other to reach at the end the user.

upstream-downstream

Although that looks rather linear, it is not. In truth each independent project has many upstream and many downstream projects. It’s an n:m relationship in each position of the stack. For example for KWin it looks something like that:
upstream-downstream1

For each project it’s important to remember that they are just one of the many, many downstreams of their upstream project. They are not the most important piece of software around, but just one of many. This is important for a working relationship. This also influences how code should flow. Code should flow up the stream. Nobody is helped if each of a downstream of a given project fixes the same bug in their code base. It should be fixed in the project upstream of them. To put it simple: I should not work around a bug in Qt, but fix it in Qt directly.

Nevertheless not all code should be upstreamed. If the code is needed for the downstream integration task it needs to be kept downstream. For example openSUSE used to have a small geeko in the Oxygen window decoration. This should not be upstreamed, because it would cause other downstreams (e.g. Kubuntu) to remove it again. Or it would bring other downstreams to the idea to have their branding feature integrated, too. So we as the upstream would have to accept the Debian logo, the Kubuntu logo, the Gentoo logo – you get the idea. The basic idea is that an upstream project should not try to integrate with their downstream as it’s the downstream’s task to do the integration. This is true for any step of the graph, not just the last one. Circular dependencies are not a good idea.

There are exceptions to the circular dependency rule, but this is very rare. An example is the relationship between the KDE and Qt project which are both upstream and downstream to each other. But the integration inside Qt with KDE is done through plugins allowing to have this integration code been kept downstream.

Also problems can arise if a project starts to become it’s own upstream replacing some of their upstreams. They might want to see their new code being exposed to more projects but other projects on the same level of them might not pick it up as they think it’s specific to this one project. Also it might harm the relationship to other upstream projects. They might think that this downstream is no longer interested and fear that their other downstreams might start to replace them, too. This is not in the interest of the user in the end.

An example of how this can look when it goes wrong could be observed this year with Cinnamon and Cinnarch. Cinnamon is too close to the Linux Mint project, in fact it started as part of Mint and depended on the GNOME version shipped with Mint. This made it impossible for Cinnarch to provide the latest of Arch and Cinnamon. It resulted in Cinnarch dropping support for Cinnamon, but also in Cinnamon trying to get more independent from Linux Mint. Whether this step came in time will be seen in the future.

So in summary: downstreams integrate their upstreams and not the other way around.

10 thoughts on “How Code flows – about Upstream and Downstream”

  1. Martin – Just FYI. In a very recent interview, Clem clarifies on CinnArch issue and how it is resolved now.
    http://www.networkworld.com/news/2013/102113-lefebvre-linux-mint-274911.html?page=5

    There was a bit of a political argument there as well, since GNOME 3.8 compatibility was not considered urgent from a Linux Mint/Cinnamon perspective, and Cinnamon itself wasn’t considered important enough by Arch to delay the GNOME 3.8 upgrade. Eventually we provided distributions using 3.8 a compatibility branch and things worked out.

    This is also the reason most of the GNOME backend was forked recently. We don’t want to release a frontend which is only compatible with specific versions of GNOME and plays catch up with changes it doesn’t need every six months. We want to provide a desktop environment which works everywhere no matter what version of GNOME, if any at all, are present.

    Cinnamon 1.8 talked to gnome-settings-daemon 3.6, cinnamon 2.0 will talk to cinnamon-settings-daemon 2.0. We’re making sure everyone can use Cinnamon and that we can backport it easily.

      1. Sure. BTW you promised you will try the window edge snapping feature in Cinnamon. :-) Had a chance?
        Now have Choqok (twitter) client tiled to one edge so can keep an eye on tweets whilst working on something else. Also, if I open a link from it, I do not lose sight of Choqok and proceed to check with other tweets.

        1. no, haven’t given it a try yet. Will probably wait till the next mint release is available.

  2. Whether or not it is intended, I chose to read between the lines and see this as a discussion about Wayland vs Mir. It’s the FLOSS drama of the day. :P

      1. What if the downstream wants to go a different path than the upstream? How does that affect the total FLOSS stream and is it wrong? Is there a way for a project to become it’s own upstream without getting hate from the community for not conforming?

        1. If a downstream wants to go a different path than an upstream, than something is broken in general. Why should a component like KWin start to fork Mesa?

    1. Well Mark Shuttleworth does seem to have differnt model of Open source. It seems to be a hub model. Of course every hub needs its self-appointed benevolent dictator for life.

      I guess only time will tell whether Mark gets the trains to run on time, but I for one shall be alighting at the first available stop.

Comments are closed.