How Code flows – about Upstream and Downstream

The FLOSS ecosystem consists of a large amount of independent projects which develop their software. In the end this software should be used by a user and we have the “distributions” to provide the software to them. As the name implies it’s about distributing the software. A distribution takes software from a large amount of independent projects and integrates them to provide a working set applications. A huge and impressive task and that’s the main task of a distribution: they are software integrators.

Various software products depend on each other and there is the chance of conflicts. The distributions need to ensure that this all works. The best matching metaphor for how this works is a river. It flows from the spring to the sea and takes on the water of many other rivers. The code flows from one project to the other to reach at the end the user.

Although that looks rather linear, it is not. In truth each independent project has many upstream and many downstream projects. It’s an n:m relationship in each position of the stack. For example for KWin it looks something like that:

For each project it’s important to remember that they are just one of the many, many downstreams of their upstream project. They are not the most important piece of software around, but just one of many. This is important for a working relationship. This also influences how code should flow. Code should flow up the stream. Nobody is helped if each of a downstream of a given project fixes the same bug in their code base. It should be fixed in the project upstream of them. To put it simple: I should not work around a bug in Qt, but fix it in Qt directly.

Nevertheless not all code should be upstreamed. If the code is needed for the downstream integration task it needs to be kept downstream. For example openSUSE used to have a small geeko in the Oxygen window decoration. This should not be upstreamed, because it would cause other downstreams (e.g. Kubuntu) to remove it again. Or it would bring other downstreams to the idea to have their branding feature integrated, too. So we as the upstream would have to accept the Debian logo, the Kubuntu logo, the Gentoo logo – you get the idea. The basic idea is that an upstream project should not try to integrate with their downstream as it’s the downstream’s task to do the integration. This is true for any step of the graph, not just the last one. Circular dependencies are not a good idea.

There are exceptions to the circular dependency rule, but this is very rare. An example is the relationship between the KDE and Qt project which are both upstream and downstream to each other. But the integration inside Qt with KDE is done through plugins allowing to have this integration code been kept downstream.

Also problems can arise if a project starts to become it’s own upstream replacing some of their upstreams. They might want to see their new code being exposed to more projects but other projects on the same level of them might not pick it up as they think it’s specific to this one project. Also it might harm the relationship to other upstream projects. They might think that this downstream is no longer interested and fear that their other downstreams might start to replace them, too. This is not in the interest of the user in the end.

An example of how this can look when it goes wrong could be observed this year with Cinnamon and Cinnarch. Cinnamon is too close to the Linux Mint project, in fact it started as part of Mint and depended on the GNOME version shipped with Mint. This made it impossible for Cinnarch to provide the latest of Arch and Cinnamon. It resulted in Cinnarch dropping support for Cinnamon, but also in Cinnamon trying to get more independent from Linux Mint. Whether this step came in time will be seen in the future.

So in summary: downstreams integrate their upstreams and not the other way around.