Linux Desktop, Package Management, and Commercial Software

Say I’m a company considering porting my program to linux. I find I’m entering a world where everything is streamlined for a different kind of software. The “right” way to install software is through the package manager, but the package manager expects software to come from centralized sources which “package” software developed by others, and to make it worse, different versions of linux use the same software packaged by different people and managed by different package managers. There’s no practical way for me to wedge my way into that system. I have a few choices:

1. I can take over the work of the package maintainers and set up repositories for all the major distros for my program and provide people with directions for how to add a repository to their package manager. (which isn’t exactly smooth sailing for a non-power-user, and which is also a lot of work for me)
2. I can provide package file downloads for the major distributions, losing the package managers ability to keep track of updates, but at least that way having a link to a file people can download and then just click in their file browsers to install. (still quite a bit of work, but at least it’s easy for my users)
3. I can provide a self-extracting installer, completely ignoring the package manager and installing to a directory owned by the user, although by that mechanism I also run the risk of having dependency issues. I don’t mention installers that ignore the package manager and install system-wide… cause that’s a no-no. (fairly easy, but can be frustrating for a user if dependency issues crop up)

All of these solutions have a tendency to build up a lot off cruft somewhere in the system if commercial software becomes seriously used on the linux desktop, which it will have to be for linux to become a serious contender in the desktop market. So, if linux is ever to succeed as a desktop OS, we need to have an elegant, simple, and broadly accepted solution for installing applications (by which I mean programs with minimal integration into the system) which is capable of allowing all of the following:

1. Unprivileged install to user-owned area, while possibly pulling in dependencies for the entire system
2. Privileged install to entire system
3. Dependency integration (one way; apps can depend on OS level stuff, but not the other way around) between it and the OS-level package managers for the different distros
4. Repository support, which needs to be flexible enough to support many ways of distributing content such as bittorrent (since some applications can be BIG), able to handle large numbers of small repositories well from a use model point of view, and easily integrated into browsers and such so that people can provide “install links” which add the repository and install the package
5. Installation from physical media which is also easily integrated into current GUIs and such
6. The ability for programs to integrate with their own update procedures through this system, since some programs cease to function altogether when they are not up to date (clients for certain large network applications, for example) so they need to automatically check that they are up to date when they are run

In order to accomplish this, a few things need to be well defined:

1. An interface to the OS level package manager that any implementation of this system will use, which will need to include a lexicon of names and common expectations for OS-level software that can be depended on, since names are not the same across distros
2. A common format for repositories, both on physical media and on the internet

If those 2 standards can be agreed upon, then implementations of such a system can compete for user approval in the usual manner of open source software, while requiring application vendors to supply only a single object to their customers regardless of distro or implementation of this higher-level system.

On compositing, APIs, performance, and simplicity

I’ve been thinking about the whole software structure involved in the sharing of video hardware among processes doing accelerated video operations (which I mean in a general sense, from video decoding to 3D). The development of compositing has certainly changed the paradigm of how this is done, and I believe it’s a great improvement. However, it came as a hack that the designers of what it was built on had never considered, much like AJAX, and as a result there are some serious design issues that need to be addressed before it will truly work well. The problem that jumps out to me is that the critical path of video content rendering between it’s origin in a userspace process and its display on the screen has to cross the CPU/GPU barrier 3 times, which is just silly. Allow me to illustrate:

1. Program(CPU) gives the GPU some data and directions for how to render it into a buffer. (Critical path goes from CPU to GPU)
2. GPU renders into the buffer and it sits around waiting for the compositor(CPU) to get a scheduling slot. (Critical path goes from GPU to CPU)
3. Compositor(CPU) tells the GPU to take the buffer and render it as a part of another 3D scene. (Critical path goes from CPU to GPU)
4. During output of the next frame, the compositor’s output buffer is put on the screen.

Obviously, the compositor process shouldn’t be in the critical path here. Consider if the path looked like this:

1. Compositor(CPU) responds to some event (such as input), calculates a description (as a function of time) of how to assemble various buffers into a 3D scene, and sends it to the GPU. (Not part of critical path)
2. Program(CPU) gives the GPU some data and directions for how to render it into a buffer. (Critical path goes from CPU to GPU)
3. The GPU uses stored descriptions to automatically composite buffers in preparation for the next frame, thereby getting the program’s data to the screen.

This would not only increase performance of composited desktops, but also greatly simplify the problems involved in avoiding tearing while maintaining low latency from generation to display, etc, since the problem wouldn’t be spread across multiple domains anymore, but entirely in the control of the graphics driver writers. Furthermore, compositors wouldn’t use constant CPU time anymore, could be stacked (which has its uses, from a software point of view), and even esoteric things like direct graphics hardware access from inside virtual machines through I/O virtualization don’t require anything particularly unusual on this layer, which I think is one of the marks of a good solution. In order to implement this though, aside from the things that might need to be done at the hardware and driver levels (about which I know very little), one would need a different kind of graphics API for compositors to allow it to send these compositing descriptions to the GPU; simply extending an existing API wouldn’t cut it.

I don’t pretend to know just how difficult this would be, but I do think it’s the right solution.