This is an HTML version of the application I made to NLNet for a NGI Zero Privacy and Trust Enhancing Technologies grant.
I would like to thank rysiek for telling me about the grant and encouraging me to apply, and Katharina Fey for her help in writing the application.
Spectrum
Can you explain the whole project and its expected outcome(s).
Security through compartmentalization has emerged as the solution to the problem that desktop computer users by and large don't have access to meaningful security on their systems.
The Spectrum system will improve significantly on many problems affecting existing compartmentalized desktop OSes. For greater hardware compatibility, the base system will run on the Linux kernel rather than a server-oriented hypervisor, and there will be multiple backends for isolated systems, from containers to virtual machines. User data and application state will be managed centrally, while remaining isolated, meaning that it can be backed up and managed as a whole, rather than mixed up in several dozen VMs. The host system and isolated environments will all be managed declaratively and reproducibly, saving the user the burden of maintaining many different virtual computers, allowing finer-grained resource access controls and making it possible to verify the software running across all environments.
The requested funding should get Spectrum to the point of being a usable system that realises these benefits. I am an EU citizen, and project infrastructure and evangelism would be primarily Europe-based.
Have you been involved with projects or organisations relevant to this project before? And if so, can you tell us a bit about your contributions?
For reasons expanded on in the comparison with existing efforts, being built on top of the Nixpkgs package repository is important to the success of the project. I am a Nixpkgs committer, and have worked with packaging, isolation, and configuration management as part of my work with Nixpkgs. Recently, I helped design and implement a system to create reproducible container images in Nixpkgs. I envisage that as part of my work on Spectrum I will upstream a lot into Nixpkgs, benefiting the wider ecosystem beyond this project.
I have used QubesOS, the main existing effort in this space, as my primary desktop operating system, and I have followed the research done by the QubesOS team. This interest, over a period of years, has given me a strong understanding of the intricacies of security through compartmentalization, and the problems with existing implementations.
€49200
Explain what the requested budget will be used for?
Does the project have other funding sources, both past and present?
Explain costs for hardware, human labor, travel cost to technical meetings, etc.
I expect to use the requested budget something like this. The funding should cover one year of development, maintenance, distribution, and promotion of the project.
I plan to purchase build hardware rather than renting build servers to protect against the threat of compromise by a malicious host. This is the same strategy that Qubes takes. Bit-for-bit reproducible builds would mitigate this attack vector, but the system is unlikely to be entirely bit-for-bit reproducible, at least initially, because each package would have to be checked for reproducibility, and development effort would have to be put into making the non-reproducible packages reproducible. This is valuable work, but not required for an initial release. There is ongoing effort to make nixpkgs bit-for-bit reproducible.
Compare your own project with existing or historical efforts
(e.g. what is new, more thorough or otherwise different)
The current standard-bearer for this approach to security is QubesOS. QubesOS runs Linux and Windows virtual machines on the Xen hypervisor, and composites them all together into a single desktop environment, with utility programs that allow users to manage their complete system and control the interactions between VMs. While Qubes has been largely successful in creating a usable system around security by compartmentalization, its fundamental technology choices leave a lot of room for a new approach. These choices are very deeply embedded into Qubes, and it would be next to impossible to build a system with the benefits I propose here on top of Qubes.
The areas in which I propose to improve over QubesOS are:
Qubes is notoriously annoying to install, even on well-supported hardware. The installer gives very little useful debugging information, and once Qubes has been installed, no information is recorded that would help to reinstall it.
The installation process of NixOS, upon which Spectrum is based, is currently a somewhat manual process — a user must partition their disk themselves, then run some shell commands to generate a system configuration and install the operating system. It is, however, much more reliable than Qubes', and the effort required to make this even easier (e.g. by including a partitioning wizard as part of the installation process, or providing a graphical tool) would be much less than attempting to improve the black box that is Qubes' installation process. This is something of general interest to the NixOS project, and so it may be that it doesn't have to be developed as part of Spectrum. Alternatively, if it was developed for Spectrum, it could be upstreamed and benefit the NixOS project as a whole.
Because a lot of activity on a Qubes system happens inside multiple long-lived virtual machines, a Qubes system requires the maintenance effort of many computer systems that must all be kept up to date, configured, and debugged. Qubes provides some utilities to mitigate this effort, such as TemplateVMs and a system-wide backup utility, but it still requires a lot of effort to be kept functioning.
One of the worst things about maintaining a Qubes system (or indeed almost any Unix system, but Qubes is particularly affected due to effectively being many Unix systems in one) is that, due to a system update or a configuration change, something that had previously been configured by the user to work correctly is now broken, and there's no way to tell why that is. There's no way to figure out what changed, so the user is left forced to restore a working backup (if they have one!) and try again. Once a machine has been configured, that configuration cannot be reproduced automatically on a reinstall or on another computer. It very quickly becomes a black box that is difficult to ever modify for fear of breaking it, because breakages are catastrophically difficult to recover from. It is extremely difficult for OS developers to debug why a user's system isn't working, because they have no way to see the whole configuration of the system, or why things are configured the way they are — did the user choose this? Was it changed automatically by a program? Is it just an old default that an operating system upgrade neglected to change? Qubes attempts to solve this problem through Salt, but this is really papering over the cracks, because Salt is not fully reproducible. Two computers configured using the same Salt configuration can still end up being different, if resources such as system packages hosted on the internet have changed. Additionally, there's nothing to prevent a Salt-configured system being modified outside of Salt, either by a user or, worse, by a program on behalf of the user (possibly without the user realising).
Qubes is a distribution of the Xen hypervisor, and as such the Qubes system is very tightly coupled to Xen. Xen is designed primarily for use of servers, rather than laptops, and therefore compatibility with personal computers, especially laptops, is very limited.
The aspect of Spectrum most likely to be a subject of heavy discussion is that, unlike Qubes, it offers support for container-based isolation as well as VMs. While container-based isolation is likely to be somewhat easier for an attacker to exploit due to the shared kernel, container-based isolation is still valuable for the vast majority of personal computers whose only other option is no virtualization. Containers are very widely used, and any discovery of a vulnerability in a container system is likely to have a huge impact on industry and be fixed extremely quickly. A large portion of Spectrum users are likely to be people who are uncomfortable that any application they run can freely access all of their files, but for whom container system zero days are not a part of the threat model. Container-based isolation is very well suited to a user of this kind who is unwilling or unable to acquire expensive hardware to be able to run Qubes.
The Qubes model encourages the user to create one isolated environment for every identity or task they wish to accomplish. There might be a "Work" environment and a "Finance" environment. This is a step in the right direction, but in most cases it still gives software far more access than is necessary for it to be able to do its job, putting more data than necessary at risk in the event of any application being compromised. There is no reason that a word processor in the "Work" environment should be able to access the user's work mail, but Qubes lacks the fine-grained permissions to be able to describe this. Instead, the user must create two virtual machines, one for work document editing, and one for work mail. To work on a word processing document received via mail, the user must copy it to their document editing virtual machine. The user who rightly tries to apply the principle of least access to the applications they use is left with an enormous list of environments for each application in each context, to the point that there are so many environments that they are unmanageable. Since data is stored per-environment, it becomes virtually impossible to keep track of which environment has the up-to-date version of the file one is looking for.
In Spectrum, data is stored in a single directory hierarchy, so there is a single source of truth, rather than data being scattered across virtual machines. A user defines isolated environments for a single application* at a time, so there is no reason for an application to end up with far more access than it needs. At the same time, it's not enough to be able to say "LibreOffice should be able to access my work documents", because then a malicious document that is available to the "Personal" workspace but isolated from the "Work" workspace would be able to use the instance of LibreOffice it's running in to be able to cross the security boundary. Instead, Spectrum configuration is based around "instances" of applications. There can be a "Work" instance of LibreOffice and a "Personal" instance of LibreOffice. Both of those can have separate state and have different data available, and neither of them are able to access the user's mail.
* an application in this sense is not necessarily a single program. For graphical applications, this is likely to be the case, but, for example, a terminal emulator is only useful if it has programs available for the user to run. It would be more technically correct to say that environments are configured for Nix expressions, but "application" is easier to understand, and in most cases the user shouldn't need to be aware of the distinction.
One area in which Qubes is superior to Spectrum is in the size of its trusted computing base — the software that must be assumed to be secure for the whole system to be secure. Qubes focuses heavily in using as small a TCB as possible. This gives a potential attacker fewer entry points in which they could find a vulnerability. Spectrum, on the other hand, chooses a larger TCB for better hardware support and ease of development, giving many more people the opportunity to use a compartmentalized operating system.
Aside from Qubes, the only other attempt of which I am aware to create a compartmentalized desktop operating system is SubgraphOS, a container-based system. The developers of the system manually create containers for common desktop applications (around 20). However, there are significant flaws with the Subgraph approach. It has no support for multiple security domains or identities, and it provides no additional security when using any application other than the 20 or so which have been chosen to be manually containerized. I don't believe that any approach which requires manual containerization can ever succeed, because the security model falls apart as soon as the user needs to use any slightly uncommon software. Finally, SubgraphOS has not seen a release since an alpha version in 2017, so I don't consider it to be a serious contender for a secure desktop operating system.
What are significant technical challenges you expect to solve during the project, if any?
The major challenge in developing the system is creating an understandable solution for configuring a Spectrum system. The model is more complex than creating virtual machines, because there are more concepts to understand — applications, application instances, and state, and hardware devices. Even coming up with and implementing an easy-to-use text-based configuration system for this is likely to be a challenging task, and I would like there to be some sort of graphical system as well so that the majority of users could configure the system for their needs without having to learn Nix. Making these concepts understandable and usable is by no means out of reach, but it will require a novel approach.
At a lower level, to provide support for multiple isolation backends (VMs, container runtimes, etc.), a Nix-based abstraction layer must be developed. Which isolation system is in use should be completely transparent to the configuration, so that it can be changed without requiring a whole system to be reconfigured.
While these challenges are significant, I think the rest of the core functionality will be reasonably straightforward to implement. A lot of features required for a compartmentalized system — for example displaying windows from many isolated systems on a single desktop with unspoofable window decorations identifying which isolated system they are running in — are solved problems now, and there are preexisting solutions that can be easily integrated into a new system. When Qubes was developed, this was not the case. Therefore, a year should be ample time to integrate these existing solutions, and solve the new problems identified above.
Describe the ecosystem of the project, and how you will engage with relevant actors and promote the outcomes?
E.g. which actors will you involve? Who should run or deploy your solution to make it a success?
I expect to work closely with the Nixpkgs project, with which I am already heavily involved, to promote as much work that comes out of Spectrum as possible and to allow all users of Nix to share in some of its benefits, without having to use a different system. There is a sizable community around Nix already, and so it presents a good opportunity to make people aware of Spectrum, and invite interested people to get involved in using or developing the system early on.
The other obvious project to engage with is Qubes. I expect this to be more challenging, because the forums for general discussion of secure desktop operating systems, such as the secure-desktop mailing list started by the Qubes project, no longer appear to be active. I think that there is a lot of potential for collaboration between the projects, though, especially with regards to discussion about security considerations that are shared between the two systems. I plan to try to engage with Qubes developers and users through project discussion channels, and also through discussions at conferences and similar events.
Finally, what Spectrum ultimately needs to succeed is users. The people who are likely to use a system like Spectrum in its infancy are people who already know about security through compartmentalization, and already know about Qubes. I will reach these people by publishing regular writings about Spectrum, its development progress, and its advantages, and sharing this work online where people with these interests are likely to see it. I will give talks about Spectrum at relevant conferences. Longer term, I expect a userbase from this core audience will spread awareness of the system, to the point where it will be mentioned in any discussion of secure desktop operating systems. This will make it easy to find for anybody who realises that they need a secure desktop OS. The next thing to do is spread awareness of secure desktop operating systems to the general public, but I consider that to be beyond the scope of this proposal — before we can spread awareness of secure desktop computing, we must first ensure that the tools are available to actually do it.