Spectrum Design

Spectrum will, for now, be a Linux-based system, derived from NixOS. This gives us an actively-developed base with good hardware support, powerful and optimised compartmentalization primitives in KVM, and the reproducible packaging and configuration system that is important for a maintainable compartmentalized system.

The current plan is to implement compartmentalization in Spectrum by running each application inside crosvm, the hypervisor used by ChromiumOS. Qubes-style isolated, composited windowing for applications can hopefully be provided through its virtio_wl, which transports Wayland over the virtio protocol. Using a full virtual machine for each application might come with high resource requirements at first, but over time we should be able to optimise this, for example by doing clever tricks like DAX to a read-only storage device shared by multiple guests to save on duplicated memory. In the short term, it might be prudent to allow multiple applications to run in a single KVM instance, but our long-term focus should be on one per application instance.

Each virtual machine will be generated by a Nix derivation. System and user files within a VM will be shared, to reduce overhead and allow the same files to be worked on from multiple VMs. An isolated VM will read a directory tree from its on-disk representation, and expose it to other VMs over a protocol chosen for simplicity to reduce attack surface. The server inside this VM will be written in a memory-safe language, to mitigate potential memory unsafety exploits. Ideally, this server program would not rely on Unix filesystem code, but would implement its own simplified file system logic, to interact with and expose a file system as a directed acyclic graph, preventing the possibility of a directory traversal attack. In simpler terms, this means that there should be no concept of .. anywhere inside the filesystem VM in relation to the exposed data.

There may be other writable mount points inside the virtual machine, but these will not persist between reboots of the VM. Using Nix to generate virtual machines allows them to be reproducibly built, rolled back, edited, and migrated as source code, rather than large, opaque virtual machine images.

This is a different model of data storage than has been used in previous implementations of security through compartmentalization. In Qubes, user data in each VM is stored in its own virtual block device. This works fine when multiple applications run in a single virtual machine, but would be unmanageable in Spectrum's VM-per-application model. As long as the appropriate precautions are taken, as described above, Spectrum's persistent model should be secure, while providing a more familiar and easier to understand model for users used to a single directory tree.

Which files and directories a VM should be able to access will be defined in Nix. Other resources and permissions, such as network cards and USB controllers, will similarly be defined in Nix. There are three logical sections for the Nix configuration -- applications, which are just packages, resources (virtual or physical devices, files and directories), and application instances, which are mappings between applications and accessible resources. This structure allows users to have multiple instances of the same application, with different permissions. It should be possible to grant additional permissions on a temporary basis at runtime, either through Spectrum system management UI, or through integrations into emerging standards for compartmentalized desktop computing like XDG Desktop Portals, which allow applications to ask for additional privileges through a standard interface, which must be confirmed by trusted user input before being granted.

Initial versions of Spectrum will have the user be responsible for writing Nix code for each application and resource, and the combinations they make between them. In future, it would be good if this could be handled by a graphical interface and a data file for simple cases, with Nix importing the data file and using it as input for the standard functions that would be called in a Nix-language configuration file. This use case should be kept in mind when writing the Nix API for Spectrum.

Spectrum is expected to largely run on personal computers, most of which will almost certainly use the x86_64 or aarch64 architectures. Unfortunately, these common architectures are the most lacking in terms of trustworthiness. All require unauditable proprietary blogs to boot, and the Intel Management Engine, AMD Platform Security Processer, and ARM TrustZone, all of which are constantly running highly privileged, unauditable code. A backdoor or compromise in any of this code could give complete access to the system, invisibly to running the operating system. As more functionality is moved into these environments, the attack surfaces grow larger and larger, and already many vulnerabilities have been demonstrated in the most studied of these systems, Intel's Management Engine. Fears of backdoors are not unjustified either — VIA C3 x86 CPUs used in personal computers have been found to contain a hardware backdoor allowing local privilege escalation.

I would like Spectrum to additionally have first class support for at least ppc64le. This is the only other architecture that can come close to the sheer performance x86_64 can offer at the high end, and in stark contrast to x86_64, it is possible to buy a new ppc64le (POWER9) system that does not require any proprietary firmware that cannot be inspected and audited. One of the advantages of Spectrum's Linux base is the extremely wide hardware support that Linux offers, so the only blocker for POWER9 support is support in crosvm for virtualizing that architecture, which is outside the expertise of anybody currently working on Spectrum but would be a very welcome contribution.

Ideally, all Spectrum packages, for all architectures, would be built on diverse hardware including a platform with POWER9's level of openness. Even if a user has to trust the x86_64 computer available to them, anti-freedom firmware, undocumented backdoors and all, they would be able to benefit from binary packages that were built on trustworthy hardware without these limitations. Unfortunately, preliminary research has shown that x86_64 virtualization on POWER9 is not currently sufficiantly performant for this to be feasible.

Permission is granted to copy, distribute and/or modify this document under either the terms of the Creative Commons Attribution-ShareAlike 4.0 International License, or the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.