Michał "rysiek" Woźniak writes: > On 6/12/20 11:54 AM, infokiller wrote: >> Michał "rysiek" Woźniak wrote: >>> Running virtual machines is extremely resource-intensive, there's no way around it. >> >> But if the issues stem from running VMs (and not switching from Xen), they won't be resolved with Spectrum's current design, right? > > Right you are about the performance issues *I think*. I would however expect > hardware compatibility to improve with kvm. > > But I'd love somebody more knowledgeable to shed some light here. I am by no means a Xen expert (and so the following could be totally wrong, and I would appreciate being corrected), but my understanding is that Linux is likely to provide better power management than Xen due to being better optimised, and not being as focused as Xen on large data center deployments. Additionally, there are interesting Linux virtualization features that may allow for reduced overall hardware requirements by securely sharing resources between VMs. For example, it may be possible to share identical read-only memory pages between VMs. There's not necessarily any reason that running identical copies of Firefox in two different VMs should require two copies of Firefox in host memory. The most promising application of this principle at the moment in virtio-fs[1][2], which supports this page sharing. DAX[3] may also be of interest. Of course, there will be security considerations when deciding whether to adopt these systems, but the point is that this sort of development just isn't happening with Xen. It's all focused on the KVM ecosystem. [1]: https://virtio-fs.gitlab.io/ [2]: https://lwn.net/Articles/788333/ [3]: https://lwn.net/Articles/610174/ Hardware compatibility should certainly be better with KVM. Linux has extremely diverse hardware support, and a much larger user base than Xen, especially on consumer hardware. I'd be confident in saying most relatively recent consumer PC hardware will support Linux reasonably well (a notable exception of course being Nvidea GPUs), and I'd be surprised if Xen worked well on anywhere near as many devices. >>> I feel one needs to be an expert to use QubesOS, but a regular user (with some >>> basic training) can use a Mint or Ubuntu-based system. And I think it makes a >>> lot of sense to offer a middle ground. >> >> Agreed, but I think the current design of Spectrum may improve Qubes' hardware issues, but not the other issues the doc mentions. Possibly switching to containers (which something like gVisor) may solve some of the other issues of Qubes, though that would further degrade security. > > Fun fact, containers were in fact initially considered. :-) > > Here's a good blogpost from Alyssa on why move to kvm, and the advantages of it > over Xen: https://alyssa.is/back-from-cccamp-2019/ Indeed, Spectrum was originally conceived as a Qubes-style operating system that used containers instead of VMs. The reason the focus shifted from containers to KVM is that as I researched the topic, I became less confident in the ability of containers to offer meaningful security, and learned that the overhead of VMs doesn't have to be as big as I thought. Of particular inspiration was "My VM is Ligther (and Faster) than your Container"[4], which demonstrates Xen-based virtual machines with resource requirements roughly equivalent to containers. I also think that Nix might put Spectrum in a unique position to provide lightweight virtual machines. Conventionally, VMs run full-featured Linux distributions, with all the memory and disk requirements that entails. It's not usually feasible to build tailored VM images running only the software that is critical to the task of the VM. With Nix, however, it suddenly is. A big part of the appeal of containers, I think, is that the tooling makes it easy to create small images focused on a single task. VM tooling has never really provided this. It's a sort of vicious cycle of assuming VMs are heavy and therefore it doesn't make sense to have a VM dedicated to a single task, which means that tooling for creating such a VM isn't developed, which means that VMs stay heavy. [4]: http://cnp.neclab.eu/projects/lightvm/lightvm.pdf