- Spectrum Discuss - spectrum-os.org

Comparison to Qubes OS
by joweill＠icloud.com 15 Jun '20

15 Jun '20

Hi there, I just discovered this project and I'm really excited about it! I've long waited for an OS that combines Qubes-like compartmentalization with the reproducibility of Nix/GUIX. I'm trying to understand what this project aims to improve over Qubes, other than the integration of Nix (which I do think is really important!). I read the motivation page [1], but I'm not yet convinced by most of the points mentioned that relate to Qubes. Going over each usability issue mentioned in the motivation doc: - "Hardware compatibility is extremely limited": I don't believe this is really the case for the minimum Qubes 4 requirements [4]: most modern computers people buy support these. Is there anything I'm missing? - "People are reluctant to use Xen on their computer for power management etc. reasons." Can you elaborate on these issues? - I know that Qubes considered using KVM and decided against it for security reasons [2]. My understanding is that the downside of this decision is the limited hardware support, which is one of the things that Spectrum views as an opportunity for improvement. Can you elaborate on this decision? - "VMs are heavy": How will Spectrum improve on this without sacrificing security? - "GUI applications are buggy, command line tools are mostly undocumented": I assume that the reason for this is the lack of resources the Qubes project has. However, I don't see how this will be be better in the case of Spectrum which is a new project with one developer. More generally, I'm wondering whether this projects' goals couldn't be better achieved by trying to work with the Qubes developers to integrate Nix. It may very well be that they would reject it for some reason, but then the logical next step would be to fork Qubes. Have you reached out to the Qubes developers? Thanks in advance! ## References [1] https://spectrum-os.org/motivation.html [2] https://www.qubes-os.org/faq/#why-does-qubes-use-xen-instead-of-kvm-or-some…

5 18

This Week in Spectrum, 2020-W24
by Alyssa Ross 14 Jun '20

14 Jun '20

An interesting week. Didn't go quite in the direction I expected it to. Documentation ------------- Cole kindly provided[1] some documentation[2] for spectrum-vm for the developer manual. This was especially nice, because it marked the first time I was able to remove[3] an item from the Contribution Ideas page[4] because somebody had done the contribution! Thanks Cole! [1]: https://spectrum-os.org/lists/archives/spectrum-devel/87zh97coks.fsf@alyssa… [2]: https://spectrum-os.org/doc/developer-manual.html#_spectrum_vm [3]: https://spectrum-os.org/git/www/commit/?id=bf7cde0ddd06455f212fb39092b87313… [4]: https://spectrum-os.org/todo.html crosvm ------ My main goal for this week was to start modifying libwayland-server to use a virtio_wl socket as the display socket, as described last week[5]. So, in pursuit of this goal, I started the week by trying to write a test program that would start a VM running a compositor, create a socket, attach one end to that VM, and then start another VM running a Wayland client, and attach the other end of the socket to that VM. Then, in theory, all I had to do was the libwayland-server modifications, and then the compositor and client would be able to talk to each other. Unfortunately, I didn't get that far. The program froze after starting the first VM. I assumed I must have done something wrong, so I rewrote it, and had the same problem. I did it again, and same problem. Some further debugging revealed that the command to add the socket to the first VM was hanging. And also, crosvm was hanging. Running the command outside this program worked fine, though. It returned immediately, and crosvm kept running with the new socket added. What I eventually realised was that the crosvm hang only happened if the command was sent to it early enough. After some intense staring at the crosvm code, I realised that there was a deadlock resulting from trying to wait for a response from a virtual device before the device had been initialized. Despite this issue manifesting with adding Wayland sockets, which is a feature unique to crosvm, a lot of the code for that is adapted from the code for resizing virtual block devices, which is an upstream feature. And that had the same issue, so this is actually an upstream bug. The fix was not easy, and required some fairly substantial refactoring. It's described in detail in the patch message[6]. I think it's probably a pretty interesting read. I also merged in the latest upstream crosvm changes. [5]: https://spectrum-os.org/lists/archives/spectrum-discuss/87tuzm287b.fsf@alys… [6]: https://spectrum-os.org/lists/archives/spectrum-devel/20200614114344.22642-… Nixpkgs ------- I spent a few more hours trying to get libvda (an optional dependency of crosvm; required for virtio-video) to build, and I got it to work, thanks to some help from Puck. There's still a hack in there I'll tidy up before I push it, but it's cool to have this packaged for future experimentation with this crosvm feature. With the crosvm issue out of the way, which ended up taking most of the week to identify and fix, hopefully next week I'll be able to actually get going on those libwayland-server changes.

1 0

High level design and other related projects
by infokiller . 14 Jun '20

14 Jun '20

Apologies for the incoherent title: I'd like to better understand the design choices of this project and discuss how it relates to other projects in this space. First I'd like to say that I think that using crosvm is a really great decision. Google has a lot of manpower working on ChromeOS and Android, and building on their work is something that should pay off, especially for a project such as Spectrum that tackles such a huge undertaking (building a secure OS). Here's a few questions to kick off the discussion: - Have you considered using a micro kernel based host like seL4, similar to what Genode does (at least as I understand it)? - Have you considered gVisor [1] for lightweight compartmentalization? - Have you considered reusing stuff from the Whonix project? ## References [1] https://github.com/google/gvisor

3 2

This Week in Spectrum, 2020-W23
by Alyssa Ross 07 Jun '20

07 Jun '20

I wasn't quite back up to normal speed after being sick last week, I think, but we're getting somewhere and I did some cool stuff! Infrastructure -------------- Upgraded Mailman. Most significantly, the Hyperkitty UI has some nice improvements, which I think will make it easier to navigate. :) Nixpkgs ------- I integrated and committed last week's wlroots patch (to make it use virtio_wl-compatible memfds). Spent quite a lot the start of the week working on packaging more Chromium OS components to build some optional crosvm features. They just added support for virtio-video[1], which sounds pretty cool. I managed to get libchrome working again, which I previously had to remove because I couldn't get it to build! I haven't committed any of this yet, because I haven't got to the point where it's useful for building optional crosvm features, and I don't want to add packages that aren't useful for anything, but I did make quite a lot of progress. After a couple of days of this, though, I put it aside to focus on other things, since it would be useful to have, but its not a priority right now. [1]: https://chromium.googlesource.com/chromiumos/platform/crosvm/+/57df6a0ab23c… crosvm ------ I posted a patch[2] to make listing a directory shared with --shared-dir not crash crosvm when sandboxing was enabled. I'd been reluctant to fix this, because --shared-dir (which exposes a host file system to a crosvm guest) is not going to be useful in Spectrum, where shared directories should come from another VM. But it's useful enough in development that not fixing it was more frustrating than just fixing it. I spent a lot of time this week thinking about Wayland connections. My original plan for inter-VM communication was to make it possible to run crosvm devices as standalone programs inside VMs. I still think this is the right track for most virtio devices, but for Wayland it's starting to feel less and less like it makes sense, because the Wayland device is basically a wrapper around a socket, and both ends will be hooked up to sockets. So what I now plan is that a VM running a Wayland client application will connect to a host socket, with the other end connected to a VM running the compositor. The client applications will be run under Sommelier, and so will be able to run unmodified. The compositor will be modified to use a virtio_wl socket. A slight complication here is that a Wayland compositor usually accepts on a socket to receive client connections, but you can't accept on a virtio_wl context. So instead, what we can do is emulate accept by attaching a new virtio_wl socket, and then sending the name of that socket over the main socket. This can be wrapped inside the VM to look very much like an accept. Unfortunately, crosvm virtio_wl sockets could only be set at startup time. So today I fixed that[3]. Adding support for adding virtio_wl sockets at runtime was pleasantly straightforward. I feel very energized now that I managed to bang that out in a day. :D [2]: https://spectrum-os.org/lists/archives/spectrum-devel/20200605234757.28848-… [3]: https://spectrum-os.org/git/crosvm/commit/?h=interguest This is one of those weeks where I can feel writing this that it might not make much sense to anybody else. But the important thing with these weekly updates is that I write _something_ we can refer back to in future, and communicate that things are happening. If you'd like to better understand what I'm talking about here, I'm happy to try walking through it on IRC. Next week, I'm going to work on getting libwayland-server to speak this weird virtio_wl faux-accept.

1 0

This Week in Spectrum, 2020-W22
by Alyssa Ross 30 May '20

30 May '20

I began this week feeling very burnt out from some Nix stuff. Later in the week, I got sick as well (thankfully not with any COVID-19 symptoms) and ended up spending several days not being able to do much beyond sleep. So, not a lot from me to report this week I'm afraid. This'll be a short one. Documentation ------------- I documented the memfd server I recently added to crosvm[1]. I sent the documentation patch to the list, and was very pleased to receive reviews[2] from impaqt and Cole Helbling. This is really the first time a patch from me has gone through an actual review cycle with other people involved, so that's very exciting. Here's to more of that! :) With that documentation, I was then able to add a memfd server quota to the Contribution Ideas page[3]. This is an important feature, so if nobody is interested in working on it I'll do it myself, but it's also a nice opportunity to do a pretty self-contained bit of work on crosvm, so I'm going to leave it up for a bit in case somebody else wants to try it. [1]: https://spectrum-os.org/doc/developer-manual.html#_the_memfd_server [2]: https://spectrum-os.org/lists/archives/spectrum-devel/C312TALEYVQS.RICE3VLE… [3]: https://spectrum-os.org/todo.html Infrastructure -------------- Increased the maximum message size on devel@ to 80K from 40K, because a patch I sent was too big and got held for moderation. Nixpkgs ------- The expected Chromium OS update was released this week (instead of last as I'd expected). crosvm now wants the minijail source code, which isn't hosted on chromium.googlesource.com. This required making the update script a bit smarter. I posted the patch series improving the update script and then updating chromiumOSPackages to the list[1]. More details there. Another shout out to Cole for having a go at doing this update, even if he didn't get all the way because he needed the cores on his computer for other things. Thanks Cole! [1]: https://spectrum-os.org/lists/archives/spectrum-devel/20200530190028.6388-1… I'm mostly feeling better now (although still getting tired very easily), so hopefully things will go faster again next week. Too early to be certain, though. I'm happy to have at least got the chromiumOSPackages update out the way, and delighted to have received the reviews on the documentation patch.

1 0

This Week in Spectrum, 2020-W21
by Alyssa Ross 24 May '20

24 May '20

A week of results! Infrastructure -------------- Fixed a misconfigured spam filter that allowed an obvious spam message through to devel@. Oops. crosvm ------ Integrated the memfd server[1] on the interguest branch. It's now all sandboxed, and optionally enabled with a command line argument to crosvm run. Not all that much to say here, but it's what took me most of the week! Getting the sandbox working was a bit weird. When I tried to get it to log seccomp failures, it seemed to just disable the sandbox. I had to track them down with strace instead. Annoying. But the sandbox does work in normal operation. I still haven't limited how much memory can be requested this way. I think implementing that would be relatively straightforward for another contributor, so I think I'll add it to the ideas list[2] and see if a patch is forthcoming. Otherwise I'll do it myself. [1]: https://spectrum-os.org/git/crosvm/commit/?h=interguest [2]: https://spectrum-os.org/todo.html wlroots ------- I took my standalone virtio_wl test program, and integrated it into wlroots' allocate_shm_file function. This has the result that, when running under Sommelier, this patched wlroots will request shared memory from the host, rather than allocating it itself. Porting from the standalone test program was nice, because it meant that this all just worked, first try! (Once I got it to compile under Nixpkgs' or wlroots' strict compiler errors, at least.) This will allow that memory to be sent between VMs! I haven't pushed the patch yet because I haven't integrated it into Spectrum's Nixpkgs yet. I plan to do that next week. I'm starting to think about moving the stuff specific to Spectrum VMs into an overlay, but I need to think a bit about how to structure that. Nixpkgs ------- There's no sign of the expected Chromium OS release so far, so I backported[3] support for multiple virtio_wl sockets from a more recent Chromium OS kernel to the one in Spectrum's Nixpkgs. We need this to be able to dedicate a named socket to the memfd server. [3]: https://spectrum-os.org/git/nixpkgs/commit/?id=f24d310275909265de32cbc831d5… It's been another week where I've been very focused on one task. I'm quite excited about the direction this is all going. It's looking like we'll be able to do almost everything inside VMs, which means it might be possible to have a host kernel that does almost nothing apart from KVM and PCI passthrough?? This would mean we'd end up with a tiny Linux a little bit (but not all that much) like a microkernel, with most hardware interaction and all user programs running in VMs. Cool stuff! It's not clear to me yet the exact extent to which this is achievable, but it's a nice vision to keep in mind. It might also make it easier for us to transition to a true microkernel at some point in the future. I'm hoping that I'll hit an NLnet milestone related to this stuff fairly soon. Until I do, I'm now living on the money I've received in the past six months through GitHub Sponsors. Thank you so much to everyone who is helping to make it possible for me to spend this time on the fundamentals so we have a good foundation to build Spectrum on. <3

1 0

This Week in Spectrum, 2020-W20
by Alyssa Ross 17 May '20

17 May '20

It feels like things are finally starting to come together. Infrastructure -------------- Server memory upgraded. We should now be able to handle more than one person cloning Spectrum's nixpkgs repo at once! Documentation ------------- I added a much requested page[1] to the website with a few tasks a potential contributor might want to look into. More to come here, too! [1]: https://spectrum-os.org/todo.html virtio_wl --------- I wrote a simple guest C program that connected to a host socket over virtio_wl, and sent some messages back and forth. In doing this, I've ended up with a nice set of generic functions for interacting with virtio_wl -- virtio_wl_connect(), virtio_wl_send(), etc. This is nice, because it allows me to treat virtio_wl inside a guest much like a regular socket (albeit one that can only be interacted with using a custom set of functions). Otherwise, I'd have to do ioctl operations every time (which seems to be what the various virtio_wl-using Chromium OS components do). Having a simple test program (rather than integrating directly into wlroots) gives me a more simple environment with which to test the other end of this connection. crosvm ------ I implemented a little Rust server that listens on a Unix stream socket. When it receives a message (consisting of an identifier {for debug purposes} and a size), it will allocate a memfd of the requested size, and then send that back to the socket, along with a status byte indicating success or failure. I then integrated this program into crosvm, so it is started as part of the VM initialization, and exposed over virtio_wl as an additional socket. Integrating this into crosvm required some careful thought, because there's no precedent in crosvm for a virtio_wl server of this sort. It has almost nothing with the existing concept in crosvm of a "device", which is something that communicates with a guest kernel driver over PCI. So Spectrum's crosvm now has the concept of "servers". I can see us wanting to provide other VM services over virtio_wl in the future. It allows us to exchange various resources between host and guest (or between guests) without having to write a kernel driver specific to each task. This all now works well enough that my program inside the VM can request a chunk of memory, and receive back something that looks a bit like a memfd, that can then be sent back to the host over a virtio_wl socket. It was great to see this working the first time! One thing crosvm devices do that I would also like to apply to this type of server is sandboxing. The sandboxing mechanism used in crosvm is pretty generic (it's really just a thin wrapper around Minijail[2]), so it should be straightforward to reuse. I just have to understand a bit better how to use it. The other crosvm change I still need to make is to put this functionality behind a command line flag. Most VMs should not be able to give themselves extra memory this way -- really only the Wayland compositor should be using it. I should also impose some upper bound on how much memory is allowed to be allocated this way. Such a limit will probably get harder to add (from a testing point of view) the longer I wait. [2]: https://lwn.net/Articles/700557/ So, at lot of very focused progress this week. From here I plan to finish up the crosvm stuff, and then integrate this protocol into wlroots. Then, not too long from now, we'll finally be able to move the Wayland compositor into its own VM, separate from applications. Exciting! There will also probably be a Chromium OS release next week, which will require some updates in Spectrum. If all goes well, I won't actually be the one doing that this time. ;)

1 0

This Week in Spectrum, 2020-W19
by Alyssa Ross 10 May '20

10 May '20

This week, I want to give a special shoutout to Cole Helbling, who sent three patches to improve our documentation. Non-Alyssa Spectrum patches are quite rare at the moment, so it was very very nice to get these all at once. Thanks Cole! This week has mostly been tidying up loose ends and working on documentation. But there's some good, concrete progress as well. Website ------- <https://spectrum-os.org/> now links directly to my blog and This Week in Spectrum, so that people who come to the website can more easily see that development is ongoing and progress is being made[1]. I've also made it a little more obvious how to find the IRC channel and mailing lists from the homepage[2], because a conversation on the fediverse made me realise that wasn't clear at all before. Cole fixed the advice in the website's CONTRIBUTING file on sending patches for the website[3]. I had written to set Git's "sendemail.subjectPrefix" configuration option, but the option is actually called "format.subjectPrefix". Cole fixed a broken link[4], and I made the incorrect URL redirect to the right one as well, just in case. [1]: https://spectrum-os.org/git/www/commit/?id=8f37213cfd4d5dbd4a982bfc1899a3b3… [2]: https://spectrum-os.org/git/www/commit/?id=6d1ce779016cd84c0413b225355715e5… [3]: https://spectrum-os.org/git/www/commit/?id=8f37213cfd4d5dbd4a982bfc1899a3b3… [4]: https://spectrum-os.org/git/www/commit/?id=13c860e1e5bb02fce683012b851c4451… Developer manual ---------------- Did you know there's a developer manual[5] for Spectrum? There's not a lot in there yet, so for now it might not be all that useful, but it will be one day! I added a section about how to hack on Spectrum's crosvm, because it can be a bit non-obvious how to get up and running with it. I got some great feedback on it on IRC, and a diff from Cole with some fixes. Thanks to everybody who gave me feedback and helped get that documentation into shape. Setting up a crosvm build still isn't all that easy, but at least it's documented. [5]: https://spectrum-os.org/doc/developer-manual.html Infrastructure -------------- Two(?) people trying to clone all of Nixpkgs from the tiny VPS running spectrum-os.org caused the server to run out of memory and return 504 to those clones. It's quite important that people are able to get Spectrum's code, so I've arranged for the VPS to be upgraded with more memory, and so now I just need to reboot the machine so it picks it up. I plan on doing this tomorrow, since there's less likely to be mailing list traffic then than there is immediately after I send this. In the medium term I'll host all this stuff on a more sensible machine, because it's quite clear the workload on this server (which also runs some other stuff) is starting to get beyond the "tiny VPS" territory. IRC --- Activity in #spectrum has really picked up all of a sudden. Hundreds of messages were sent in the second half of this week, mostly talking either about Spectrum or about the future of secure computing in general, and I'm extremely here for all of it. The best thing is that I wasn't even around for most of these conversations! It's hugely motivating to me to feel that people are so interested in Spectrum, and it's great to have people with all sorts of useful knowledge and questions around. If you weren't there, I recommend actually checking out the logs from this period[6], because the conversations being had were just a great read full of interesting stuff. #spectrum feels like it's now at the point where you can start a discussion about secure computing, and be confident that you'll get at least a couple of extremely knowledgeable people involved in the conversation, and that's just fantastic. [6]: https://logs.spectrum-os.org/spectrum/2020-05-06 Wayfire ------- I replaced Sway in the test VM derivation with our newly packaged Wayfire. I was worried at first because wf-shell wasn't starting properly, but strace helped me realise that wf-shell likes HOME to be set, and I hadn't actually done that in the VMs yet. So now that works. There hasn't been much movement on my Wayfire PRs, or my PR to add Wayfire to upstream Nixpkgs, and that's down to me. I felt like after working on it all week last week, I needed a bit of a break. I'll try to get back to it next week so all those PRs can be gotten over the line. crosvm ------ I merged the latest crosvm changes from Google into Spectrum's crosvm tree. Doing this at the moment takes quite a while, because my work-in-progress "interguest" branch has about 50 commits that have to be rebased onto the new master, and quite a lot of those usually result in conflicts. This won't be as much of a problem once that code is ready to be included in master, because updating then will just be a single merge commit, but for now it's a bit of a pain, especially since I try to make sure every commit works. I haven't had much of a look over what's changed yet. There's usually at least one cool new toy to be excited about when I merge Google's crosvm, though, so I look forward to finding out what it is this time. I think that's it for this week, although honestly there have been so many different things going on I've probably missed something. Hopefully next week will be a bit more focused and I'll make more progress with interguest communication. This week, though, we've made some great improvements in several important areas. I'm optimistic that the increased IRC and patch volume are a sign of things to come. I'm looking forward to seeing where things will go from here. :)

1 1

This week in Spectrum, 2020-W18
by Alyssa Ross 03 May '20

03 May '20

A belated Happy International Workers' Day. :) I had some other stuff going on this week but I'm still happy with what I got done. Wayfire ------- I've been working on packaging Wayfire in Nixpkgs so we can use it in Spectrum. (See last week's update for why I've chosen Wayfire for this.) This has required quite a bit of work on Wayfire itself, because up until now Wayfire has assumed that every component, including plugins, will be installed into Wayfire's prefix. With Nix, a package's prefix is only writable by that package, so we need to make some changes to Wayfire to support each component having its own prefix. This is necessary even for our simple case because even a basic Wayfire setup has at least one plugin to provide the desktop shell. Everything is supposed to be a plugin, after all. And we'll want to add our own plugins at some point. To implement plugins in other search paths, I added two environment variables WAYFIRE_PLUGIN_PATH (for libraries) and WAYFIRE_PLUGIN_XML_PATH (for the XML files that define Wayfire plugin options). These are then searched in addition to Wayfire's prefix. Support for plugins coming from multiple prefixes required some small changes in several parts of the Wayfire ecosystem. Wayfire has so far proven to be a very friendly upstream. They understood the problem I was having, even though it's less of an issue on traditional distributions, and I always got quick answers on IRC when I needed to ask for advice on implementing the search paths. Here's the issue I filed describing the need for search paths: https://github.com/WayfireWM/wayfire/issues/493 And here are all the pull requests I made to implement them across the ecosystem: https://github.com/WayfireWM/wayfire/pull/496 https://github.com/WayfireWM/wf-config/pull/25 https://github.com/WayfireWM/wayfire/pull/497 https://github.com/WayfireWM/wf-shell/pull/52 https://github.com/WayfireWM/wf-shell/pull/53 https://github.com/WayfireWM/wcm/pull/17 https://github.com/WayfireWM/wf-shell/pull/54 https://github.com/WayfireWM/wcm/pull/18 With these changes, I can have Wayfire all nicely packaged up. We can even provide a nice Nix interface for configuring a Wayfire with some specific plugins: (wayfireApplications.withPlugins (plugins: with plugins; [ wf-shell ])).wayfire I made a pull request to add Wayfire, with the interface demonstrated above, to Nixpkgs: https://github.com/NixOS/nixpkgs/pull/86606 Unbeknownst to me, while I was working on this, another PR was also opened to add the latest version of Wayfire to Nixpkgs: https://github.com/NixOS/nixpkgs/pull/86569 This PR does a good job of making the basic Wayfire installation work, but it doesn't support extra plugins. Still, it's likely that if this had been available before I started my work, it would have been good enough for us for now. This sort of thing is always a risk, and even if I hadn't done the Wayfire work now I'd have had to come back later anyway to do plugins, so it's not too bad. The author of that PR and I have agreed to work together from here to get Wayfire into Nixpkgs, using my PR as a base because it has the plugins support. So, hopefully by the time I write my next status update all my Wayfire pull requests will have been merged, and Wayfire will be in Nixpkgs. Reviews on the Nixpkgs PR in particular are very much appreciated. :) From here, I'll start trying to modify Wayfire/wlroots to make it request host memory rather than allocating its own, as described in last week's status update. I had a quick look, and it appears that there aren't many places these allocations are happening, so hopefully that shouldn't require too much modification. Then, we should be able to run an application in one VM, displayed on a compositor in another. :)

1 0

This Week in Spectrum, 2020-W17
by Alyssa Ross 26 Apr '20

26 Apr '20

It's been a bit of a slow week, I'm afraid. spectrum-vm ----------- As mentioned last week, it was becoming extremely inconvenient to test out custom kernel/crosvm/rootfs builds by modifying a Nix expression every time. So I wrote spectrum-vm, a small program that takes command line arguments for kernel, crosvm, and rootfs paths to use, and falls back to pre-built defaults if any of those aren't specified. This makes turnaround time on testing different kernel builds much quicker, because they can be built incrementally outside of nix sandboxes. To make things more concrete, suppose I have built a kernel in /home/src/linux/arch/x86_64/boot/bzImage, and I want to test it with the Spectrum crosvm demo setup. I can run spectrum-vm -k /home/src/linux/arch/x86_64/boot/bzImage and I get a VM booted from that kernel, with no need to nix-build anything. By default, the launched VM will run Weston Terminal inside a Wayland compositor. But you can even replace this with a custom command: spectrum-vm -c 'echo "Hello, world!"' The implementation of that last bit is quite the hack. I'll leave figuring out how it works as an exercise to the reader. ;) spectrum-vm is available now in Spectrum's Nixpkgs as spectrumPackages.spectrum-vm, as of commit 817a1a9b0de30d8bbc2ad2e80a802d87c56e96ff. Implementing this has meant that the huge start-vm.nix file that used to exist on the crosvm-demo branch had to be split up into a bunch of discreet units. This is really great, because start-vm.nix was starting to grow enormous, and experimenting with changes was becoming A Problem. Now, that's all nice and modular under the new spectrumPackages namespace. And _that_ means that the crosvm-demo branch can finally go away, and Spectrum's Nixpkgs is no longer split across two branches. (The purpose of the separation was to avoid there being a horrible start-vm.nix file in the root on master.) So in addition to a lovely helpful tool, this has also resulted in much cleaner code. Yay! One feature I would still like to add is an option that maps to crosvm run's --shared-dir, to mount a directory into a VM over 9p. That would make getting code into a VM to test it extremely convenient. I haven't gotten around to it yet though. Guest memory sharing -------------------- Last week I shared a bunch of research I had done on sharing guest-allocated buffers with other VMs. I was hoping to avoid having to patch a Wayland compositor to request buffers from the host instead of allocating them itself, but I've since come to the conclusion that avoiding that won't be feasible, for now at least. Basically, the kernel primitives aren't there yet, and it would be silly for me, with almost zero kernel development experience, to attempt to write them. There are all sorts of horrible edge cases to think about, like what happens if the guest-allocated memory moves around. Compared to this, patching a Wayland compositor doesn't seem too bad. Having to go through a request/response over a vsock to the host will be a bit of a performance overhead compared to a smarter solution, but I think this kind of allocation happens rarely enough that it probably doesn't matter anyway. A nice middle ground would have been to use virtio-gpu resources rather than a custom message over a vsock. It would have been a much cleaner way to do it, and it might not even involve writing any kernel code, but the practical benefits of it would be extremely limited when contrasted with the time it would take me to implement that. I don't expect the approach I'm taking now of patching the Wayland compositor to necessarily be permanent. As I linked to last week, there are kernel efforts happening to add the necessary primitives to share guest allocations. At some point in the future we might be able to use them. And at basically any time we could switch to using virtio-gpu resources if somebody wanted to put in the work. Patching a compositor really isn't a big deal, though. It's unlikely Spectrum will support more than one compositor anyway, because we'll need to customise it to do things like Qubes-style window borders that are coloured based on security domain. And, even then, if we're using a wlroots-based compositor, it's probably only wlroots that needs patching, so in that case any other wlroots-based compositor would probably work. Speaking of compositors... Wayfire ------- If you know what Wayfire is, you're probably surprised to see it as an item in a Spectrum status update it. If you don't, Wayfire is a Wayland compositor notable for having a bunch of over-the-top Compiz-style graphical effects, like wobbly windows and the titular fire. I recommend checking out the demo videos on https://wayfire.org/ if you haven't seen them -- it's quite something. But I bring this up not because I think support for wobbly windows is imperative to Spectrum's chances of success, but because fundamentally Wayfire is just an extremely modular Wayland compositor. Virtually everything in Wayfire is implemented as a plugin. Switching between windows is a plugin, screen zoom is a plugin, and so on. This is really cool because in Spectrum we'll want to have, for example, coloured window decorations like in Qubes. With any other compositor, we'd have to hack that into the compositor internals somehow. With Wayfire, it can just be a plugin. Wayfire is also a pleasantly simple but featureful windowing experience. You can drag windows around, put two side by side by dragging them to screen edges, have multiple workspaces, etc. I think it will be straightforward enough to not be a problem for people coming to Spectrum regardless of their computing background, and powerful enough to not get in the way of Serious Computer Business. And anything missing could of course be implemented as a plugin. ;) So, I've been doing some work to package Wayfire in Nixpkgs. This is not all that easy, because Wayfire seems to assume that plugins will be installed into Wayfire's prefix. This isn't really possible with Nix unless you compile all the plugins along with Wayfire, so I'm working some patches for Wayfire to look up plugins in a search path specified as an environment variable, which on the Nix side we can then wrap in the idiomatic wayfireWithPackages style. My initial impressions of the Wayfire codebase are that it is relatively clean, and straightforward enough to understand. The core compositor is about 13000 SLoC, which is pleasantly small. That, unfortunately, is it for this week. I didn't get as much done as I'd hoped, but I did at least have loads of good thoughts about things to do later in Spectrum's development, that aren't quite concrete enough to put into words yet.

1 0