Hi! I've had a busy day today and am pretty tired, so I'm not sure how coherent my writing is at the moment. But I'd rather get this out on time, especially since tomorrow is also busy today.
As I said last week, I took some time off this week as a preventative measure against burnout.
One of my patches was accepted, but another is still waiting. As I said last week, I have more fixes for rust-vmm planned, but want to let them catch up with the changes I've already sent upstream first so I know what base I'm using for the next stuff. I expect the one that's still waiting to be accepted next week.
: https://github.com/rust-vmm/vmm-sys-util/pull/135 : https://github.com/rust-vmm/vhost/pull/69
Last week, I'd just integrated dm-verity into the Spectrum live image I've been working on. When it came time to work on the actual root filesystem, instead of the initramfs, I hit a bit of a brick wall. I realised that trying to generate a whole operating system image using Nix was giving me real writers' block. There was too much inbetween me and how the files ended up on disk, and that meant there was too much overhead to keep in mind when I was thinking about how things should be designed and laid out. It might feel like making a Linux root filesystem should be a solved problem, but Spectrum has a bunch of special requirements. You might want to do something like start a VM for each hardware device of a specific type, and that's something that isn't really addressed by most standard stuff. All this stuff is definitely solveable, but it requires some experimentation to get right, and Nix was getting a bit in the way of that.
So I created a new directory, and I wrote a Makefile that builds an ext4 image, and I just started putting files in an etc/ directory. This made reasoning about the system way easier, and I was immediately making progress.
Nix is great for building known targets, and for making customisable systems (there's no way my Makefile-based system would allow the amount of customisation I'd easily be able to provide with Nix), but for experimentation, it's a lot nicer to be closer to the end product. So once I know how all this should look I'll make it Nix-aware.
Currently, I have a root filesystem with a service manager that can respond to hardware appearing and disappearing. The next (more exciting) step will be to have it start some VMs, and assign hardware to them appropriately. I'm looking forward to getting to that next week. An interesting challenge I'll have to solve will be figuring out simple categories (e.g. "ethernet device") from the huge amount of very specific information the kernel provides. I think I might be able abuse the modules.alias file from the kernel, that defines the mappings from PCI etc. information to default drivers. Then all I'll have to do will be to write mappings from default drivers to whatever categories I come up with, or what VM I want to assign them to, or whatever.
One neat thing I'm using for the first time here is tar2ext4, a utility program that's part of a larger Microsoft open source project I don't entirely understand the purpose of. It's really useful for me it builds ext4 images entirely in userspace, which will be great for using in Nix derivations where it's not possible to just mount an ext4 image and write directly to it. For previous Spectrum experiments, I'd always used SquashFS, entirely because I already knew of a tar2sqfs program that made creating filesystem images really easy. I've added tar2ext4 to Nixpkgs, which will hopefully help other people who have similar problems discover it.
: https://github.com/microsoft/hcsshim/blob/master/cmd/tar2ext4/tar2ext4.go : https://en.wikipedia.org/wiki/SquashFS : https://github.com/AgentD/squashfs-tools-ng : https://github.com/NixOS/nixpkgs/pull/134650
While I was working on the root filesystem, I noticed that mount -a, which mounts all filesystems described in fstab(5), wasn't working. This turned out to be because of a bug in Musl's implementation of getmntent(3), a libc function for parsing fstab files.
So I wrote some tests and a fix and sent them to Musl.
Hope that all made sense.
Next week, I'll continue working on Spectrum live, and maybe fix the next rust-vmm issue I have on my todo list if my final outstanding PR is merged.