April 2022 - Spectrum Development

[PATCH] host/rootfs: fix Weston hotplugging
by Alyssa Ross 11 May '22

11 May '22

On the Spectrum host, the kernel delivers uevents to mdevd. By default, if mdevd doesn't have a handler for an event, it won't do anything, and other processes that might be interested in the uevent hear about it. This means that Weston isn't notified about input device hotplugs. (And USB devices can appear after init has started, so any USB device counts as hotplugging even if it's attached before boot or even built in to the computer.) To fix this, we configure mdevd to rebroadcast events it doesn't handle on Netlink group 4. These events will then be picked up by Weston via libudev-zero, which listens on group 4 for hotplug events. Reported-by: Ville Ilvonen <ville.ilvonen(a)unikie.com> Fixes: 372bacd ("etc/s6-rc: add weston service") --- Ville, does this fix the internal keyboard and mouse on your ThinkPad T560? host/rootfs/etc/s6-rc/mdevd/run | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/host/rootfs/etc/s6-rc/mdevd/run b/host/rootfs/etc/s6-rc/mdevd/run index f838d2d..c4615c8 100644 --- a/host/rootfs/etc/s6-rc/mdevd/run +++ b/host/rootfs/etc/s6-rc/mdevd/run @@ -2,4 +2,4 @@ # SPDX-License-Identifier: EUPL-1.2 # SPDX-FileCopyrightText: 2020-2022 Alyssa Ross <hi(a)alyssa.is> -mdevd -D3 -b134217728 +mdevd -D3 -O4 -b134217728 base-commit: 402c7fd0637e834f97b0ea1a2abb89c417e7469d -- 2.35.1

3 2

[PATCH] host: use a bigger kernel buffer for mdevd
by Alyssa Ross 23 Apr '22

23 Apr '22

This should fix a boot error on a Thinkpad P51s: > mdevd: fatal: unable to receive netlink message: No buffer space available From the mdevd documentation: > • -b kbufsz : try and reserve a kernel buffer of kbufsz bytes for > the netlink queue. Too large a buffer wastes kernel memory; too > small a buffer risks losing events. The default is 500 kB, which > should be enough for most systems; if you're getting "No buffer > space available" errors from mdevd at coldplug time, try > increasing this number. The new value of 128MiB is used by default by both udev and Busybox's mdev. --- I'm hoping this will fix the problem reported by terwiz in #spectrum. host/initramfs/etc/init | 2 +- host/rootfs/etc/s6-rc/mdevd/run | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/host/initramfs/etc/init b/host/initramfs/etc/init index bb58c84..e6236bc 100755 --- a/host/initramfs/etc/init +++ b/host/initramfs/etc/init @@ -19,7 +19,7 @@ if { mkfifo /dev/rootfs.poll } background { fdclose 3 - mdevd -C + mdevd -C -b134217728 } importas -iu mdevd_pid ! diff --git a/host/rootfs/etc/s6-rc/mdevd/run b/host/rootfs/etc/s6-rc/mdevd/run index 03ad5e3..22f0127 100644 --- a/host/rootfs/etc/s6-rc/mdevd/run +++ b/host/rootfs/etc/s6-rc/mdevd/run @@ -2,4 +2,4 @@ # SPDX-License-Identifier: EUPL-1.2 # SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi(a)alyssa.is> -mdevd -D3 +mdevd -D3 -b134217728 base-commit: 92e918a477dc7a16460a73d466fc2d42f7e7c421 -- 2.35.1

2 1

[PATCH] Documentation: document the binary cache
by Alyssa Ross 22 Apr '22

22 Apr '22

--- Documentation/Makefile | 1 + Documentation/binary-cache.adoc | 53 +++++++++++++++++++++++++++++ Documentation/getting-spectrum.adoc | 4 +++ 3 files changed, 58 insertions(+) create mode 100644 Documentation/binary-cache.adoc diff --git a/Documentation/Makefile b/Documentation/Makefile index 59bc3ae..e9d5886 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -8,6 +8,7 @@ datarootdir = $(prefix)/share docdir = $(datarootdir)/doc/spectrum SOURCES = \ + binary-cache.adoc \ creating-vms.adoc \ development.adoc \ getting-spectrum.adoc \ diff --git a/Documentation/binary-cache.adoc b/Documentation/binary-cache.adoc new file mode 100644 index 0000000..5ce6a4d --- /dev/null +++ b/Documentation/binary-cache.adoc @@ -0,0 +1,53 @@ +// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> +// SPDX-License-Identifier: GFDL-1.3-or-later OR CC-BY-SA-4.0 + += Binary cache + +Building Spectrum from source can take a very long time. To avoid +having to wait when building the system to try it out or test patches, +an x86_64 binary cache service is available. If configured to do so, +Nix will download build outputs from the cache, instead of building +them locally. + +Builds are run on the latest NixOS unstable on +https://www.vultr.com/[Vultr], and are signed to ensure authenticity. +The build infrastructure and signing key are accessible by Alyssa Ross +and Puck Meerburg. Storage is provided by Daniel Kuehn. Please +https://spectrum-os.org/participating.html[get in touch] if you +encounter any trouble with it. + +The binary cache is currently not able to provide logs, due to a +https://github.com/NixOS/nix/issues/6403[Nix bug]. + +== Setting up the binary cache + +=== NixOS + +The following configuration adds the Spectrum binary cache as a +substituter, and tells Nix to trust builds signed with its public key. + +[source,nix] +---- +{ ... }: + +{ + nix.settings.substituters = [ "https://cache.dataaturservice.se/spectrum/" ]; + nix.settings.trusted-public-keys = [ + "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" + "spectrum-os.org-1:rnnSumz3+Dbs5uewPlwZSTP0k3g/5SRG4hD7Wbr9YuQ=" + ]; +} +---- + +=== Non-NixOS systems + +Add the following configuration to /etc/nix/nix.conf: + +---- +substituters = https://cache.dataaturservice.se/spectrum/ https://cache.nixos.org/ +trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= spectrum-os.org-1:rnnSumz3+Dbs5uewPlwZSTP0k3g/5SRG4hD7Wbr9YuQ= +---- + +If you have other binary caches configured besides cache.nixos.org, +make sure to add the Spectrum cache URL and public key to the existing +configuration lines in your nix.conf file. diff --git a/Documentation/getting-spectrum.adoc b/Documentation/getting-spectrum.adoc index 4ff6b05..fb69dce 100644 --- a/Documentation/getting-spectrum.adoc +++ b/Documentation/getting-spectrum.adoc @@ -10,6 +10,10 @@ from other operating systems, as long as they're not too invasive.] and have https://nixos.org/[Nix] and https://git-scm.com/[git] installed, it's very easy to do. +Before you start, consider setting up the +xref:binary-cache.adoc[Spectrum binary cache], which will save you a +lot of time waiting for builds. + == Trying Spectrum If you want to try Spectrum out to get a feel for it, without -- 2.35.1

2 1

[PATCH 0/2] img/combined: avoid chainloading in GRUB
by Alyssa Ross 21 Apr '22

21 Apr '22

Ville, could you give these patches a try on your P43s and see if "Try Spectrum" can boot now? Alyssa Ross (2): img/live: switch to systemd-boot img/combined: avoid chainloading in GRUB img/combined/default.nix | 10 ++++++++++ img/combined/grub.cfg.in | 5 +++-- img/live/Makefile | 23 +++++++++++------------ img/live/default.nix | 2 +- img/live/etc/os-release | 5 ----- 5 files changed, 25 insertions(+), 20 deletions(-) delete mode 100644 img/live/etc/os-release base-commit: 82212bb2207b9cb43f4510f373fb9ba8acfde21e -- 2.35.1

3 5

Binary cache
by Alyssa Ross 14 Apr '22

14 Apr '22

I'm happy to report that we finally have a Spectrum binary cache set up, which mitigate the very long build times that have been frustrating people attempting to try out Spectrum for themselves. URL: https://cache.dataaturservice.se/spectrum/ Signing key: spectrum-os.org-1:rnnSumz3+Dbs5uewPlwZSTP0k3g/5SRG4hD7Wbr9YuQ= The build infrastructure is hosted on Vultr, and is accessible by me and Puck. Storage is provided by Daniel Kuehn (lejonet). We don't have logs in the cache yet, due to a Nix bug[1]. My aim with this service is to provide low-cost and easy-to-run convenience for people interested in trying out Spectrum early, so don't assume this architecture is reflective of that of any future, higher stakes project infrastructure. Thanks to everybody who's helped me get this up and running over the last couple of weeks. I'll write some documentation about how to use it and link to it from the "Getting Spectrum" docs, and publish the configuration etc. for the build infrastructure soon but I've been pushing myself very hard to get this done in the first place, so I need a break from it first. In future I'd also like to explore possibilities for additional mirrors of the binary cache (lots of people have offered me storage hosting over the years!), but there are some additional technical challenges to solve before I can commit to that. Alyssa [1]: https://github.com/NixOS/nix/issues/6403

1 0

[PATCH] Don't assume /var/empty exists in Makefiles
by Alyssa Ross 10 Apr '22

10 Apr '22

It apparently doesn't on Ubuntu, and this way we also don't have to hack around it being missing in the Nix sandbox. Reported-by: Ville Ilvonen <ville.ilvonen(a)unikie.com> --- host/rootfs/Makefile | 9 ++++++--- host/rootfs/default.nix | 5 ----- vm/app/catgirl/Makefile | 9 ++++++--- vm/app/catgirl/default.nix | 5 ----- vm/app/lynx/Makefile | 9 ++++++--- vm/app/lynx/default.nix | 5 ----- vm/sys/net/Makefile | 9 ++++++--- vm/sys/net/default.nix | 5 ----- 8 files changed, 24 insertions(+), 32 deletions(-) diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile index 87bc15e..d78cc4c 100644 --- a/host/rootfs/Makefile +++ b/host/rootfs/Makefile @@ -45,15 +45,18 @@ LINKS = bin sbin BUILD_FILES = build/etc/mdev/modalias.sh build/etc/s6-rc MOUNTPOINTS = dev ext run proc sys -build/rootfs.tar: $(PACKAGES_TAR) $(FILES) $(BUILD_FILES) +build/empty: + mkdir -p $@ + +build/rootfs.tar: build/empty $(PACKAGES_TAR) $(FILES) $(BUILD_FILES) cp --no-preserve=mode -f $(PACKAGES_TAR) $@ tar $(TARFLAGS) --append -f $@ $(FILES) $(LINKS) echo $(BUILD_FILES) | cut -d/ -f2 | \ tar $(TARFLAGS) --append -f $@ -C build -T - for m in $(MOUNTPOINTS); do \ - tar $(TARFLAGS) --append -hf $@ --xform="s,.*,$$m," /var/empty ; \ + tar $(TARFLAGS) --append -hf $@ --xform="s,.*,$$m," build/empty ; \ done - tar $(TARFLAGS) --append -hf $@ --xform='s,.*,etc/service,' /var/empty + tar $(TARFLAGS) --append -hf $@ --xform='s,.*,etc/service,' build/empty build/etc/mdev/modules.map: scripts/modprobe/gen_modules.map.awk mkdir -p $$(dirname $@) diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index cb02445..0bef7ba 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -94,11 +94,6 @@ stdenv.mkDerivation { MODULES_ORDER = "${kernel}/lib/modules/${kernel.modDirVersion}/modules.order"; PACKAGES_TAR = packagesTar; - postPatch = '' - mkdir $NIX_BUILD_TOP/empty - substituteInPlace Makefile --replace /var/empty $NIX_BUILD_TOP/empty - ''; - installPhase = '' cp build/rootfs.ext4 $out ''; diff --git a/vm/app/catgirl/Makefile b/vm/app/catgirl/Makefile index 5029bb4..3ee5842 100644 --- a/vm/app/catgirl/Makefile +++ b/vm/app/catgirl/Makefile @@ -55,15 +55,18 @@ VM_LINKS = bin etc/ssl/certs/ca-certificates.crt VM_BUILD_FILES = build/etc/s6-rc VM_MOUNTPOINTS = dev run proc sys -build/rootfs.tar: $(PACKAGES_TAR) $(VM_FILES) $(VM_BUILD_FILES) +build/empty: + mkdir -p $@ + +build/rootfs.tar: build/empty $(PACKAGES_TAR) $(VM_FILES) $(VM_BUILD_FILES) cp --no-preserve=mode -f $(PACKAGES_TAR) $@ tar $(TARFLAGS) --append -f $@ $(VM_FILES) $(VM_LINKS) echo $(VM_BUILD_FILES) | cut -d/ -f2 | \ tar $(TARFLAGS) --append -f $@ -C build -T - for m in $(VM_MOUNTPOINTS); do \ - tar $(TARFLAGS) --append -hf $@ --xform="s,.*,$$m," /var/empty ; \ + tar $(TARFLAGS) --append -hf $@ --xform="s,.*,$$m," build/empty ; \ done - tar $(TARFLAGS) --append -hf $@ --xform='s,.*,etc/service,' /var/empty + tar $(TARFLAGS) --append -hf $@ --xform='s,.*,etc/service,' build/empty VM_S6_RC_FILES = \ etc/s6-rc/catgirl/run \ diff --git a/vm/app/catgirl/default.nix b/vm/app/catgirl/default.nix index 230da00..bf35d04 100644 --- a/vm/app/catgirl/default.nix +++ b/vm/app/catgirl/default.nix @@ -74,11 +74,6 @@ stdenv.mkDerivation { PACKAGES_TAR = packagesTar; VMLINUX = "${kernel.dev}/vmlinux"; - postPatch = '' - mkdir $NIX_BUILD_TOP/empty - substituteInPlace Makefile --replace /var/empty $NIX_BUILD_TOP/empty - ''; - installPhase = '' mv build/svc $out ''; diff --git a/vm/app/lynx/Makefile b/vm/app/lynx/Makefile index f449d27..016b1f5 100644 --- a/vm/app/lynx/Makefile +++ b/vm/app/lynx/Makefile @@ -55,15 +55,18 @@ VM_LINKS = bin etc/ssl/certs/ca-certificates.crt VM_BUILD_FILES = build/etc/s6-rc VM_MOUNTPOINTS = dev run proc sys -build/rootfs.tar: $(PACKAGES_TAR) $(VM_FILES) $(VM_BUILD_FILES) +build/empty: + mkdir -p $@ + +build/rootfs.tar: build/empty $(PACKAGES_TAR) $(VM_FILES) $(VM_BUILD_FILES) cp --no-preserve=mode -f $(PACKAGES_TAR) $@ tar $(TARFLAGS) --append -f $@ $(VM_FILES) $(VM_LINKS) echo $(VM_BUILD_FILES) | cut -d/ -f2 | \ tar $(TARFLAGS) --append -f $@ -C build -T - for m in $(VM_MOUNTPOINTS); do \ - tar $(TARFLAGS) --append -hf $@ --xform="s,.*,$$m," /var/empty ; \ + tar $(TARFLAGS) --append -hf $@ --xform="s,.*,$$m," build/empty ; \ done - tar $(TARFLAGS) --append -hf $@ --xform='s,.*,etc/service,' /var/empty + tar $(TARFLAGS) --append -hf $@ --xform='s,.*,etc/service,' build/empty VM_S6_RC_FILES = \ etc/s6-rc/lynx/run \ diff --git a/vm/app/lynx/default.nix b/vm/app/lynx/default.nix index f7791d8..b188794 100644 --- a/vm/app/lynx/default.nix +++ b/vm/app/lynx/default.nix @@ -74,11 +74,6 @@ stdenv.mkDerivation { PACKAGES_TAR = packagesTar; VMLINUX = "${kernel.dev}/vmlinux"; - postPatch = '' - mkdir $NIX_BUILD_TOP/empty - substituteInPlace Makefile --replace /var/empty $NIX_BUILD_TOP/empty - ''; - installPhase = '' mv build/svc $out ''; diff --git a/vm/sys/net/Makefile b/vm/sys/net/Makefile index 642c815..7103c0e 100644 --- a/vm/sys/net/Makefile +++ b/vm/sys/net/Makefile @@ -55,15 +55,18 @@ VM_LINKS = bin var/run VM_BUILD_FILES = build/etc/s6-rc VM_MOUNTPOINTS = dev run proc sys var/lib/connman -build/rootfs.tar: $(PACKAGES_TAR) $(VM_FILES) $(VM_BUILD_FILES) +build/empty: + mkdir -p $@ + +build/rootfs.tar: build/empty $(PACKAGES_TAR) $(VM_FILES) $(VM_BUILD_FILES) cp --no-preserve=mode -f $(PACKAGES_TAR) $@ tar $(TARFLAGS) --append -f $@ $(VM_FILES) $(VM_LINKS) echo $(VM_BUILD_FILES) | cut -d/ -f2 | \ tar $(TARFLAGS) --append -f $@ -C build -T - for m in $(VM_MOUNTPOINTS); do \ - tar $(TARFLAGS) --append -hf $@ --xform="s,.*,$$m," /var/empty ; \ + tar $(TARFLAGS) --append -hf $@ --xform="s,.*,$$m," build/empty ; \ done - tar $(TARFLAGS) --append -hf $@ --xform='s,.*,etc/service,' /var/empty + tar $(TARFLAGS) --append -hf $@ --xform='s,.*,etc/service,' build/empty VM_S6_RC_FILES = \ etc/s6-rc/connman/dependencies \ diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix index e3bd996..55ae9cb 100644 --- a/vm/sys/net/default.nix +++ b/vm/sys/net/default.nix @@ -83,11 +83,6 @@ stdenv.mkDerivation { PACKAGES_TAR = packagesTar; VMLINUX = "${kernel.dev}/vmlinux"; - postPatch = '' - mkdir $NIX_BUILD_TOP/empty - substituteInPlace Makefile --replace /var/empty $NIX_BUILD_TOP/empty - ''; - installPhase = '' mv build/svc $out ''; -- 2.35.1

2 1

Spectrum Status Update, March 2022
by Alyssa Ross 02 Apr '22

02 Apr '22

Hi all, I thought I'd try a different format of update. It's difficult to find the time for the big This Week in Spectrum updates I've tried to do before, but I'd like to provide some sort of account of what I've been doing. So, here's an overview of what I did in March. I'm happy to expand on any of it that sounds interesting — just hit Reply All and tell me what you'd like to hear more about! I'm also interested to hear what you think of this status update format. I'd like to get better about communicating what I'm working on, in a sustainable way. Let me know if you have any suggestions! Miscellaneous ------------- • Edited and published demo video[1] • Set up an IRC bot to post incoming mailing list messages • Switched from Busybox's modprobe to kmod • Removed unused dependencies • Various other cleanups and fixes • Started work towards CI for Spectrum • Prototyped a shared base image for application VMs [1]: https://diode.zone/w/dWAWHR38Zu3feRtDKjVEJb virtiofs investigation ---------------------- • Prototyped virtiofs VM filesystem access • Reported a bug: "Can't run unprivileged any more due to setgroups"[2] • Participated in discussion and testing of Musl port[3][4][5] [2]: https://gitlab.com/virtio-fs/virtiofsd/-/issues/36 [3]: https://github.com/slp/capng/pull/2#issuecomment-1059976861 [4]: https://github.com/slp/capng/pull/3 [5]: https://github.com/rust-lang/libc/pull/2713 Spectrum-related upstream Nixpkgs commits ----------------------------------------- • lvm2: don't use targetPlatform (05a6c124e65) • coreutils: add debug output (e30f0f31e8d) • pkgsMusl.systemd: fix build for 250.4 (39eee39fd92) • nghttp2: only run tests on GNU (8685cea963b) • python3.pkgs.importlib-metadata: fix cross (3c7b77e638b) • spidermonkey: use the same LLVM as rustc (3ff5f0eb764) • pkgsStatic.stdenv.cc.cc: put static libs in $lib (12c37aec377) • Revert "gcc: Always pass `--enable-shared` by default" (c6dd11ca39a) • libudev-zero: 1.0.0 -> 1.0.1 (c7b7ad77985) • linux_latest: 5.16.14 -> 5.17 (58ae11758e8) • crosvm: 81.12871.0.0-rc1 -> 99.14468.0.0-rc1 (6aefdafbed9) • shadow: 4.8.1 -> 4.8.11 (8d35d7e2bf1) • pkgsMusl.libnetfilter_conntrack: fix build (2cc5ec86571) • pkgsMusl.systemdMinimal: fix build (b8734c50e29) • linux.configfile: fix alts containing "/m" (fb079c3110d) • cloud-hypervisor: 21.0 -> 22.0 (36a211e1ee3) • edk2: 202108 -> 202202 (9222b68380e) • kmod: add dev and lib outputs (dc1303185f8) • systemd: update patchShebangs comment (a0bfc8e7c1f) • systemd: fix a whole bunch of typos (479b1cb510b) Pending Spectrum related Nixpkgs PRs ------------------------------------ • crosvm: add support for virgl_renderer{,_next} (#165128) • qemu: 6.2.0 -> 7.0.0 (#165291) Spectrum infra related upstream Nixpkgs commits ----------------------------------------------- • irccat: init at 0.4.8 (ce8cbe3c01f) • git: enable debug info (4345b27dedf) • cgit-pink: init at 1.3.0 (deab83e1167) • mailman-web: fix django version check removal (3512f5b7075) Demo video related upstream Nixpkgs commits ------------------------------------------- • ccsymbols: init at 2020-04-19 (cf7556eea5a)

1 0