1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
|
SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0
The code in this repository controls the Spectrum binary cache
builder. [1]
A binary cache build is started by running
vultr_key=… $(nix-build run-vultr.nix)
run-vultr.nix produces a script that creates a new cloud server
instance on Vultr[2].
The life cycle of the cloud server is as follows:
1. The server boots into a Vultr-provided iPXE[3] image, with a
command line generated by ipxe.nix.
2. iPXE downloads a nix-community NixOS netboot image[4], and boots
it with a custom kernel command line.
3. The custom kernel command line configures systemd to create and
start a unit that downloads and runs the "run" script from the
copy of this repository on https://spectrum-os.org/git/.
4. The "run" script sets up the block device as an encrypted swap
device, and grows the tmpfs at / to the size of the block device.
5. The "run" script fetches and unpacks a tarball from the server's
configured "userdata" (set up by the script produced by
run-vultr.nix earlier), to install the private signing key for Nix
builds, and the SSH key pair it uses to connect to the storage
server.
6. The "run" script downloads and switches to the NixOS configuration
in configuration.nix, which sets up read-only web log access using
systemd-journal-gatewayd(8), and configures Nix to copy completed
builds and their associated logs to the storage server.
7. The "run" script creates and starts a systemd unit to do a Nix
build of Spectrum's release.nix.
Currently, the server is not automatically terminated at the end of
the build, to allow for inspection and debugging if required.
The reason for using this dynamic netboot-based setup is to avoid the
need to create and maintain custom images. Because Spectrum builds
are only required when changes are pushed to the Spectrum repository,
and that happens at unpredictable intervals, if we built our own
images and wanted to guarantee that they were suitably up to date,
we'd need infrastructure for doing so, and there's a good chance that
most images produced would go unused. With this method, we get an
always up-to-date NixOS, without having to create the images
ourselves.
[1]: https://spectrum-os.org/doc/installation/binary-cache.html
[2]: https://www.vultr.com/
[3]: https://ipxe.org/
[4]: https://github.com/nix-community/nixos-images
|