November 2022 - Spectrum Development

[PATCH 00/22] Implement managing VMs with Nix
by Alyssa Ross 26 Feb '23

26 Feb '23

IMPORTANT NOTE: this series should be applied on top of v2 of my previous series "Introduce a shared base for application VMs" [1]. I'm much happier with v2 of that series, but I only posted it yesterday so I still want to leave a little more opportunity for comment before applying it. [1]: https://spectrum-os.org/lists/archives/spectrum-devel/20221009114036.463071… This series contains the final big chunk of work I had left to do on Spectrum's original NLnet grant. It adds support for managing Spectrum VMs from the Spectrum system itself using Nix. Nix is optional, and can co-exist with VMs provided in some other way. More information is included in the new documentation. Most of this work was done earlier this year, but I got stuck on some implementation details that prevented me from getting over the last hurdle until I came up with a solution. That's explained in more detail in patch 15. Patches 1–10 add support for configuring VMs with read/write access to host directories using virtiofs. Then, in patches 11–14, come various changes that make the default user data partition more suitable as a mutable filesystem, which we haven't actually used it for before. And then the remaining patches actually implement support for a VM that can run Nix and easily build VMs that are available on the host. Alyssa Ross (22): host/start-vm: use MAP_SHARED memory for VMs host/start-vm: implement shared directories host/rootfs: generate virtiofsd services Documentation: explain VM shared directories vm-lib/make-vm.nix: support shared directories img/app: add support for testing virtiofs img/app: don't block app startup on network online img/app: auto-mount virtiofs0 filesystem vm/app/mg.nix: init vm/app/mg.nix: open virtio filesystem in dired host/rootfs: move ext mounting to s6-rc service host/rootfs: automatically grow user partition host/rootfs: use a bigger test ext partition host/initramfs/extfs.nix: tar2ext4 -> mkfs.ext4 -d host/start-vm: resolve VM symlinks with /ext root host/rootfs: resolve VM symlinks with /ext root Documentation: explain /ext symlink resolution host/start-vm: increase memory size to 512M vm/app/nix: add vm-lib/make-vms.nix: add host/initramfs/extfs.nix: add example Nix-built VM Documentation: add how-to guide for Nix-built VMs .gitignore | 5 +- Documentation/_sass/custom/custom.scss | 22 ++++++ Documentation/creating-vms.adoc | 12 ++- Documentation/nix-vms.adoc | 22 ++++++ host/initramfs/extfs.nix | 29 +++++-- host/rootfs/Makefile | 13 +++- host/rootfs/default.nix | 16 +++- host/rootfs/etc/mdev/block/add | 1 - host/rootfs/etc/s6-rc/ext-rc-init/up | 19 ++++- host/rootfs/etc/s6-rc/ext/up | 5 +- host/rootfs/etc/template/fs/notification-fd | 1 + .../etc/template/fs/notification-fd.license | 2 + host/rootfs/etc/template/fs/run | 11 +++ host/rootfs/etc/template/fs/type | 1 + host/rootfs/etc/template/fs/type.license | 2 + host/start-vm/fs.c | 17 +++++ host/start-vm/fs.rs | 68 +++++++++++++++++ host/start-vm/lib.rs | 54 ++++++++++--- host/start-vm/meson.build | 2 +- host/start-vm/start-vm.rs | 15 ++-- host/start-vm/tests/meson.build | 4 + host/start-vm/tests/vm_command-basic.rs | 6 +- .../tests/vm_command-config-symlink.rs | 30 ++++++++ host/start-vm/tests/vm_command-shared-dir.rs | 43 +++++++++++ img/app/Makefile | 24 +++++- img/app/etc/mdev.conf | 3 +- img/app/etc/mdev/iface | 4 +- img/app/etc/mdev/listen | 12 +++ img/app/etc/mdev/virtiofs | 10 +++ img/app/etc/mdev/wait | 15 ++++ img/app/etc/s6-rc/ok-all/contents | 1 + img/app/shell.nix | 6 +- tools/resolve_in_root/default.nix | 23 ++++++ tools/resolve_in_root/meson.build | 10 +++ tools/resolve_in_root/resolve_in_root.c | 76 +++++++++++++++++++ tools/resolve_in_root/test.sh | 11 +++ vm-lib/make-vm.nix | 20 ++++- vm-lib/make-vms.nix | 19 +++++ vm/app/catgirl.nix | 1 + vm/app/lynx.nix | 1 + vm/app/{lynx.nix => mg.nix} | 10 ++- vm/app/nix/bin/vm-rebuild | 25 ++++++ vm/app/nix/default.nix | 43 +++++++++++ vm/app/nix/example.nix | 13 ++++ 44 files changed, 673 insertions(+), 54 deletions(-) create mode 100644 Documentation/_sass/custom/custom.scss create mode 100644 Documentation/nix-vms.adoc create mode 100644 host/rootfs/etc/template/fs/notification-fd create mode 100644 host/rootfs/etc/template/fs/notification-fd.license create mode 100755 host/rootfs/etc/template/fs/run create mode 100644 host/rootfs/etc/template/fs/type create mode 100644 host/rootfs/etc/template/fs/type.license create mode 100644 host/start-vm/fs.c create mode 100644 host/start-vm/fs.rs create mode 100644 host/start-vm/tests/vm_command-config-symlink.rs create mode 100644 host/start-vm/tests/vm_command-shared-dir.rs create mode 100755 img/app/etc/mdev/listen create mode 100755 img/app/etc/mdev/virtiofs create mode 100755 img/app/etc/mdev/wait create mode 100644 tools/resolve_in_root/default.nix create mode 100644 tools/resolve_in_root/meson.build create mode 100644 tools/resolve_in_root/resolve_in_root.c create mode 100755 tools/resolve_in_root/test.sh create mode 100644 vm-lib/make-vms.nix copy vm/app/{lynx.nix => mg.nix} (52%) create mode 100755 vm/app/nix/bin/vm-rebuild create mode 100644 vm/app/nix/default.nix create mode 100644 vm/app/nix/example.nix -- 2.37.1

2 44

[PATCH] Documentation: Architecture Decision Record
by Ville Ilvonen 06 Jan '23

06 Jan '23

* ADRs based on discussions with Alyssa * A note on ADRs to architecture.adoc Signed-off-by: Ville Ilvonen <ville.ilvonen(a)unikie.com> --- Documentation/architecture.adoc | 13 ++++++++++ ...architecture-decision-record-template.adoc | 13 ++++++++++ .../decisions/001-host-update-mechanism.adoc | 19 ++++++++++++++ .../decisions/002-install-options.adoc | 18 +++++++++++++ Documentation/decisions/003-partitioning.adoc | 25 +++++++++++++++++++ .../004-data-at-rest-encryption.adoc | 16 ++++++++++++ .../005-virtual-machine-manager.adoc | 24 ++++++++++++++++++ .../decisions/006-drivers-on-host.adoc | 17 +++++++++++++ .../decisions/007-USB-virtual-machine.adoc | 14 +++++++++++ ...008-Inter-VM-communication-mechanisms.adoc | 18 +++++++++++++ 10 files changed, 177 insertions(+) create mode 100644 Documentation/decisions/000-lightweight-architecture-decision-record-template.adoc create mode 100644 Documentation/decisions/001-host-update-mechanism.adoc create mode 100644 Documentation/decisions/002-install-options.adoc create mode 100644 Documentation/decisions/003-partitioning.adoc create mode 100644 Documentation/decisions/004-data-at-rest-encryption.adoc create mode 100644 Documentation/decisions/005-virtual-machine-manager.adoc create mode 100644 Documentation/decisions/006-drivers-on-host.adoc create mode 100644 Documentation/decisions/007-USB-virtual-machine.adoc create mode 100644 Documentation/decisions/008-Inter-VM-communication-mechanisms.adoc diff --git a/Documentation/architecture.adoc b/Documentation/architecture.adoc index 157907f..185740c 100644 --- a/Documentation/architecture.adoc +++ b/Documentation/architecture.adoc @@ -20,6 +20,19 @@ devices and provides network services to application VMs). Refer to xref:creating-vms.adoc[Creating VMs] and xref:running-vms.adoc[Running VMs] for more information about using VMs in Spectrum. +== Architecture Decision Record (ADR) + +https://adr.github.io/[Architecturally significant decisions] are +recorded as https://github.com/joelparkerhenderson/architecture-decision-record/blob/ma… ADRs] + +Status of Spectrum OS ADRs: +Accepted - Implemented and likely not to change. +Proposed - Designed and possibly partially implmented. May change. +Other - Not yet in use. + +Comments and contributions to ADRs are welcome. ADRs can be found at +Documentation/decisions + == The Spectrum host system Compartmentalization is implemented using diff --git a/Documentation/decisions/000-lightweight-architecture-decision-record-template.adoc b/Documentation/decisions/000-lightweight-architecture-decision-record-template.adoc new file mode 100644 index 0000000..087ec44 --- /dev/null +++ b/Documentation/decisions/000-lightweight-architecture-decision-record-template.adoc @@ -0,0 +1,13 @@ +# Title + +## Status +What is the status, such as proposed, accepted, rejected, deprecated, superseded, etc.? + +## Context +What is the issue that we're seeing that is motivating this decision or change? + +## Decision +What is the change that we're proposing and/or doing? + +## Consequences +What becomes easier or more difficult to do because of this change? diff --git a/Documentation/decisions/001-host-update-mechanism.adoc b/Documentation/decisions/001-host-update-mechanism.adoc new file mode 100644 index 0000000..03bbae2 --- /dev/null +++ b/Documentation/decisions/001-host-update-mechanism.adoc @@ -0,0 +1,19 @@ +# Host update mechanism + +## Status +Proposed + +## Context +Spectrum OS has no implementation for software update. The host - consisting of +Linux kernel, KVM, cloud-hypervisor and minimal user space tools - software +updates are required to support feature development and security fixes. + +## Decision +A-B partitioning created by Spectrum installer Installer sets up the system on +partition A of the block device A-B update scheme where user (or installer) +writes the update image to partition B Bootloader provides four boot options: +A, A mutable, B, B mutable + +## Consequences +Default boot selection, incremental updates (e.g. overlays), network update +postponed for later. diff --git a/Documentation/decisions/002-install-options.adoc b/Documentation/decisions/002-install-options.adoc new file mode 100644 index 0000000..f5857c5 --- /dev/null +++ b/Documentation/decisions/002-install-options.adoc @@ -0,0 +1,18 @@ +# Install options + +## Status +Proposed + +## Context +Based on identified different audiences for the Spectrum OS release it is +proposed we support three base configurations to use with Spectrum OS in the +first boot. + +## Decision +* Minimal - Spectrum OS host + system VMs: netvm, guivm, usbvm + home-directory +(optionally encrypted - see 004-disk-encryption.md) +* Common - Minimal + browser app VM + 2-3 selected app VMs +* Power - Common + NixOS VM + +## Consequences +Requires first-boot-vm (like wizard) to support user to get started. diff --git a/Documentation/decisions/003-partitioning.adoc b/Documentation/decisions/003-partitioning.adoc new file mode 100644 index 0000000..345619f --- /dev/null +++ b/Documentation/decisions/003-partitioning.adoc @@ -0,0 +1,25 @@ +# Partitioning + +## Status +Proposed + +## Context +Partitions are required to install the Spectrum OS, VMs and store user data. + +## Decision +---- +<blockdevice> # EFI system partition +<blockdevice> # XBOOTLDR +<blockdevice> # A +<blockdevice> # B +# first 32 GB are reserved for Spectrum system +# rest of the disk is reserved for user data +<blockdevice>n-1 # bootstrap user data +<blockdevice>n to the end of disk # user data +---- + +## Consequences +LVM may support resizing - both increasing and decreasing with some limitation +when there's alreay data on volume(s). Does LVM work with all disk types? We +have to implement XBOOTLDR to support EFI system partition created by Windows - +to support dual boot diff --git a/Documentation/decisions/004-data-at-rest-encryption.adoc b/Documentation/decisions/004-data-at-rest-encryption.adoc new file mode 100644 index 0000000..3ed9abb --- /dev/null +++ b/Documentation/decisions/004-data-at-rest-encryption.adoc @@ -0,0 +1,16 @@ +# Data at rest encryption + +## Status +Proposed + +## Context +To support user data and privacy protection, encryption of data at rest is +required. + +## Decision +User data is encrypted. + +## Consequences +Spectrum OS needs to come with enough SW to get the encryption key via different +methods (password, usb, fido, etc.) Can we use dm-crypt for everything instead +of LUKS? diff --git a/Documentation/decisions/005-virtual-machine-manager.adoc b/Documentation/decisions/005-virtual-machine-manager.adoc new file mode 100644 index 0000000..b4af595 --- /dev/null +++ b/Documentation/decisions/005-virtual-machine-manager.adoc @@ -0,0 +1,24 @@ +# Virtual Machine Manager + +## Status +Accepted + +## Context +rust-vmm-based VMM provides memory and concurrency safe solution. +cloud-hypervisor was chosen because firecrack does not support other +virtio-devices than net or block. crosvm was not chosen because cloud-hypervisor +has more flexible IPC mechanisms, more engaging community as LF-project. +cloud-hypervisor has more core features - such as snapshotting, live migration +and more general hot plugging. crosvm supports more devices we will also need. +It was seen easier to port devices from crosvm to cloud-hypervisor than to port +core features from cloud-hypervisor to crosvm. + +## Decision +Spectrum OS design and implementation decision is to use cloud-hypervisor as the +primary VMM. + +## Consequences +We gotta port some stuff from crosvm to cloud-hypervisor. It's easier for +Spectrum to handle virtualization dynamically with cloud-hypervisor. If the +primary VMM, cloud-hypervisor, is exchanged for trials etc. functionality is +expected to break or not supported. diff --git a/Documentation/decisions/006-drivers-on-host.adoc b/Documentation/decisions/006-drivers-on-host.adoc new file mode 100644 index 0000000..052b596 --- /dev/null +++ b/Documentation/decisions/006-drivers-on-host.adoc @@ -0,0 +1,17 @@ +# Drivers on host + +## Status +Accepted + +## Context +To harden the trusted computing base and make it more minimal, the target is to +minimize the amount of drivers on the Spectrum host kernel. + +## Decision +We are aiming to have as few drivers as possible on the host. + +## Consequences +No networking on the host. Responsibilities of the host are expected to get +smaller over time. More flexible management of devices. We need to decouple +device classes - like net, usb, bluetooth and gui - from host to their +respective VMs. diff --git a/Documentation/decisions/007-USB-virtual-machine.adoc b/Documentation/decisions/007-USB-virtual-machine.adoc new file mode 100644 index 0000000..2072427 --- /dev/null +++ b/Documentation/decisions/007-USB-virtual-machine.adoc @@ -0,0 +1,14 @@ +# USB Virtual Machine + +## Status +Proposed + +## Context +To support specific USB devices on specific VMs + +## Decision +The decision is to pass-through USB controller to a VM with authorization +controls inside the VMs to forward a specific USB device using USBIP. + +## Consequences +We need to modify the upstream USBIP daemon to support authorization. diff --git a/Documentation/decisions/008-Inter-VM-communication-mechanisms.adoc b/Documentation/decisions/008-Inter-VM-communication-mechanisms.adoc new file mode 100644 index 0000000..1007037 --- /dev/null +++ b/Documentation/decisions/008-Inter-VM-communication-mechanisms.adoc @@ -0,0 +1,18 @@ +### Inter VM communication mechanisms + +### Status +Proposed + +### Context +Guest VM software needs to communicate with software in other guest VMs + +### Decision +Spectrum provides two mechanism +- TCP/IP with virtio-net +- Wayland with virtio-gpu (nevermind the semantics) for streamed IPC protocol to + send references to shared memory + + +### Consequences +- Examples required on how to write applications which communicate over + virtio-gpu -- 2.36.0

4 9

Enabling forwarding Wayland from appVM to host with waypipe
by Vadim Likholetov 30 Nov '22

30 Nov '22

This patchset enables forwarding Wayland connections from appVM to the host with the help of waypipe tool. It doesn't reqire major changes in Spectrum and tries to utilise existing packages and solutions. Main idea is to create VSOCK virtual device on the start of appVM and use this virtual socket to connect host and guest. On the host side I implement vm-start-way tool that is based on vm-start, but does necessary preparations for wayland forwarfing. The guest VM is appvm-hello-waypipe and described in hello-waypipe.nix file. To test it boot the Spectrum and do 'vm-start-way appvm-hello-waypipe'. Be patient and in couple of seconds will appear a terminal window from appVM. This patches are for test the concept, so possibly redisign is needed for production use. ps sorry for mess in the mailing list

2 7

[PATCH] Patches to waypipe package to enable static build with the cost of fancy feature
by vadim likholetov 30 Nov '22

30 Nov '22

Signed-off-by: vadim likholetov <vadim.likholetov(a)unikie.com> --- .../applications/networking/remote/waypipe/default.nix | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/pkgs/applications/networking/remote/waypipe/default.nix b/pkgs/applications/networking/remote/waypipe/default.nix index ec94d504b9d..bcc3770ecf7 100644 --- a/pkgs/applications/networking/remote/waypipe/default.nix +++ b/pkgs/applications/networking/remote/waypipe/default.nix @@ -20,7 +20,15 @@ stdenv.mkDerivation rec { nativeBuildInputs = [ meson ninja pkg-config scdoc ]; buildInputs = [ # Optional dependencies: - mesa lz4 zstd ffmpeg libva + # mesa lz4 zstd ffmpeg libva # TODO: one day recompile with acceleration + lz4 zstd + ]; + + mesonFlags = [ + "-Dwith_video=disabled" + "-Dwith_systemtap=false" + "-Dwith_vaapi=disabled" + "-Dwith_dmabuf=disabled" ]; meta = with lib; { -- 2.36.2

3 2

[PATCH] Enables forwarding Wayland from appVM to host with waypipe
by vadim likholetov 30 Nov '22

30 Nov '22

Signed-off-by: vadim likholetov <vadim.likholetov(a)unikie.com> --- host/initramfs/extfs.nix | 4 +++- host/rootfs/Makefile | 1 + host/rootfs/default.nix | 4 ++-- host/rootfs/usr/bin/vm-start-way | 10 ++++++++++ img/app/default.nix | 3 +++ vm/app/hello-waypipe.nix | 23 +++++++++++++++++++++++ 6 files changed, 42 insertions(+), 3 deletions(-) create mode 100755 host/rootfs/usr/bin/vm-start-way create mode 100644 vm/app/hello-waypipe.nix diff --git a/host/initramfs/extfs.nix b/host/initramfs/extfs.nix index 5c5850f..f49e519 100644 --- a/host/initramfs/extfs.nix +++ b/host/initramfs/extfs.nix @@ -11,12 +11,13 @@ let appvm-catgirl = import ../../vm/app/catgirl.nix { inherit config; }; appvm-lynx = import ../../vm/app/lynx.nix { inherit config; }; + appvm-hello-waypipe = import ../../vm/app/hello-waypipe.nix { inherit config; }; in runCommand "ext.ext4" { nativeBuildInputs = [ e2fsprogs ]; } '' - mkdir -p root/svc/data/appvm-{catgirl,lynx} + mkdir -p root/svc/data/appvm-{catgirl,lynx,hello-waypipe} cd root tar -C ${netvm} -c data | tar -C svc -x @@ -24,6 +25,7 @@ runCommand "ext.ext4" { tar -C ${appvm-catgirl} -c . | tar -C svc/data/appvm-catgirl -x tar -C ${appvm-lynx} -c . | tar -C svc/data/appvm-lynx -x + tar -C ${appvm-hello-waypipe} -c . | tar -C svc/data/appvm-hello-waypipe -x mkfs.ext4 -d . $out 16T resize2fs -M $out diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile index 9559c06..06e3e8e 100644 --- a/host/rootfs/Makefile +++ b/host/rootfs/Makefile @@ -39,6 +39,7 @@ FILES = \ usr/bin/lsvm \ usr/bin/vm-console \ usr/bin/vm-start \ + usr/bin/vm-start-way \ usr/bin/vm-stop DIRS = dev etc/s6-linux-init/env ext run proc sys diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index 0a84f55..de4a6b6 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -7,7 +7,7 @@ pkgs.pkgsStatic.callPackage ( { lib, stdenvNoCC, nixos, runCommand, writeReferencesToFile, s6-rc, tar2ext4 , busybox, cloud-hypervisor, cryptsetup, execline, e2fsprogs, jq, kmod -, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg +, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg, waypipe }: let @@ -49,7 +49,7 @@ let packages = [ cloud-hypervisor e2fsprogs execline jq kmod mdevd s6 s6-linux-init s6-rc - socat start-vm + socat start-vm waypipe (cryptsetup.override { programs = { diff --git a/host/rootfs/usr/bin/vm-start-way b/host/rootfs/usr/bin/vm-start-way new file mode 100755 index 0000000..bda9934 --- /dev/null +++ b/host/rootfs/usr/bin/vm-start-way @@ -0,0 +1,10 @@ +#!/bin/sh +# SPDX-License-Identifier: EUPL-1.2+ + +s6-rc -bu change ext-rc +s6-rc -l /run/s6-rc.ext -u change $1 +ch-remote --api-socket /run/service/ext-${1}/env/cloud-hypervisor.sock add-vsock cid=4,socket=/run/u.${1}.socket +cd /run +nohup waypipe --socket /run/waypipe.sock client & +nohup socat unix-listen:/run/u.${1}.socket_5000,reuseaddr,fork unix-connect:/run/waypipe.sock & + diff --git a/img/app/default.nix b/img/app/default.nix index e7d5366..80f23c2 100644 --- a/img/app/default.nix +++ b/img/app/default.nix @@ -59,6 +59,9 @@ let DRM_BOCHS = yes; DRM = yes; AGP = yes; + VSOCKETS = yes; + VIRTIO_VSOCKETS = yes; + VIRTIO_VSOCKETS_COMMON = yes; }; }; in diff --git a/vm/app/hello-waypipe.nix b/vm/app/hello-waypipe.nix new file mode 100644 index 0000000..601b638 --- /dev/null +++ b/vm/app/hello-waypipe.nix @@ -0,0 +1,23 @@ +# SPDX-License-Identifier: MIT +# SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> + +{ config ? import ../../../nix/eval-config.nix {} }: + +import ../make-vm.nix { inherit config; } { + providers.net = [ "netvm" ]; + run = config.pkgs.callPackage ( + { writeScript, waypipe, havoc, foot, hello-wayland, socat}: + writeScript "run-waypipe-app" '' + #!/bin/sh + mkdir /run/0 + export XDG_RUNTIME_DIR=/run/0 + ${socat}/bin/socat unix-listen:/run/waypipe.sock,reuseaddr,fork vsock-connect:2:5000 & + sleep 1 + ${waypipe}/bin/waypipe --display wayland-local --socket /run/waypipe.sock server -- sleep inf & + export WAYLAND_DISPLAY=wayland-local + ${havoc}/bin/havoc + ${hello-wayland}/bin/hello-wayland + ${foot}/bin/foot + '' + ) { }; +} -- 2.36.2

1 0

[PATCH] Patches to waypipe package to enable static build with the cost of fancy feature
by vadim likholetov 29 Nov '22

29 Nov '22

--- .../applications/networking/remote/waypipe/default.nix | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/pkgs/applications/networking/remote/waypipe/default.nix b/pkgs/applications/networking/remote/waypipe/default.nix index ec94d504b9d..bcc3770ecf7 100644 --- a/pkgs/applications/networking/remote/waypipe/default.nix +++ b/pkgs/applications/networking/remote/waypipe/default.nix @@ -20,7 +20,15 @@ stdenv.mkDerivation rec { nativeBuildInputs = [ meson ninja pkg-config scdoc ]; buildInputs = [ # Optional dependencies: - mesa lz4 zstd ffmpeg libva + # mesa lz4 zstd ffmpeg libva # TODO: one day recompile with acceleration + lz4 zstd + ]; + + mesonFlags = [ + "-Dwith_video=disabled" + "-Dwith_systemtap=false" + "-Dwith_vaapi=disabled" + "-Dwith_dmabuf=disabled" ]; meta = with lib; { -- 2.36.2

1 0

[PATCH] This patchset enables forwarding Wayland connections from appVM to the host with the help of waypipe tool. It doesn't reqire major changes in Spectrum and tries to utilise existing packages and solutions. Main idea is to create VSOCK virtual device on the start of appVM and use this virtual socket to connect host and guest. On the host side I implement vm-start-way tool that is based on vm-start, but does nesessary preparations for wayland forwarfing. The guest VM is appvm-hello-waypipe and describe
by vadim likholetov 29 Nov '22

29 Nov '22

--- host/initramfs/extfs.nix | 4 +++- host/rootfs/Makefile | 1 + host/rootfs/default.nix | 4 ++-- host/rootfs/usr/bin/vm-start-way | 10 ++++++++++ img/app/default.nix | 3 +++ vm/app/hello-waypipe.nix | 23 +++++++++++++++++++++++ 6 files changed, 42 insertions(+), 3 deletions(-) create mode 100755 host/rootfs/usr/bin/vm-start-way create mode 100644 vm/app/hello-waypipe.nix diff --git a/host/initramfs/extfs.nix b/host/initramfs/extfs.nix index 5c5850f..f49e519 100644 --- a/host/initramfs/extfs.nix +++ b/host/initramfs/extfs.nix @@ -11,12 +11,13 @@ let appvm-catgirl = import ../../vm/app/catgirl.nix { inherit config; }; appvm-lynx = import ../../vm/app/lynx.nix { inherit config; }; + appvm-hello-waypipe = import ../../vm/app/hello-waypipe.nix { inherit config; }; in runCommand "ext.ext4" { nativeBuildInputs = [ e2fsprogs ]; } '' - mkdir -p root/svc/data/appvm-{catgirl,lynx} + mkdir -p root/svc/data/appvm-{catgirl,lynx,hello-waypipe} cd root tar -C ${netvm} -c data | tar -C svc -x @@ -24,6 +25,7 @@ runCommand "ext.ext4" { tar -C ${appvm-catgirl} -c . | tar -C svc/data/appvm-catgirl -x tar -C ${appvm-lynx} -c . | tar -C svc/data/appvm-lynx -x + tar -C ${appvm-hello-waypipe} -c . | tar -C svc/data/appvm-hello-waypipe -x mkfs.ext4 -d . $out 16T resize2fs -M $out diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile index 9559c06..06e3e8e 100644 --- a/host/rootfs/Makefile +++ b/host/rootfs/Makefile @@ -39,6 +39,7 @@ FILES = \ usr/bin/lsvm \ usr/bin/vm-console \ usr/bin/vm-start \ + usr/bin/vm-start-way \ usr/bin/vm-stop DIRS = dev etc/s6-linux-init/env ext run proc sys diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index 0a84f55..de4a6b6 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -7,7 +7,7 @@ pkgs.pkgsStatic.callPackage ( { lib, stdenvNoCC, nixos, runCommand, writeReferencesToFile, s6-rc, tar2ext4 , busybox, cloud-hypervisor, cryptsetup, execline, e2fsprogs, jq, kmod -, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg +, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg, waypipe }: let @@ -49,7 +49,7 @@ let packages = [ cloud-hypervisor e2fsprogs execline jq kmod mdevd s6 s6-linux-init s6-rc - socat start-vm + socat start-vm waypipe (cryptsetup.override { programs = { diff --git a/host/rootfs/usr/bin/vm-start-way b/host/rootfs/usr/bin/vm-start-way new file mode 100755 index 0000000..bda9934 --- /dev/null +++ b/host/rootfs/usr/bin/vm-start-way @@ -0,0 +1,10 @@ +#!/bin/sh +# SPDX-License-Identifier: EUPL-1.2+ + +s6-rc -bu change ext-rc +s6-rc -l /run/s6-rc.ext -u change $1 +ch-remote --api-socket /run/service/ext-${1}/env/cloud-hypervisor.sock add-vsock cid=4,socket=/run/u.${1}.socket +cd /run +nohup waypipe --socket /run/waypipe.sock client & +nohup socat unix-listen:/run/u.${1}.socket_5000,reuseaddr,fork unix-connect:/run/waypipe.sock & + diff --git a/img/app/default.nix b/img/app/default.nix index e7d5366..80f23c2 100644 --- a/img/app/default.nix +++ b/img/app/default.nix @@ -59,6 +59,9 @@ let DRM_BOCHS = yes; DRM = yes; AGP = yes; + VSOCKETS = yes; + VIRTIO_VSOCKETS = yes; + VIRTIO_VSOCKETS_COMMON = yes; }; }; in diff --git a/vm/app/hello-waypipe.nix b/vm/app/hello-waypipe.nix new file mode 100644 index 0000000..601b638 --- /dev/null +++ b/vm/app/hello-waypipe.nix @@ -0,0 +1,23 @@ +# SPDX-License-Identifier: MIT +# SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> + +{ config ? import ../../../nix/eval-config.nix {} }: + +import ../make-vm.nix { inherit config; } { + providers.net = [ "netvm" ]; + run = config.pkgs.callPackage ( + { writeScript, waypipe, havoc, foot, hello-wayland, socat}: + writeScript "run-waypipe-app" '' + #!/bin/sh + mkdir /run/0 + export XDG_RUNTIME_DIR=/run/0 + ${socat}/bin/socat unix-listen:/run/waypipe.sock,reuseaddr,fork vsock-connect:2:5000 & + sleep 1 + ${waypipe}/bin/waypipe --display wayland-local --socket /run/waypipe.sock server -- sleep inf & + export WAYLAND_DISPLAY=wayland-local + ${havoc}/bin/havoc + ${hello-wayland}/bin/hello-wayland + ${foot}/bin/foot + '' + ) { }; +} -- 2.36.2

1 0

[PATCH] Documentation: add ADR for VM Sources
by Alyssa Ross 29 Nov '22

29 Nov '22

Signed-off-by: Alyssa Ross <alyssa.ross(a)unikie.com> --- Documentation/decisions/009-vm-sources.adoc | 42 +++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 Documentation/decisions/009-vm-sources.adoc diff --git a/Documentation/decisions/009-vm-sources.adoc b/Documentation/decisions/009-vm-sources.adoc new file mode 100644 index 0000000..457491d --- /dev/null +++ b/Documentation/decisions/009-vm-sources.adoc @@ -0,0 +1,42 @@ += 009 VM Sources +:page-parent: Architecture Decision Records +:page-grand_parent: About Spectrum + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +== Status + +Proposed + +== Context + +It should be possible for Spectrum VMs to come from the root +filesystem (for built-in VMs), or one or more +xref:../appendices/user-partition.adoc[user partitions]. One part of +making this work will be to have the system search multiple sources of +VM definitions, to produce a single view of what VMs are available. + +== Decision + +The system will scan /usr/img/builtin, as well as the svc/data +partition on each user partition, for VM definitions. A single s6-rc +service directory will contain generated services for VMs from all +sources, so that tools like vm-start don't have to be aware of where +VMs come from. (This will continue to be distinct from the s6-rc +database used for host system services.) When a new source of VMs is +discovered, an updated database will be generated and atomically +replace the old one using +xref:https://www.skarnet.org/software/s6-rc/s6-rc-update.html[s6-rc-update]. + +Changes to how user partitions will be discovered and mounted, to +support having multiple user partitions, will be determined later. + +== Consequences + +Built-in VMs can be moved into the root filesystem, allowing the +default user partition to be empty. This will make it easier for +people making custom Spectrum images to add custom VMs without having +to build Spectrum itself themselves -- they can simply replace the +default, empty, partition with a custom one they can produce however +they like. base-commit: 95fba8ac6c872c2ba5e2ea8be5da9f2eb63374c8 -- 2.35.1

1 0

Waypipe Wayland guest to host forwarding prototype
by Vadim Likholetov 23 Nov '22

23 Nov '22

Hi, I’ve built a prototype of forwarding Wayland client connections with the help of waypipe. It is broken the same way same way as current Alyssa’s implementation. On imx8 green cat is ok, foot don’t start with no clear diagnostics. Is this prototype have interest in community? Should I submit patches of this prototype?

2 1

[PATCH] Documentation: move Appendices to Development, fixed links
by Jenni Nikolaenko 23 Nov '22

23 Nov '22

Both sections "User Partition" and "UUID Reference" are more development topics, so by Alyssa's advice they should go directly under the Development chapter. The section Appendices was removed. In addition, I replaced broken links. Signed-off-by: Jenni Nikolaenko <evgeniia.nikolaenko(a)unikie.com> --- Documentation/about/architecture.adoc | 6 +++--- Documentation/appendices/index.adoc | 13 ------------- .../{appendices => development}/user-partition.adoc | 6 +++--- .../{appendices => development}/uuid-reference.adoc | 11 +++++------ Documentation/index.adoc | 8 ++++---- Documentation/using-spectrum/creating-vms.adoc | 2 +- 6 files changed, 16 insertions(+), 30 deletions(-) delete mode 100644 Documentation/appendices/index.adoc rename Documentation/{appendices => development}/user-partition.adoc (87%) rename Documentation/{appendices => development}/uuid-reference.adoc (85%) diff --git a/Documentation/about/architecture.adoc b/Documentation/about/architecture.adoc index b311e77..38e0a22 100644 --- a/Documentation/about/architecture.adoc +++ b/Documentation/about/architecture.adoc @@ -15,8 +15,8 @@ The default set of virtual machines includes two application VMs, _appvm-catgirl_ (an IRC client) and _appvm-lynx_ (a text-based web browser); and a system VM, _netvm_ (which handles hardware network devices and provides network services to application VMs). -Refer to xref:../getting-started/creating-vms.adoc[Creating VMs] and -xref:../getting-started/running-vms.adoc[Running +Refer to xref:../using-spectrum/creating-vms.adoc[Creating VMs] and +xref:../using-spectrum/running-vms.adoc[Running VMs] for more information about using VMs in Spectrum. == Architecture Decision Records (ADRs) @@ -28,7 +28,7 @@ https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions[AD…. == Spectrum Host System Compartmentalization is implemented using -https://cloud-hypervisor.org/[cloud-hypervisor] virtual machines. +https://cloudhypervisor.org/[cloud-hypervisor] virtual machines. cloud-hypervisor uses Linux's https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine[Kernel-based Virtual Machine] (KVM) to provide lightweight, hardware-accelerated VMs. diff --git a/Documentation/appendices/index.adoc b/Documentation/appendices/index.adoc deleted file mode 100644 index ebacda7..0000000 --- a/Documentation/appendices/index.adoc +++ /dev/null @@ -1,13 +0,0 @@ -= Appendices -:description: Additional information. -:page-nav_order: 5 -:page-has_children: true -:page-has_toc: false - -// SPDX-FileCopyrightText: 2022 Unikie -// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 - -This section provides additional information on how to work with: - -* xref:../appendices/user-partition.adoc[user partition] -* xref:../appendices/uuid-reference.adoc[universally unique identifier (UUID)]. diff --git a/Documentation/appendices/user-partition.adoc b/Documentation/development/user-partition.adoc similarity index 87% rename from Documentation/appendices/user-partition.adoc rename to Documentation/development/user-partition.adoc index 4121340..d8fd956 100644 --- a/Documentation/appendices/user-partition.adoc +++ b/Documentation/development/user-partition.adoc @@ -1,6 +1,6 @@ -= Appendix A. User Partition -:page-parent: Appendices -:page-nav_order: 1 += User Partition +:page-parent: Development +:page-nav_order: 5 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/appendices/uuid-reference.adoc b/Documentation/development/uuid-reference.adoc similarity index 85% rename from Documentation/appendices/uuid-reference.adoc rename to Documentation/development/uuid-reference.adoc index 5099ec3..81300ee 100644 --- a/Documentation/appendices/uuid-reference.adoc +++ b/Documentation/development/uuid-reference.adoc @@ -1,7 +1,6 @@ -= Appendix B. UUID Reference -:page-parent: Appendices -:toclevels: 1 -:page-nav_order: 2 += UUID Reference +:page-parent: Development +:page-nav_order: 6 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -25,13 +24,13 @@ Specification]. === `4f68bce3-e8cd-4db1-96e7-fbcaf984b709` "Root Partition (amd64/x86_64)" from the -https://systemd.io/DISCOVERABLE_PARTITIONS/[Discoverable Partitions +https://uapi-group.org/specifications/specs/discoverable_partitions_specification/[Discoverable Partitions Specification]. === `2c7357ed-ebd2-46d9-aec1-23d437ec2bf5` "Root Verity Partition (amd64/x86_64)" from the -https://systemd.io/DISCOVERABLE_PARTITIONS/[Discoverable Partitions +https://uapi-group.org/specifications/specs/discoverable_partitions_specification/[Discoverable Partitions Specification]. === `9293e1ff-cee4-4658-88be-898ec863944f` diff --git a/Documentation/index.adoc b/Documentation/index.adoc index e0a45c7..51a6e7d 100644 --- a/Documentation/index.adoc +++ b/Documentation/index.adoc @@ -15,7 +15,7 @@ lower barrier to entry and is easy to use and maintain. To learn about what Spectrum is and how it's implemented, start with the xref:about/architecture.adoc[architecture overview]. -If you want to try Spectrum, see xref:../installation/index.adoc[Build and Run] +If you want to try Spectrum, see xref:installation/index.adoc[Build and Run] to setup a development environment. @@ -25,8 +25,8 @@ Spectrum is made of free and open-source software. It is free for anyone to use, modify, and distribute. Once you are up and running, see - xref:../development/index.adoc[Development] to understand how to work with + xref:development/index.adoc[Development] to understand how to work with Spectrum's code and participate in the collaborative development process. -If you are thinking of contributing to Spectrum docs, see - xref:../development/building-documentation.adoc[Building Documentation]. +If you are thinking of contributing to the Spectrum docs, see + xref:development/building-documentation.adoc[Building Documentation]. diff --git a/Documentation/using-spectrum/creating-vms.adoc b/Documentation/using-spectrum/creating-vms.adoc index 6847249..2c57562 100644 --- a/Documentation/using-spectrum/creating-vms.adoc +++ b/Documentation/using-spectrum/creating-vms.adoc @@ -8,7 +8,7 @@ == Configuration Each VM is configured using a directory under svc/data/ on the -xref:user-partition.adoc[user partition]. The name of the directory +xref:../development/user-partition.adoc[user partition]. The name of the directory determines the name of the VM. The directory can contain the following files: -- 2.34.1

3 2