Hi all,
As Ivan (@jonne) updated earlier this week on #spectrum, we've got initial cross-compilation support for aarch64 available in Spectrum development fork. It still needs some TLC before Spectrum PRs but is a great step forward. Our current ref. board is NXP i.MX8 but rootfs build should be fairly easy to modify for RPi4 configs already in nixpkgs for anyone who cares to try.
Quick how-to for non-nixos host like Ubuntu:
git clone -b aarch64-imx8-crosscompile https://github.com/tiiuae/nixpkgs-spectrum.git
git clone -b aarch64-imx8-crosscompile https://github.com/tiiuae/spectrum.git
cd spectrum/host/rootfs
nix-build -I nixpkgs=../../../nixpkgs-spectrum
Know issue:
- crossSystem from https://github.com/tiiuae/spectrum/commit/aad5f90cf2ec931f2af5c8d5111a9fac1… is ignored on NixOS host but not on Ubuntu - maybe someone can point us why.
Thanks,
-Ville
* Initial architecture document view a view to high level
stack of Spectrum.
* Stack view is generated to svg from drawio using Alyssa's
drawio-headless in nixpkgs-upstream. Cherry-picked to
nixpkgs-spectrum for testing.
* An example to analyze the details of Spectrum dependencies
interactively is also provided.
Signed-off-by: Ville Ilvonen <ville.ilvonen(a)unikie.com>
---
Documentation/architecture.adoc | 39 +++++++++++++++++++++++++++++
Documentation/default.nix | 8 +++---
Documentation/diagrams/stack.drawio | 1 +
3 files changed, 45 insertions(+), 3 deletions(-)
create mode 100644 Documentation/architecture.adoc
create mode 100644 Documentation/diagrams/stack.drawio
diff --git a/Documentation/architecture.adoc b/Documentation/architecture.adoc
new file mode 100644
index 0000000..2f89e68
--- /dev/null
+++ b/Documentation/architecture.adoc
@@ -0,0 +1,39 @@
+= Architecture
+// SPDX-FileCopyrightText: 2022 Ville Ilvonen <ville.ilvonen(a)unikie.com>
+// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0
+
+== Introduction
+
+Spectrum operating system stack is based on the principle of security by compartmentalization. The high level system stack is illustrated in the following diagram.
+
+image::diagrams/stack.svg[]
+
+=== Kernel space
+
+In the stack, kernel space security by compartmentalization is supported with linux kernel that includes kernel-based virtual machine (KVM) module enabling the kernel to work as virtual-machine manager, hypervisor. Kernel side hypervisor supports virtualization of hardware resources - computational cores, memory and devices - securely. Userspace virtual machine guests are managed with cloud-hypervisor. Linux with KVM also supports portability to several hardware architectures. Currently Spectrum is supported only on x86_64 but ARM64 is under works. In addition, hardened kernel is to be enabled.
+
+=== Host user space
+
+This section provides high level overview of host user space tools and libraries.
+
+User space stack is build on musl standard C library with added safety on resource exhaustion and security hardening on memory allocation.
+
+https://skarnet.org/software/s6-rc/overview.html[s6-rc] service manager is used for services. kmod, util-linux and busybox are provided for essential system administration.
+
+https://github.com/cloud-hypervisor/cloud-hypervisor[cloud-hypervisor] is a host tooling for virtual machine management, written in Rust with a strong focus on security.
+
+Wayland refers to whole display stack providing communication with compositor (weston) for desktop services, including libraries and drivers for direct rendering and event devices. Clients are implemented as application virtual machines (see next section). Minimal host provides only Wayland terminal client, foot. Wayland, a simpler and more secure, protocol for compositor could provide support for legacy X applications as well but as of now none are provided. https://wayland.freedesktop.org/architecture.html[Wayland architecture] is well documented here.
+
+=== Application and system virtual machines
+
+Security by compartmentalization in Spectrum is implemented with virtual machines. Virtual machines currently launch using terminal and support only wayland-console. Wayland graphics support for appvms is under work. Please refer to https://spectrum-os.org/doc/running-vms.html[running VMs] for more information.
+
+Reference set of virtual machines includes system machine, netvm, and application VMs, appvm-catgirl and appvm-elinks. Please refer to https://spectrum-os.org/doc/creating-vms.html[creating VMs] for more information.
+
+== Details of Spectrum dependency tree
+
+High level overview of Spectrum stack is limited view to the system. For detailed, interactive view to dependencies please use `nix-tree` under the spectrum repository:
+
+`nix-build img/live -I nixpkgs=https://spectrum-os.org/git/nixpkgs/snapshot/nixpkgs-rootfs.tar.gz --no-out-link | xargs -o nix-tree`
+
+https://diode.zone/w/8DBDQ6HQUe5UUdLkpDuL35[See video of Spectrum live image interactive analysis with nix-tree]
\ No newline at end of file
diff --git a/Documentation/default.nix b/Documentation/default.nix
index 02b3c31..8b969d4 100644
--- a/Documentation/default.nix
+++ b/Documentation/default.nix
@@ -3,7 +3,7 @@
{ pkgs ? import <nixpkgs> {} }: pkgs.callPackage (
-{ lib, runCommand, jekyll }:
+{ lib, runCommand, jekyll, drawio-headless }:
runCommand "spectrum-docs" {
src = with lib; cleanSourceWith {
@@ -14,11 +14,13 @@ runCommand "spectrum-docs" {
!(hasSuffix ".nix" name);
};
- nativeBuildInputs = [ jekyll ];
+ nativeBuildInputs = [ jekyll drawio-headless ];
passthru = { inherit jekyll; };
-} ''
+}
+ ''
jekyll build --disable-disk-cache -b /doc -s $src -d $out
+ drawio --recursive $out/diagrams/ --export -f svg $out/assets/images/
''
) {
jekyll = import ./jekyll.nix { inherit pkgs; };
diff --git a/Documentation/diagrams/stack.drawio b/Documentation/diagrams/stack.drawio
new file mode 100644
index 0000000..23feae7
--- /dev/null
+++ b/Documentation/diagrams/stack.drawio
@@ -0,0 +1 @@
+<mxfile host="Electron" modified="2022-05-24T12:19:30.186Z" agent="5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/15.7.3 Chrome/91.0.4472.164 Electron/13.6.1 Safari/537.36" etag="8D9FBMK4C5lWSHGzQpql" version="15.7.3" type="device"><diagram id="C5RBs43oDa-KdzZeNtuy" name="Page-1">7VrbcuI4EP0aqiYPTPkOPBIyJJtkdneSnUnmaUtggVWRLUeSueTrV7JljC8ZHArjhZoXbLdal+5z1FJLdMyRv7qmIPS+EhfijqG5q4551TEMXdcM8ZCStZJohp1I5hS5SpYJHtEbTBWVNEIuZDlFTgjmKMwLpyQI4JTnZIBSssyrzQjO9xqCOSwJHqcAl6VPyOVeIu0bvUx+A9HcS3vWnUFS4oNUWVnCPOCS5ZbI/NIxR5QQnrz5qxHE0nupX57+WD/h+xfn+vYbewXfL+/++fNHN2ls/JEqGxMoDPjeTU+e7/r61e34/vvPt8Xz8hX2//qmqmgLgCPlr47hYNHJ5YyIvoS/wTQpcF4jaejlA5kQTrJv6Ri+xnklWbnLYi4MhYJhhKvtGs5cPm8AdZfCJqHwadV3LtKehQlJ54macv6mD4OSKHChtEoTxUsPcfgYJsNcChILmcd9LL70Te2drlMuXkDK4WqLOMqV15D4kNO1UFGlG36reeFY6nuZkcx2lMzbIlgqA4rX803TGXbiRcH3ASj1FqG8R0G0io0Xthva3Y+vJw2m3W8bTKNFMKeYRG7XW4eQLhAj9KSgNPt5KC2tDKWhHRNKswTlHaRBvMyy2D1FPwpLed5ZjFPyAkcECzDMq4AEUGKJMC6IAEbzQHxOhVOhkF9KvyGxFg5VgY9cV3ZTiU4ePwn1GPgIS0cqciVStcZLFJJvEtG4HY9zsagbtuSWLXwmf6QC+zwnZI4hCBH7PCV+XDBlsep4lvQhXrNeGiNHYZoP7BI39KppbjbFDavEjRvC5PSOmMDvN0FaJojVa5sgdovrgB8xMT4H+BL7YMLkoyu0R0XhuwtEIhYkDt4fhYI3Hsem4Y5hmlbMOGVMXixV499adjAOAlfsNX8x7GSI+WE372CMJhRIkpzQ+lraKhltb5V6FVOk4DkYuEOZSmaBcMtTtaPawaNYMVa6gHnxQD4OIHRzWXAZvm14KtBJZRRiwNEinztXQaZ6+JugeJak7HhnH522wGIfqkrbuWqhHX2woyEO6BzyUkMxgTZW1+LUM354ewrGV7fL4XDQ5Xr47+T+IGnxR4LgzhDmdOl0R7xqIHyUwkJdQtYOH5ZTDh+DhqJHJdLl3Xm7SC/BGovl6hywtmukYrp+TLDL2+12wZ5EbD0hq3MA29F2T2y9d0ywD7F1PiTYM3lQHO9x48YmNC34JFIx/+IsSDDYPeP7x+SA8z/jwItPziK0m3Vmu3VMpAcHQHpfVCOOcBcnR+Clye2jAPkAX5SLTgpxy6qBeFN5XyXiepsXHkwkILy78E8Kw9Lmu2pDdtQ1Wm/zniOAPANwa8ayNVv4pz5fi1hvMvxtrKsyLd1uDOyqXOtsTmoKJzD7A3ikkxo7Rw9zMMi3UP+kZkdDDZ/U6IfI6faNIIgyhioiCAjD048gllYjglStFg1GkDavQ6DY4b2wc0Xb7u1Gu/LevEG0y9nbA5xBYaa8E9UWiPII4LLffTD1UABZydG/702bYk/h3tTUayYHe9ybis/sL3fJKpL9c9H88h8=</diagram></mxfile>
\ No newline at end of file
--
2.33.3
This will allow removing some workarounds (661dfd83478 and the
preceding commits) that I had to do to get aarch64 musl to bootstrap,
and hopefully will also fix some of the bootstrapping failures we've
been seeing since then on newer aarch64 hardware.
This approach unfortunately requires users to have the Spectrum binary
cache set up, but it's the best we can do for now. When it's
stabilized, we could use builtins.fetchClosure, which would avoid that
requirement. But hopefully by that point we can have the bootstrap
tools updated upstream anyway.
Signed-off-by: Alyssa Ross <hi(a)alyssa.is>
---
.../linux/bootstrap-files/aarch64-musl.nix | 16 +++++++---------
1 file changed, 7 insertions(+), 9 deletions(-)
diff --git a/pkgs/stdenv/linux/bootstrap-files/aarch64-musl.nix b/pkgs/stdenv/linux/bootstrap-files/aarch64-musl.nix
index 252783cce47..64c950529e9 100644
--- a/pkgs/stdenv/linux/bootstrap-files/aarch64-musl.nix
+++ b/pkgs/stdenv/linux/bootstrap-files/aarch64-musl.nix
@@ -1,11 +1,9 @@
+let
+ # This path is available on the Spectrum binary cache.
+ # See <https://spectrum-os.org/doc/binary-cache.html>.
+ build = builtins.storePath /nix/store/bq9z1m3dvcsk522j9li43gm61caj5ybr-stdenv-bootstrap-tools-aarch64-unknown-linux-musl;
+in
{
- busybox = import <nix/fetchurl.nix> {
- url = "https://wdtz.org/files/wjzsj9cmdkc70f78yh072483x8656nci-stdenv-bootstrap-to…";
- sha256 = "01s6bwq84wyrjh3rdsgxni9gkzp7ss8rghg0cmp8zd87l79y8y4g";
- executable = true;
- };
- bootstrapTools = import <nix/fetchurl.nix> {
- url = "https://wdtz.org/files/wjzsj9cmdkc70f78yh072483x8656nci-stdenv-bootstrap-to…";
- sha256 = "0pbqrw9z4ifkijpfpx15l2dzi00rq8c5zg9ghimz5qgr5dx7f7cl";
- };
+ busybox = "${build}/on-server/busybox";
+ bootstrapTools = "${build}/on-server/bootstrap-tools.tar.xz";
}
--
2.35.1
