patches and low-level development discussion
 help / color / mirror / code / Atom feed
* [PATCH 0/6] Introduce a shared base for application VMs
@ 2022-09-19  7:36 Alyssa Ross
  2022-09-19  7:36 ` [PATCH 1/6] host/start-vm: support multiple block devices Alyssa Ross
                   ` (5 more replies)
  0 siblings, 6 replies; 7+ messages in thread
From: Alyssa Ross @ 2022-09-19  7:36 UTC (permalink / raw)
  To: devel

Here are some changes I made months ago to enable work I never finished
around building VMs inside the Spectrum system with Nix.  I'm coming
back to it now because it makes it much easier to add more
applications for testing, and with upcoming features like Wayland and
a persistent filesystem we're going to want more example applications.

The idea here is to reduce duplication between application VMs, both
in terms of source code size and output size.  After this change,
creating a new VM just requires writing a very small Nix file like
this:

	{ config ? import ../../../nix/eval-config.nix {} }:

	import ../../vm-lib/make-vm.nix { inherit config; } {
	  name = "appvm-lynx";
	  providers.net = [ "netvm" ];
	  run = config.pkgs.pkgsStatic.callPackage (
	    { writeScript, lynx }:
	    writeScript "run-lynx" ''
	      #!/bin/execlineb -P
	      ${lynx}/bin/lynx https://spectrum-os.org
	    ''
	  ) { };
	}

Rather than a whole big source tree as before, most of which was
duplicated with every other application VM.

When a VM generated this way is started, it gets two disk images.  One
is the shared base image, which is part of the Spectrum base system,
and the other contains only the application-specific stuff: the run
script, and any store path dependencies that are not already present
in the base image.  This means that the amount of storage required for
each new application VM is substantially reduced.

Of course, this isn't the only way to generate VMs.  Monolithic VMs
like we had before would still work, with some small adjustments for
the new disk layout.

I also see this fitting well into making it possible to configure
extra VMs at build time.  It doesn't directly help with that, but
making it so that each VM doesn't need to provide everything itself
will make creating external VMs easier when it does happen.

In future we might want to apply a similar mechanism to service VMs,
like netvm, but since we only have one of those so far, it's not clear
which parts exactly would be duplicated, so I'm leaving it for now.

Other future work is considering the impacts of the shared base image
on guest isolation.  Can guests observe whether reads of the shared
base image hit the host page cache, or even an internal disk cache?
At the moment I suspect that the base image doesn't have enough
specialised code in it that there would be any interesting results,
but it's worth thinking about if the shared image grows new
functionality, whether it would be interesting to another guest to
be able to observe whether those resources have previously been loaded
or not.  If this _does_ turn out to be a concern, it could be
mitigated by simply copying the base image to temporary storage before
booting a VM, and then booting the VM from the copy.

Alyssa Ross (6):
  host/start-vm: support multiple block devices
  scripts/make-gpt.sh: add support for labels
  vm: build GPT images
  host/start-vm: boot using partition label
  release: rename from "img"
  img/app: extract from appvm-{lynx,catgirl}

 Documentation/creating-vms.adoc               |   8 +-
 Documentation/getting-spectrum.adoc           |   2 +-
 host/initramfs/extfs.nix                      |  11 +-
 host/rootfs/default.nix                       |  11 +-
 host/start-vm/start-vm.rs                     |  43 ++++--
 {vm/app/lynx => img/app}/Makefile             |  57 ++++----
 {vm/app/catgirl => img/app}/bin               |   0
 {vm/app/catgirl => img/app}/default.nix       |  22 ++--
 img/app/etc/fstab                             |   8 ++
 {vm/app/catgirl => img/app}/etc/init          |   0
 {vm/app/catgirl => img/app}/etc/mdev.conf     |   0
 {vm/app/lynx => img/app}/etc/mdev/iface       |   2 +-
 {vm/app/catgirl => img/app}/etc/passwd        |   0
 .../catgirl => img/app}/etc/passwd.license    |   0
 {vm/app/catgirl => img/app}/etc/resolv.conf   |   0
 .../app}/etc/s6-linux-init/scripts/rc.init    |   1 +
 .../s6-rc/lynx => img/app/etc/s6-rc/app}/run  |   3 +-
 .../catgirl => img/app/etc/s6-rc/app}/type    |   0
 .../app/etc/s6-rc/app}/type.license           |   0
 .../etc/s6-rc/mdevd-coldplug/dependencies     |   0
 .../app}/etc/s6-rc/mdevd-coldplug/type        |   0
 .../etc/s6-rc/mdevd-coldplug/type.license     |   0
 .../app}/etc/s6-rc/mdevd-coldplug/up          |   0
 .../app}/etc/s6-rc/mdevd/notification-fd      |   0
 .../etc/s6-rc/mdevd/notification-fd.license   |   0
 .../catgirl => img/app}/etc/s6-rc/mdevd/run   |   0
 .../catgirl => img/app}/etc/s6-rc/mdevd/type  |   0
 .../app}/etc/s6-rc/mdevd/type.license         |   0
 .../app}/etc/s6-rc/ok-all/contents            |   0
 .../catgirl => img/app}/etc/s6-rc/ok-all/type |   0
 .../app}/etc/s6-rc/ok-all/type.license        |   0
 .../app}/etc/ssl/certs/ca-certificates.crt    |   0
 {vm/app/lynx => img/app}/shell.nix            |  11 +-
 release.nix                                   |   2 +-
 {img => release}/combined/default.nix         |   0
 {img => release}/combined/eosimages.nix       |   0
 {img => release}/combined/grub.cfg.in         |   0
 {img => release}/combined/run-vm.nix          |   0
 ...ble-gpt-partition-attribute-55-check.patch |   0
 ...pt-disable-partition-table-CRC-check.patch |   0
 .../0003-install-remove-Endless-OS-ad.patch   |   0
 ...4-finished-don-t-run-eos-diagnostics.patch |   0
 ...omote-spectrum-not-the-Endless-forum.patch |   0
 {img => release}/installer/app/default.nix    |   0
 .../installer/app/vendor-customer-support.ini |   0
 {img => release}/installer/configuration.nix  |   0
 {img => release}/installer/default.nix        |   0
 {img => release}/installer/run-vm.nix         |   0
 {img => release}/installer/seat.rules         |   0
 {img => release}/live/Makefile                |   0
 {img => release}/live/default.nix             |   0
 {img => release}/live/shell.nix               |   0
 scripts/make-gpt.sh                           |   4 +-
 scripts/sfdisk-field.awk                      |   2 +-
 vm-lib/make-vm.nix                            |  46 +++++++
 vm/app/catgirl.nix                            |  18 +++
 vm/app/catgirl/Makefile                       | 123 ------------------
 vm/app/catgirl/etc/fstab                      |   6 -
 vm/app/catgirl/etc/mdev/iface                 |  36 -----
 .../catgirl/etc/s6-linux-init/scripts/rc.init |  10 --
 vm/app/catgirl/etc/s6-rc/catgirl/run          |  31 -----
 .../data/appvm-catgirl/providers/net/netvm    |   0
 vm/app/catgirl/shell.nix                      |  17 ---
 vm/app/lynx.nix                               |  16 +++
 vm/app/lynx/bin                               |   1 -
 vm/app/lynx/default.nix                       |  92 -------------
 vm/app/lynx/etc/fstab                         |   6 -
 vm/app/lynx/etc/init                          |   5 -
 vm/app/lynx/etc/mdev.conf                     |   5 -
 vm/app/lynx/etc/passwd                        |   1 -
 vm/app/lynx/etc/passwd.license                |   2 -
 vm/app/lynx/etc/resolv.conf                   |   4 -
 vm/app/lynx/etc/s6-rc/lynx/type               |   1 -
 vm/app/lynx/etc/s6-rc/lynx/type.license       |   2 -
 .../etc/s6-rc/mdevd-coldplug/dependencies     |   4 -
 vm/app/lynx/etc/s6-rc/mdevd-coldplug/type     |   1 -
 .../etc/s6-rc/mdevd-coldplug/type.license     |   2 -
 vm/app/lynx/etc/s6-rc/mdevd-coldplug/up       |   4 -
 vm/app/lynx/etc/s6-rc/mdevd/notification-fd   |   1 -
 .../etc/s6-rc/mdevd/notification-fd.license   |   2 -
 vm/app/lynx/etc/s6-rc/mdevd/run               |   5 -
 vm/app/lynx/etc/s6-rc/mdevd/type              |   1 -
 vm/app/lynx/etc/s6-rc/mdevd/type.license      |   2 -
 vm/app/lynx/etc/s6-rc/ok-all/contents         |   4 -
 vm/app/lynx/etc/s6-rc/ok-all/type             |   1 -
 vm/app/lynx/etc/s6-rc/ok-all/type.license     |   2 -
 vm/app/lynx/etc/ssl/certs/ca-certificates.crt |   1 -
 .../host/data/appvm-lynx/providers/net/netvm  |   0
 vm/sys/net/Makefile                           |  23 ++--
 vm/sys/net/default.nix                        |  10 +-
 90 files changed, 221 insertions(+), 451 deletions(-)
 rename {vm/app/lynx => img/app}/Makefile (66%)
 rename {vm/app/catgirl => img/app}/bin (100%)
 rename {vm/app/catgirl => img/app}/default.nix (77%)
 create mode 100644 img/app/etc/fstab
 rename {vm/app/catgirl => img/app}/etc/init (100%)
 rename {vm/app/catgirl => img/app}/etc/mdev.conf (100%)
 rename {vm/app/lynx => img/app}/etc/mdev/iface (98%)
 rename {vm/app/catgirl => img/app}/etc/passwd (100%)
 rename {vm/app/catgirl => img/app}/etc/passwd.license (100%)
 rename {vm/app/catgirl => img/app}/etc/resolv.conf (100%)
 rename {vm/app/lynx => img/app}/etc/s6-linux-init/scripts/rc.init (90%)
 rename {vm/app/lynx/etc/s6-rc/lynx => img/app/etc/s6-rc/app}/run (80%)
 rename {vm/app/catgirl/etc/s6-rc/catgirl => img/app/etc/s6-rc/app}/type (100%)
 rename {vm/app/catgirl/etc/s6-rc/catgirl => img/app/etc/s6-rc/app}/type.license (100%)
 rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/dependencies (100%)
 rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/type (100%)
 rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/type.license (100%)
 rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/up (100%)
 rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/notification-fd (100%)
 rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/notification-fd.license (100%)
 rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/run (100%)
 rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/type (100%)
 rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/type.license (100%)
 rename {vm/app/catgirl => img/app}/etc/s6-rc/ok-all/contents (100%)
 rename {vm/app/catgirl => img/app}/etc/s6-rc/ok-all/type (100%)
 rename {vm/app/catgirl => img/app}/etc/s6-rc/ok-all/type.license (100%)
 rename {vm/app/catgirl => img/app}/etc/ssl/certs/ca-certificates.crt (100%)
 rename {vm/app/lynx => img/app}/shell.nix (50%)
 rename {img => release}/combined/default.nix (100%)
 rename {img => release}/combined/eosimages.nix (100%)
 rename {img => release}/combined/grub.cfg.in (100%)
 rename {img => release}/combined/run-vm.nix (100%)
 rename {img => release}/installer/app/0001-gpt-disable-gpt-partition-attribute-55-check.patch (100%)
 rename {img => release}/installer/app/0002-gpt-disable-partition-table-CRC-check.patch (100%)
 rename {img => release}/installer/app/0003-install-remove-Endless-OS-ad.patch (100%)
 rename {img => release}/installer/app/0004-finished-don-t-run-eos-diagnostics.patch (100%)
 rename {img => release}/installer/app/0005-finished-promote-spectrum-not-the-Endless-forum.patch (100%)
 rename {img => release}/installer/app/default.nix (100%)
 rename {img => release}/installer/app/vendor-customer-support.ini (100%)
 rename {img => release}/installer/configuration.nix (100%)
 rename {img => release}/installer/default.nix (100%)
 rename {img => release}/installer/run-vm.nix (100%)
 rename {img => release}/installer/seat.rules (100%)
 rename {img => release}/live/Makefile (100%)
 rename {img => release}/live/default.nix (100%)
 rename {img => release}/live/shell.nix (100%)
 create mode 100644 vm-lib/make-vm.nix
 create mode 100644 vm/app/catgirl.nix
 delete mode 100644 vm/app/catgirl/Makefile
 delete mode 100644 vm/app/catgirl/etc/fstab
 delete mode 100755 vm/app/catgirl/etc/mdev/iface
 delete mode 100755 vm/app/catgirl/etc/s6-linux-init/scripts/rc.init
 delete mode 100755 vm/app/catgirl/etc/s6-rc/catgirl/run
 delete mode 100644 vm/app/catgirl/host/data/appvm-catgirl/providers/net/netvm
 delete mode 100644 vm/app/catgirl/shell.nix
 create mode 100644 vm/app/lynx.nix
 delete mode 120000 vm/app/lynx/bin
 delete mode 100644 vm/app/lynx/default.nix
 delete mode 100644 vm/app/lynx/etc/fstab
 delete mode 100755 vm/app/lynx/etc/init
 delete mode 100644 vm/app/lynx/etc/mdev.conf
 delete mode 100644 vm/app/lynx/etc/passwd
 delete mode 100644 vm/app/lynx/etc/passwd.license
 delete mode 100644 vm/app/lynx/etc/resolv.conf
 delete mode 100644 vm/app/lynx/etc/s6-rc/lynx/type
 delete mode 100644 vm/app/lynx/etc/s6-rc/lynx/type.license
 delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/dependencies
 delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/type
 delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/type.license
 delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/up
 delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/notification-fd
 delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/notification-fd.license
 delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/run
 delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/type
 delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/type.license
 delete mode 100644 vm/app/lynx/etc/s6-rc/ok-all/contents
 delete mode 100644 vm/app/lynx/etc/s6-rc/ok-all/type
 delete mode 100644 vm/app/lynx/etc/s6-rc/ok-all/type.license
 delete mode 120000 vm/app/lynx/etc/ssl/certs/ca-certificates.crt
 delete mode 100644 vm/app/lynx/host/data/appvm-lynx/providers/net/netvm


base-commit: 2cc5bf637f944ec950f7a8162018f421e6671315
-- 
2.37.1



^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2022-09-19  7:38 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-19  7:36 [PATCH 0/6] Introduce a shared base for application VMs Alyssa Ross
2022-09-19  7:36 ` [PATCH 1/6] host/start-vm: support multiple block devices Alyssa Ross
2022-09-19  7:36 ` [PATCH 2/6] scripts/make-gpt.sh: add support for labels Alyssa Ross
2022-09-19  7:36 ` [PATCH 3/6] vm: build GPT images Alyssa Ross
2022-09-19  7:36 ` [PATCH 4/6] host/start-vm: boot using partition label Alyssa Ross
2022-09-19  7:36 ` [PATCH 5/6] release: rename from "img" Alyssa Ross
2022-09-19  7:37 ` [PATCH 6/6] img/app: extract from appvm-{lynx,catgirl} Alyssa Ross

Code repositories for project(s) associated with this public inbox

	https://spectrum-os.org/git/crosvm
	https://spectrum-os.org/git/doc
	https://spectrum-os.org/git/mktuntap
	https://spectrum-os.org/git/nixpkgs
	https://spectrum-os.org/git/spectrum
	https://spectrum-os.org/git/ucspi-vsock
	https://spectrum-os.org/git/www

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).