patches and low-level development discussion
 help / color / mirror / code / Atom feed
* [RFC PATCH 00/10] Initial support for VM Wayland
@ 2022-09-30 12:49 Alyssa Ross
  2022-09-30 12:49 ` [RFC PATCH 01/10] host/start-vm: use MAP_SHARED memory for VMs Alyssa Ross
                   ` (9 more replies)
  0 siblings, 10 replies; 12+ messages in thread
From: Alyssa Ross @ 2022-09-30 12:49 UTC (permalink / raw)
  To: devel; +Cc: Puck Meerburg, Ville Ilvonen

IMPORTANT NOTE: this series should be applied on top of my previous
series "Introduce a shared base for application VMs" [1].  This is a
bit inconvenient I know, but I haven't committed the other series yet
due to wanting to take a second pass at it.

This series adds the Spectrum-side support for running VMs that can
display Wayland windows on the host compositor, using virtio-gpu.
There are various small things still to be resolved, but it's ready
for other people to try out and test.

The easiest way to test this out is to run "vm-start hello-wayland" on
the Spectrum host.  If everything goes well, a small window with a
picture of a cat (hello-wayland) should appear.  This window is
running in a VM.

One particular known issue is that on one run of the new
appvm-hello-wayland, I got this error:

	[12:11:52.107] libwayland: error in client communication (pid 863)

I'm not yet aware of what would cause such an error.  It went away
after I rebooted and tried again.  There are some other known issues
with my cloud-hypervisor patch too.  See the other thread for
information on those.

[1]: https://spectrum-os.org/lists/archives/spectrum-devel/20220919073659.1703271-1-hi@alyssa.is/
[2]: https://spectrum-os.org/lists/archives/spectrum-devel/20220928170128.1583791-1-alyssa.ross@unikie.com/

Alyssa Ross (10):
  host/start-vm: use MAP_SHARED memory for VMs
  img/app: don't block app startup on network online
  img/app: add Wayland over virtio-gpu support to kernel
  vm-lib: add mesa drivers to VM
  img/app: add support for testing virtio-gpu
  img/app: add support for testing in crosvm
  host/start-vm: factor out VM definition path
  host: add support for Wayland in VMs
  vm/app: add hello-wayland demo VM
  host/start-vm: disable cloud-hypervisor sandbox

 Documentation/creating-vms.adoc               |  5 +++
 host/initramfs/extfs.nix                      |  3 ++
 host/rootfs/Makefile                          |  4 +++
 host/rootfs/default.nix                       |  4 +--
 host/rootfs/etc/s6-rc/ext-rc-init/up          |  8 +++++
 host/rootfs/etc/template/gpu/data/check       |  5 +++
 host/rootfs/etc/template/gpu/notification-fd  |  1 +
 .../etc/template/gpu/notification-fd.license  |  2 ++
 host/rootfs/etc/template/gpu/run              |  9 ++++++
 host/rootfs/etc/template/gpu/type             |  1 +
 host/rootfs/etc/template/gpu/type.license     |  2 ++
 host/start-vm/start-vm.rs                     | 26 +++++++++------
 img/app/Makefile                              | 32 +++++++++++++++++--
 img/app/default.nix                           |  3 +-
 img/app/etc/mdev/iface                        |  2 +-
 img/app/etc/mdev/listen                       | 12 +++++++
 img/app/etc/mdev/wait                         | 15 +++++++++
 img/app/etc/s6-rc/ok-all/contents             |  1 +
 img/app/shell.nix                             |  2 +-
 vm-lib/make-vm.nix                            | 14 ++++++--
 vm/app/catgirl.nix                            |  1 +
 vm/app/hello-wayland.nix                      | 25 +++++++++++++++
 vm/app/lynx.nix                               |  1 +
 23 files changed, 159 insertions(+), 19 deletions(-)
 create mode 100755 host/rootfs/etc/template/gpu/data/check
 create mode 100644 host/rootfs/etc/template/gpu/notification-fd
 create mode 100644 host/rootfs/etc/template/gpu/notification-fd.license
 create mode 100755 host/rootfs/etc/template/gpu/run
 create mode 100644 host/rootfs/etc/template/gpu/type
 create mode 100644 host/rootfs/etc/template/gpu/type.license
 create mode 100755 img/app/etc/mdev/listen
 create mode 100755 img/app/etc/mdev/wait
 create mode 100644 vm/app/hello-wayland.nix

-- 
2.37.1



^ permalink raw reply	[flat|nested] 12+ messages in thread

* [RFC PATCH 01/10] host/start-vm: use MAP_SHARED memory for VMs
  2022-09-30 12:49 [RFC PATCH 00/10] Initial support for VM Wayland Alyssa Ross
@ 2022-09-30 12:49 ` Alyssa Ross
  2022-09-30 12:49 ` [RFC PATCH 02/10] img/app: don't block app startup on network online Alyssa Ross
                   ` (8 subsequent siblings)
  9 siblings, 0 replies; 12+ messages in thread
From: Alyssa Ross @ 2022-09-30 12:49 UTC (permalink / raw)
  To: devel; +Cc: Puck Meerburg, Ville Ilvonen

From: Alyssa Ross <hi@alyssa.is>

This is required for any use of vhost-user.  Since any VM might have a
vhost-user device hotplugged, this needs to be on from the start for
all VMs.

Signed-off-by: Alyssa Ross <alyssa.ross@unikie.com>
---
This patch comes from before I started working on this at Unikie,
hence the non-Unikie author email.

 host/start-vm/start-vm.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/host/start-vm/start-vm.rs b/host/start-vm/start-vm.rs
index 302c020..b2cccc8 100644
--- a/host/start-vm/start-vm.rs
+++ b/host/start-vm/start-vm.rs
@@ -30,7 +30,7 @@ fn vm_command(dir: PathBuf) -> Result<Command, String> {
     command.arg("cloud-hypervisor");
     command.args(&["--api-socket", "env/cloud-hypervisor.sock"]);
     command.args(&["--cmdline", "console=ttyS0 root=PARTLABEL=root"]);
-    command.args(&["--memory", "size=128M"]);
+    command.args(&["--memory", "size=128M,shared=on"]);
     command.args(&["--console", "pty"]);
 
     let mut net_providers_dir = PathBuf::new();
-- 
2.37.1



^ permalink raw reply	[flat|nested] 12+ messages in thread

* [RFC PATCH 02/10] img/app: don't block app startup on network online
  2022-09-30 12:49 [RFC PATCH 00/10] Initial support for VM Wayland Alyssa Ross
  2022-09-30 12:49 ` [RFC PATCH 01/10] host/start-vm: use MAP_SHARED memory for VMs Alyssa Ross
@ 2022-09-30 12:49 ` Alyssa Ross
  2022-09-30 12:49 ` [RFC PATCH 03/10] img/app: add Wayland over virtio-gpu support to kernel Alyssa Ross
                   ` (7 subsequent siblings)
  9 siblings, 0 replies; 12+ messages in thread
From: Alyssa Ross @ 2022-09-30 12:49 UTC (permalink / raw)
  To: devel; +Cc: Puck Meerburg, Ville Ilvonen

From: Alyssa Ross <hi@alyssa.is>

Not all VMs will even have networking!  So it makes more sense to put
waiting for the network in the application-specific part.

The "listen" and "wait" scripts are copied from the host system.

Signed-off-by: Alyssa Ross <alyssa.ross@unikie.com>
---
This patch comes from before I started working on this at Unikie,
hence the non-Unikie author email.

 img/app/Makefile                  |  2 ++
 img/app/etc/mdev/iface            |  2 +-
 img/app/etc/mdev/listen           | 12 ++++++++++++
 img/app/etc/mdev/wait             | 15 +++++++++++++++
 img/app/etc/s6-rc/ok-all/contents |  1 +
 vm/app/catgirl.nix                |  1 +
 vm/app/lynx.nix                   |  1 +
 7 files changed, 33 insertions(+), 1 deletion(-)
 create mode 100755 img/app/etc/mdev/listen
 create mode 100755 img/app/etc/mdev/wait

diff --git a/img/app/Makefile b/img/app/Makefile
index c5a4684..a3fefc3 100644
--- a/img/app/Makefile
+++ b/img/app/Makefile
@@ -47,6 +47,8 @@ VM_FILES = \
 	etc/init \
 	etc/mdev.conf \
 	etc/mdev/iface \
+	etc/mdev/listen \
+	etc/mdev/wait \
 	etc/passwd \
 	etc/resolv.conf \
 	etc/s6-linux-init/scripts/rc.init
diff --git a/img/app/etc/mdev/iface b/img/app/etc/mdev/iface
index d8ceda5..4cdc112 100755
--- a/img/app/etc/mdev/iface
+++ b/img/app/etc/mdev/iface
@@ -33,4 +33,4 @@ foreground {
   }
 }
 
-s6-rc -u change app
+/etc/mdev/listen network-online
diff --git a/img/app/etc/mdev/listen b/img/app/etc/mdev/listen
new file mode 100755
index 0000000..6bc6f2c
--- /dev/null
+++ b/img/app/etc/mdev/listen
@@ -0,0 +1,12 @@
+#!/bin/execlineb -s1
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2021 Alyssa Ross <hi@alyssa.is>
+
+foreground {
+  redirfd -w 2 /dev/null
+  foreground { mkdir /run/wait }
+  mkfifo /run/wait/${1}
+}
+
+redirfd -w 1 /run/wait/${1}
+echo
diff --git a/img/app/etc/mdev/wait b/img/app/etc/mdev/wait
new file mode 100755
index 0000000..3b85de8
--- /dev/null
+++ b/img/app/etc/mdev/wait
@@ -0,0 +1,15 @@
+#!/bin/execlineb -s1
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2021 Alyssa Ross <hi@alyssa.is>
+
+foreground {
+  redirfd -w 2 /dev/null
+  foreground { mkdir /run/wait }
+  mkfifo /run/wait/${1}
+}
+
+foreground {
+  redirfd -w 1 /dev/null
+  head -1 /run/wait/${1}
+}
+rm /run/wait/${1}
diff --git a/img/app/etc/s6-rc/ok-all/contents b/img/app/etc/s6-rc/ok-all/contents
index c4ea84f..92f3ef1 100644
--- a/img/app/etc/s6-rc/ok-all/contents
+++ b/img/app/etc/s6-rc/ok-all/contents
@@ -2,3 +2,4 @@
 # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi@alyssa.is>
 #
 mdevd-coldplug
+app
diff --git a/vm/app/catgirl.nix b/vm/app/catgirl.nix
index 107ff82..79c5d8b 100644
--- a/vm/app/catgirl.nix
+++ b/vm/app/catgirl.nix
@@ -10,6 +10,7 @@ import ../../vm-lib/make-vm.nix { inherit config; } {
     { writeScript, catgirl }:
     writeScript "run-catgirl" ''
       #!/bin/execlineb -P
+      if { /etc/mdev/wait network-online }
       foreground { printf "IRC nick (to join #spectrum): " }
       backtick -E nick { head -1 }
       ${catgirl}/bin/catgirl -h irc.libera.chat -j "#spectrum" -n $nick
diff --git a/vm/app/lynx.nix b/vm/app/lynx.nix
index 8fc3bd6..6abaf9b 100644
--- a/vm/app/lynx.nix
+++ b/vm/app/lynx.nix
@@ -10,6 +10,7 @@ import ../../vm-lib/make-vm.nix { inherit config; } {
     { writeScript, lynx }:
     writeScript "run-lynx" ''
       #!/bin/execlineb -P
+      if { /etc/mdev/wait network-online }
       ${lynx}/bin/lynx https://spectrum-os.org
     ''
   ) { };
-- 
2.37.1



^ permalink raw reply	[flat|nested] 12+ messages in thread

* [RFC PATCH 03/10] img/app: add Wayland over virtio-gpu support to kernel
  2022-09-30 12:49 [RFC PATCH 00/10] Initial support for VM Wayland Alyssa Ross
  2022-09-30 12:49 ` [RFC PATCH 01/10] host/start-vm: use MAP_SHARED memory for VMs Alyssa Ross
  2022-09-30 12:49 ` [RFC PATCH 02/10] img/app: don't block app startup on network online Alyssa Ross
@ 2022-09-30 12:49 ` Alyssa Ross
  2022-09-30 12:49 ` [RFC PATCH 04/10] vm-lib: add mesa drivers to VM Alyssa Ross
                   ` (6 subsequent siblings)
  9 siblings, 0 replies; 12+ messages in thread
From: Alyssa Ross @ 2022-09-30 12:49 UTC (permalink / raw)
  To: devel; +Cc: Puck Meerburg, Ville Ilvonen

The current LTS kernel does not support virtio-gpu context types, so
we have to switch to the latest kernel.

Signed-off-by: Alyssa Ross <alyssa.ross@unikie.com>
---
I haven't had time yet to figure out why it doesn't work if I let
DRM_VIRTIO_GPU default to "m" and modprobe it inside the VM.

 img/app/default.nix | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/img/app/default.nix b/img/app/default.nix
index e7d5366..a6734d2 100644
--- a/img/app/default.nix
+++ b/img/app/default.nix
@@ -49,12 +49,13 @@ let
         -T ${writeReferencesToFile packagesSysroot} .
   '';
 
-  kernel = buildPackages.linux.override {
+  kernel = buildPackages.linux_latest.override {
     structuredExtraConfig = with lib.kernel; {
       VIRTIO = yes;
       VIRTIO_PCI = yes;
       VIRTIO_BLK = yes;
       VIRTIO_CONSOLE = yes;
+      DRM_VIRTIO_GPU = yes;
       EXT4_FS = yes;
       DRM_BOCHS = yes;
       DRM = yes;
-- 
2.37.1



^ permalink raw reply	[flat|nested] 12+ messages in thread

* [RFC PATCH 04/10] vm-lib: add mesa drivers to VM
  2022-09-30 12:49 [RFC PATCH 00/10] Initial support for VM Wayland Alyssa Ross
                   ` (2 preceding siblings ...)
  2022-09-30 12:49 ` [RFC PATCH 03/10] img/app: add Wayland over virtio-gpu support to kernel Alyssa Ross
@ 2022-09-30 12:49 ` Alyssa Ross
  2022-09-30 12:49 ` [RFC PATCH 05/10] img/app: add support for testing virtio-gpu Alyssa Ross
                   ` (5 subsequent siblings)
  9 siblings, 0 replies; 12+ messages in thread
From: Alyssa Ross @ 2022-09-30 12:49 UTC (permalink / raw)
  To: devel; +Cc: Puck Meerburg, Ville Ilvonen

Signed-off-by: Alyssa Ross <alyssa.ross@unikie.com>
---

This is a dynamically-linked, Glibc version of Mesa.  Dynamic linking
is basically required for GUI stuff, but maybe it would be better to
do this with Musl in the end.  Not sure yet.  It depends exactly what
this shared base VM will be used for — will applications run it
directly, or inside some other layer like Flatpak or Docker?  How will
that interact with Mesa?

 vm-lib/make-vm.nix | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/vm-lib/make-vm.nix b/vm-lib/make-vm.nix
index 10646d3..20cdba4 100644
--- a/vm-lib/make-vm.nix
+++ b/vm-lib/make-vm.nix
@@ -29,9 +29,10 @@ runCommand "spectrum-vm-${name}" {
   mkdir root
   cd root
   ln -s ${run} run
-  comm -23 <(sort ${writeReferencesToFile run}) \
+  ln -s ${config.pkgs.mesa.drivers}/lib
+  comm -23 <(sort -u ${writeReferencesToFile run} ${writeReferencesToFile config.pkgs.mesa.drivers}) \
       <(sort ${writeReferencesToFile basePackages}) |
-      tar -cf ../run.tar --verbatim-files-from -T - run
+      tar -cf ../run.tar --verbatim-files-from -T - *
   tar2ext4 -i ../run.tar -o "$out/data/${name}/blk/run.img"
   e2label "$out/data/${name}/blk/run.img" ext
 
-- 
2.37.1



^ permalink raw reply	[flat|nested] 12+ messages in thread

* [RFC PATCH 05/10] img/app: add support for testing virtio-gpu
  2022-09-30 12:49 [RFC PATCH 00/10] Initial support for VM Wayland Alyssa Ross
                   ` (3 preceding siblings ...)
  2022-09-30 12:49 ` [RFC PATCH 04/10] vm-lib: add mesa drivers to VM Alyssa Ross
@ 2022-09-30 12:49 ` Alyssa Ross
  2022-09-30 13:03   ` Alyssa Ross
  2022-09-30 12:49 ` [RFC PATCH 06/10] img/app: add support for testing in crosvm Alyssa Ross
                   ` (4 subsequent siblings)
  9 siblings, 1 reply; 12+ messages in thread
From: Alyssa Ross @ 2022-09-30 12:49 UTC (permalink / raw)
  To: devel; +Cc: Puck Meerburg, Ville Ilvonen

crosvm sadly doesn't support socket activation or readiness
notification, hence the sleep loop here.

I've removed the tap device as it required privileges to set up, and
was difficult to actually test with since it wouldn't be set up
correctly on the host in the same way the Spectrum host would set it
up.

Signed-off-by: Alyssa Ross <alyssa.ross@unikie.com>
---
 img/app/Makefile | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/img/app/Makefile b/img/app/Makefile
index a3fefc3..145001c 100644
--- a/img/app/Makefile
+++ b/img/app/Makefile
@@ -6,6 +6,8 @@
 # QEMU_KVM = qemu-system-x86_64 -enable-kvm.
 QEMU_KVM = qemu-kvm
 CLOUD_HYPERVISOR = cloud-hypervisor
+CROSVM = crosvm
+CROSVM_DEVICE_GPU = $(CROSVM) device gpu
 
 prefix = /usr/local
 imgdir = $(prefix)/img
@@ -96,6 +98,17 @@ build/etc/s6-rc: $(VM_S6_RC_FILES)
 	    s6-rc-compile $@ $$dir; \
 	    exit=$$?; rm -r $$dir; exit $$exit
 
+start-vhost-user-gpu:
+	rm -f vhost-user-gpu.sock
+	$(CROSVM_DEVICE_GPU) \
+	    --socket vhost-user-gpu.sock \
+	    --wayland-sock "$$XDG_RUNTIME_DIR/$$WAYLAND_DISPLAY" \
+	    --params '{"context-types":"cross-domain"}' &
+	while ! [ -S vhost-user-gpu.sock ]; do
+	    sleep 1
+	done
+.PHONY: start-vhost-user-gpu
+
 run-qemu: build/host/appvm/blk/root.img
 	$(QEMU_KVM) -m 128 -cpu host -machine q35,kernel=$(KERNEL) -vga none \
 	  -drive file=build/host/appvm/blk/root.img,if=virtio,format=raw,readonly=on \
@@ -108,13 +121,14 @@ run-qemu: build/host/appvm/blk/root.img
 	  -device virtconsole,chardev=virtiocon0
 .PHONY: run-qemu
 
-run-cloud-hypervisor: build/host/appvm/blk/root.img
+run-cloud-hypervisor: build/host/appvm/blk/root.img start-vhost-user-gpu
 	$(CLOUD_HYPERVISOR) \
 	    --api-socket path=vmm.sock \
-	    --memory size=128M \
+	    --memory size=128M,shared=on \
 	    --disk path=build/host/appvm/blk/root.img,readonly=on \
 	           path=$(RUN_IMG),readonly=on \
 	    --net tap=tap0,mac=0A:B3:EC:00:00:00 \
+	    --gpu socket=vhost-user-gpu.sock \
 	    --kernel $(KERNEL) \
 	    --cmdline "console=ttyS0 root=PARTLABEL=root" \
 	    --console tty \
-- 
2.37.1



^ permalink raw reply	[flat|nested] 12+ messages in thread

* [RFC PATCH 06/10] img/app: add support for testing in crosvm
  2022-09-30 12:49 [RFC PATCH 00/10] Initial support for VM Wayland Alyssa Ross
                   ` (4 preceding siblings ...)
  2022-09-30 12:49 ` [RFC PATCH 05/10] img/app: add support for testing virtio-gpu Alyssa Ross
@ 2022-09-30 12:49 ` Alyssa Ross
  2022-09-30 12:49 ` [RFC PATCH 07/10] host/start-vm: factor out VM definition path Alyssa Ross
                   ` (3 subsequent siblings)
  9 siblings, 0 replies; 12+ messages in thread
From: Alyssa Ross @ 2022-09-30 12:49 UTC (permalink / raw)
  To: devel; +Cc: Puck Meerburg, Ville Ilvonen

This is useful because it allows comparing how our patched
cloud-hypervisor behaves against crosvm's implementation.

Signed-off-by: Alyssa Ross <alyssa.ross@unikie.com>
---
 img/app/Makefile  | 12 ++++++++++++
 img/app/shell.nix |  2 +-
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/img/app/Makefile b/img/app/Makefile
index 145001c..9f70032 100644
--- a/img/app/Makefile
+++ b/img/app/Makefile
@@ -8,6 +8,7 @@ QEMU_KVM = qemu-kvm
 CLOUD_HYPERVISOR = cloud-hypervisor
 CROSVM = crosvm
 CROSVM_DEVICE_GPU = $(CROSVM) device gpu
+CROSVM_RUN = $(CROSVM) run
 
 prefix = /usr/local
 imgdir = $(prefix)/img
@@ -135,6 +136,17 @@ run-cloud-hypervisor: build/host/appvm/blk/root.img start-vhost-user-gpu
 	    --serial pty
 .PHONY: run-cloud-hypervisor
 
+run-crosvm: build/host/appvm/blk/root.img start-vhost-user-gpu
+	$(CROSVM_RUN) \
+	    --disk build/host/appvm/blk/root.img \
+	    --disk $(RUN_IMG) \
+	    -p "console=ttyS0 root=PARTLABEL=root" \
+	    --vhost-user-gpu vhost-user-gpu.sock \
+	    --serial type=file,hardware=serial,path=/tmp/crosvm.log \
+	    --serial type=stdout,hardware=virtio-console,stdin=true \
+	    $(KERNEL)
+.PHONY: run-crosvm
+
 run: run-$(VMM)
 .PHONY: run
 
diff --git a/img/app/shell.nix b/img/app/shell.nix
index d6b627c..c9ed1a6 100644
--- a/img/app/shell.nix
+++ b/img/app/shell.nix
@@ -12,7 +12,7 @@ with config.pkgs;
 
 {
   nativeBuildInputs = nativeBuildInputs ++ [
-    cloud-hypervisor jq qemu_kvm reuse
+    cloud-hypervisor crosvm jq qemu_kvm reuse
   ];
 
   KERNEL = "${passthru.kernel.dev}/vmlinux";
-- 
2.37.1



^ permalink raw reply	[flat|nested] 12+ messages in thread

* [RFC PATCH 07/10] host/start-vm: factor out VM definition path
  2022-09-30 12:49 [RFC PATCH 00/10] Initial support for VM Wayland Alyssa Ross
                   ` (5 preceding siblings ...)
  2022-09-30 12:49 ` [RFC PATCH 06/10] img/app: add support for testing in crosvm Alyssa Ross
@ 2022-09-30 12:49 ` Alyssa Ross
  2022-09-30 12:49 ` [RFC PATCH 08/10] host: add support for Wayland in VMs Alyssa Ross
                   ` (2 subsequent siblings)
  9 siblings, 0 replies; 12+ messages in thread
From: Alyssa Ross @ 2022-09-30 12:49 UTC (permalink / raw)
  To: devel; +Cc: Puck Meerburg, Ville Ilvonen

Signed-off-by: Alyssa Ross <alyssa.ross@unikie.com>
---
 host/start-vm/start-vm.rs | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)

diff --git a/host/start-vm/start-vm.rs b/host/start-vm/start-vm.rs
index b2cccc8..41a4fbc 100644
--- a/host/start-vm/start-vm.rs
+++ b/host/start-vm/start-vm.rs
@@ -1,5 +1,6 @@
 // SPDX-License-Identifier: EUPL-1.2+
 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is>
+// SPDX-FileCopyrightText: 2022 Unikie
 
 mod ch;
 mod net;
@@ -33,11 +34,11 @@ fn vm_command(dir: PathBuf) -> Result<Command, String> {
     command.args(&["--memory", "size=128M,shared=on"]);
     command.args(&["--console", "pty"]);
 
-    let mut net_providers_dir = PathBuf::new();
-    net_providers_dir.push("/ext/svc/data");
-    net_providers_dir.push(vm_name);
-    net_providers_dir.push("providers/net");
+    let mut definition_path = PathBuf::new();
+    definition_path.push("/ext/svc/data");
+    definition_path.push(vm_name);
 
+    let net_providers_dir = definition_path.join("providers/net");
     match net_providers_dir.read_dir() {
         Ok(entries) => {
             for r in entries {
@@ -78,10 +79,7 @@ fn vm_command(dir: PathBuf) -> Result<Command, String> {
 
     command.arg("--disk");
 
-    let mut blk_dir = PathBuf::new();
-    blk_dir.push("/ext/svc/data");
-    blk_dir.push(vm_name);
-    blk_dir.push("blk");
+    let blk_dir = definition_path.join("blk");
     match blk_dir.read_dir() {
         Ok(entries) => {
             for result in entries {
-- 
2.37.1



^ permalink raw reply	[flat|nested] 12+ messages in thread

* [RFC PATCH 08/10] host: add support for Wayland in VMs
  2022-09-30 12:49 [RFC PATCH 00/10] Initial support for VM Wayland Alyssa Ross
                   ` (6 preceding siblings ...)
  2022-09-30 12:49 ` [RFC PATCH 07/10] host/start-vm: factor out VM definition path Alyssa Ross
@ 2022-09-30 12:49 ` Alyssa Ross
  2022-09-30 12:49 ` [RFC PATCH 09/10] vm/app: add hello-wayland demo VM Alyssa Ross
  2022-09-30 12:49 ` [RFC PATCH 10/10] host/start-vm: disable cloud-hypervisor sandbox Alyssa Ross
  9 siblings, 0 replies; 12+ messages in thread
From: Alyssa Ross @ 2022-09-30 12:49 UTC (permalink / raw)
  To: devel; +Cc: Puck Meerburg, Ville Ilvonen

When a VM is configured with Wayland support, the ext-rc-init service
will create an additional service to supervise the crosvm GPU backend,
and start-vm will pass the required arguments to cloud-hypervisor to
tell it how to connect to crosvm.

Signed-off-by: Alyssa Ross <alyssa.ross@unikie.com>
---

We're using the Glibc version of crosvm here, rather than getting it
from pkgsGui like we should be.  That's blocked on making
pkgsMusl.cargo work in Nixpkgs:

https://github.com/NixOS/nixpkgs/pull/190796

 Documentation/creating-vms.adoc                      | 5 +++++
 host/rootfs/Makefile                                 | 4 ++++
 host/rootfs/default.nix                              | 4 ++--
 host/rootfs/etc/s6-rc/ext-rc-init/up                 | 8 ++++++++
 host/rootfs/etc/template/gpu/data/check              | 5 +++++
 host/rootfs/etc/template/gpu/notification-fd         | 1 +
 host/rootfs/etc/template/gpu/notification-fd.license | 2 ++
 host/rootfs/etc/template/gpu/run                     | 9 +++++++++
 host/rootfs/etc/template/gpu/type                    | 1 +
 host/rootfs/etc/template/gpu/type.license            | 2 ++
 host/start-vm/start-vm.rs                            | 9 +++++++++
 vm-lib/make-vm.nix                                   | 9 ++++++++-
 12 files changed, 56 insertions(+), 3 deletions(-)
 create mode 100755 host/rootfs/etc/template/gpu/data/check
 create mode 100644 host/rootfs/etc/template/gpu/notification-fd
 create mode 100644 host/rootfs/etc/template/gpu/notification-fd.license
 create mode 100755 host/rootfs/etc/template/gpu/run
 create mode 100644 host/rootfs/etc/template/gpu/type
 create mode 100644 host/rootfs/etc/template/gpu/type.license

diff --git a/Documentation/creating-vms.adoc b/Documentation/creating-vms.adoc
index 6d4fde0..a4d5acf 100644
--- a/Documentation/creating-vms.adoc
+++ b/Documentation/creating-vms.adoc
@@ -2,6 +2,7 @@
 :page-parent: Reference
 
 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is>
+// SPDX-FileCopyrightText: 2022 Unikie
 // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0
 
 == Configuration
@@ -23,6 +24,10 @@ providers/net:: A directory containing a file named for each VM that
 should provide networking to this VM.  The contents of these files are
 ignored.
 
+wayland:: An empty file, whose presence indicates that the host should
+set up a virtio-gpu device supporting the cross-domain context type,
+for the VM to send Wayland messages over.
+
 === Example
 
 A configuration directory for a VM called "appvm-lynx" dedicated to
diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile
index 31f76d2..f0f6a4b 100644
--- a/host/rootfs/Makefile
+++ b/host/rootfs/Makefile
@@ -28,6 +28,10 @@ FILES = \
 	etc/mdev/wait \
 	etc/parse-devname \
 	etc/passwd \
+	etc/template/gpu/data/check \
+	etc/template/gpu/notification-fd \
+	etc/template/gpu/run \
+	etc/template/gpu/type \
 	etc/s6-linux-init/run-image/service/getty-tty1/run \
 	etc/s6-linux-init/run-image/service/getty-tty2/run \
 	etc/s6-linux-init/run-image/service/getty-tty3/run \
diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix
index 4788628..ad6ea1f 100644
--- a/host/rootfs/default.nix
+++ b/host/rootfs/default.nix
@@ -44,8 +44,8 @@ let
   foot = pkgsGui.foot.override { allowPgo = false; };
 
   packages = [
-    cloud-hypervisor execline jq kmod mdevd s6 s6-linux-init s6-rc socat
-    start-vm
+    cloud-hypervisor pkgs.crosvm execline jq kmod mdevd s6 s6-linux-init s6-rc
+    socat start-vm
 
     (cryptsetup.override {
       programs = {
diff --git a/host/rootfs/etc/s6-rc/ext-rc-init/up b/host/rootfs/etc/s6-rc/ext-rc-init/up
index 1aec7fb..2ab3f03 100644
--- a/host/rootfs/etc/s6-rc/ext-rc-init/up
+++ b/host/rootfs/etc/s6-rc/ext-rc-init/up
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: EUPL-1.2+
 # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2022 Unikie
 
 if { mkdir -p /run/s6-rc.ext.src }
 
@@ -15,6 +16,13 @@ if {
   if { redirfd -w 1 ${name}/notification-fd echo 3 }
   if { ln -s -- /bin/start-vm ${name}/run }
 
+  if {
+    if -t { test -e ${dir}/wayland }
+    if { cp -R /etc/template/gpu /run/s6-rc.ext.src/${name}-gpu }
+    if { mkdir /run/s6-rc.ext.src/${name}-gpu/env }
+    touch /run/s6-rc.ext.src/${name}/dependencies.d/${name}-gpu
+  }
+
   elglob -0 paths /ext/svc/data/${name}/providers/net/*
   forx -pE path { $paths }
   backtick -E dep { basename -- $path }
diff --git a/host/rootfs/etc/template/gpu/data/check b/host/rootfs/etc/template/gpu/data/check
new file mode 100755
index 0000000..868346b
--- /dev/null
+++ b/host/rootfs/etc/template/gpu/data/check
@@ -0,0 +1,5 @@
+#!/bin/execlineb -P
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: Unikie
+
+test -S env/crosvm.sock
diff --git a/host/rootfs/etc/template/gpu/notification-fd b/host/rootfs/etc/template/gpu/notification-fd
new file mode 100644
index 0000000..00750ed
--- /dev/null
+++ b/host/rootfs/etc/template/gpu/notification-fd
@@ -0,0 +1 @@
+3
diff --git a/host/rootfs/etc/template/gpu/notification-fd.license b/host/rootfs/etc/template/gpu/notification-fd.license
new file mode 100644
index 0000000..2241beb
--- /dev/null
+++ b/host/rootfs/etc/template/gpu/notification-fd.license
@@ -0,0 +1,2 @@
+SPDX-License-Identifier: CC0-1.0
+SPDX-FileCopyrightText: 2022 Unikie
diff --git a/host/rootfs/etc/template/gpu/run b/host/rootfs/etc/template/gpu/run
new file mode 100755
index 0000000..d1913dd
--- /dev/null
+++ b/host/rootfs/etc/template/gpu/run
@@ -0,0 +1,9 @@
+#!/bin/execlineb -P
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: Unikie
+
+s6-notifyoncheck -d
+crosvm --no-syslog device gpu
+  --socket env/crosvm.sock
+  --wayland-sock /run/user/0/wayland-1
+  --params "{\"context-types\": \"cross-domain\"}"
diff --git a/host/rootfs/etc/template/gpu/type b/host/rootfs/etc/template/gpu/type
new file mode 100644
index 0000000..5883cff
--- /dev/null
+++ b/host/rootfs/etc/template/gpu/type
@@ -0,0 +1 @@
+longrun
diff --git a/host/rootfs/etc/template/gpu/type.license b/host/rootfs/etc/template/gpu/type.license
new file mode 100644
index 0000000..2241beb
--- /dev/null
+++ b/host/rootfs/etc/template/gpu/type.license
@@ -0,0 +1,2 @@
+SPDX-License-Identifier: CC0-1.0
+SPDX-FileCopyrightText: 2022 Unikie
diff --git a/host/start-vm/start-vm.rs b/host/start-vm/start-vm.rs
index 41a4fbc..b954ebd 100644
--- a/host/start-vm/start-vm.rs
+++ b/host/start-vm/start-vm.rs
@@ -104,6 +104,15 @@ fn vm_command(dir: PathBuf) -> Result<Command, String> {
         Err(e) => return Err(format!("reading directory {:?}: {}", blk_dir, e)),
     }
 
+    if definition_path.join("wayland").exists() {
+        command.arg("--gpu").arg({
+            let mut gpu = OsString::from("socket=../");
+            gpu.push(vm_name);
+            gpu.push("-gpu/env/crosvm.sock");
+            gpu
+        });
+    }
+
     if command.get_args().last() == Some(OsStr::new("--disk")) {
         return Err("no block devices specified".to_string());
     }
diff --git a/vm-lib/make-vm.nix b/vm-lib/make-vm.nix
index 20cdba4..f595481 100644
--- a/vm-lib/make-vm.nix
+++ b/vm-lib/make-vm.nix
@@ -1,12 +1,13 @@
 # SPDX-License-Identifier: MIT
 # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2022 Unikie
 
 { config ? import ../nix/eval-config.nix {} }:
 config.pkgs.pkgsStatic.callPackage (
 
 { lib, runCommand, writeReferencesToFile, e2fsprogs, tar2ext4 }:
 
-{ name, run, providers ? {} }:
+{ name, run, providers ? {}, wayland ? false }:
 
 let
   inherit (lib)
@@ -20,6 +21,8 @@ assert !(any (hasInfix "\n") (concatLists (attrValues providers)));
 runCommand "spectrum-vm-${name}" {
   nativeBuildInputs = [ e2fsprogs tar2ext4 ];
 
+  inherit wayland;
+
   providerDirs = concatStrings (concatLists
     (mapAttrsToList (kind: map (vm: "${kind}/${vm}\n")) providers));
   passAsFile = [ "providerDirs" ];
@@ -41,6 +44,10 @@ runCommand "spectrum-vm-${name}" {
   xargs -rd '\n' touch -- < "$providerDirsPath"
   popd
 
+  if [ -n "$wayland" ]; then
+      touch "$out/data/${name}/wayland"
+  fi
+
   ln -s /usr/img/appvm/blk/root.img "$out/data/${name}/blk"
   ln -s /usr/img/appvm/vmlinux "$out/data/${name}"
 ''
-- 
2.37.1



^ permalink raw reply	[flat|nested] 12+ messages in thread

* [RFC PATCH 09/10] vm/app: add hello-wayland demo VM
  2022-09-30 12:49 [RFC PATCH 00/10] Initial support for VM Wayland Alyssa Ross
                   ` (7 preceding siblings ...)
  2022-09-30 12:49 ` [RFC PATCH 08/10] host: add support for Wayland in VMs Alyssa Ross
@ 2022-09-30 12:49 ` Alyssa Ross
  2022-09-30 12:49 ` [RFC PATCH 10/10] host/start-vm: disable cloud-hypervisor sandbox Alyssa Ross
  9 siblings, 0 replies; 12+ messages in thread
From: Alyssa Ross @ 2022-09-30 12:49 UTC (permalink / raw)
  To: devel; +Cc: Puck Meerburg, Ville Ilvonen

hello-wayland is the simplest possible Wayland client.  This VM is
used to demonstrate Spectrum's new Wayland capabilities.

Signed-off-by: Alyssa Ross <alyssa.ross@unikie.com>
---

More complex applications may not work yet.  We've had issues with
Sway hosts, where cloud-hypervisor will attempt to map the keymap
read/write, which isn't allowed.  I'm not sure if this is also the
case for Weston.  Testing them is the next thing on my todo list.

 host/initramfs/extfs.nix |  3 +++
 vm/app/hello-wayland.nix | 25 +++++++++++++++++++++++++
 2 files changed, 28 insertions(+)
 create mode 100644 vm/app/hello-wayland.nix

diff --git a/host/initramfs/extfs.nix b/host/initramfs/extfs.nix
index bfaaf17..3c4ac1c 100644
--- a/host/initramfs/extfs.nix
+++ b/host/initramfs/extfs.nix
@@ -10,6 +10,7 @@ let
   };
 
   appvm-catgirl = import ../../vm/app/catgirl.nix { inherit config; };
+  appvm-hello-wayland = import ../../vm/app/hello-wayland.nix { inherit config; };
   appvm-lynx = import ../../vm/app/lynx.nix { inherit config; };
 in
 
@@ -22,6 +23,8 @@ runCommand "ext.ext4" {
   chmod +w svc/data
   tar -C ${appvm-catgirl} -c data | tar -C svc -x
   chmod +w svc/data
+  tar -C ${appvm-hello-wayland} -c data | tar -C svc -x
+  chmod +w svc/data
   tar -C ${appvm-lynx} -c data | tar -C svc -x
 
   tar -cf ext.tar svc
diff --git a/vm/app/hello-wayland.nix b/vm/app/hello-wayland.nix
new file mode 100644
index 0000000..410220d
--- /dev/null
+++ b/vm/app/hello-wayland.nix
@@ -0,0 +1,25 @@
+# SPDX-License-Identifier: MIT
+# SPDX-FileCopyrightText: 2022 Unikie
+
+{ config ? import ../../../nix/eval-config.nix {} }:
+
+import ../../vm-lib/make-vm.nix { inherit config; } {
+  name = "appvm-hello-wayland";
+  wayland = true;
+  run = config.pkgs.callPackage (
+    { writeScript, hello-wayland, wayland-proxy-virtwl }:
+    writeScript "run-hello-wayland" ''
+      #!/bin/execlineb -P
+      if { modprobe virtio-gpu }
+      foreground { ln -ns /run/ext /run/opengl-driver }
+      foreground { mkdir /run/user }
+      foreground {
+        umask 077
+        mkdir /run/user/0
+      }
+      export XDG_RUNTIME_DIR /run/user/0
+      ${wayland-proxy-virtwl}/bin/wayland-proxy-virtwl --virtio-gpu
+      ${hello-wayland}/bin/hello-wayland
+    ''
+  ) { };
+}
-- 
2.37.1



^ permalink raw reply	[flat|nested] 12+ messages in thread

* [RFC PATCH 10/10] host/start-vm: disable cloud-hypervisor sandbox
  2022-09-30 12:49 [RFC PATCH 00/10] Initial support for VM Wayland Alyssa Ross
                   ` (8 preceding siblings ...)
  2022-09-30 12:49 ` [RFC PATCH 09/10] vm/app: add hello-wayland demo VM Alyssa Ross
@ 2022-09-30 12:49 ` Alyssa Ross
  9 siblings, 0 replies; 12+ messages in thread
From: Alyssa Ross @ 2022-09-30 12:49 UTC (permalink / raw)
  To: devel; +Cc: Puck Meerburg, Ville Ilvonen

The current version of my virtio-gpu patches for cloud-hypervisor
aren't compatible with sandboxing[1].  The next version of them will fix
this, which will allow this patch to be dropped.

[1]: https://spectrum-os.org/lists/archives/spectrum-devel/20220929085338.lazjtztmryniskz2@x220.qyliss.net/

Signed-off-by: Alyssa Ross <alyssa.ross@unikie.com>
---
 host/start-vm/start-vm.rs | 1 +
 1 file changed, 1 insertion(+)

diff --git a/host/start-vm/start-vm.rs b/host/start-vm/start-vm.rs
index b954ebd..f07711b 100644
--- a/host/start-vm/start-vm.rs
+++ b/host/start-vm/start-vm.rs
@@ -33,6 +33,7 @@ fn vm_command(dir: PathBuf) -> Result<Command, String> {
     command.args(&["--cmdline", "console=ttyS0 root=PARTLABEL=root"]);
     command.args(&["--memory", "size=128M,shared=on"]);
     command.args(&["--console", "pty"]);
+    command.args(&["--seccomp", "log"]);
 
     let mut definition_path = PathBuf::new();
     definition_path.push("/ext/svc/data");
-- 
2.37.1



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [RFC PATCH 05/10] img/app: add support for testing virtio-gpu
  2022-09-30 12:49 ` [RFC PATCH 05/10] img/app: add support for testing virtio-gpu Alyssa Ross
@ 2022-09-30 13:03   ` Alyssa Ross
  0 siblings, 0 replies; 12+ messages in thread
From: Alyssa Ross @ 2022-09-30 13:03 UTC (permalink / raw)
  To: devel; +Cc: Puck Meerburg, Ville Ilvonen

[-- Attachment #1: Type: text/plain, Size: 2027 bytes --]

Alyssa Ross <alyssa.ross@unikie.com> writes:

> diff --git a/img/app/Makefile b/img/app/Makefile
> index a3fefc3..145001c 100644
> --- a/img/app/Makefile
> +++ b/img/app/Makefile
> @@ -6,6 +6,8 @@
>  # QEMU_KVM = qemu-system-x86_64 -enable-kvm.
>  QEMU_KVM = qemu-kvm
>  CLOUD_HYPERVISOR = cloud-hypervisor
> +CROSVM = crosvm
> +CROSVM_DEVICE_GPU = $(CROSVM) device gpu
>  
>  prefix = /usr/local
>  imgdir = $(prefix)/img
> @@ -96,6 +98,17 @@ build/etc/s6-rc: $(VM_S6_RC_FILES)
>  	    s6-rc-compile $@ $$dir; \
>  	    exit=$$?; rm -r $$dir; exit $$exit
>  
> +start-vhost-user-gpu:
> +	rm -f vhost-user-gpu.sock
> +	$(CROSVM_DEVICE_GPU) \
> +	    --socket vhost-user-gpu.sock \
> +	    --wayland-sock "$$XDG_RUNTIME_DIR/$$WAYLAND_DISPLAY" \
> +	    --params '{"context-types":"cross-domain"}' &
> +	while ! [ -S vhost-user-gpu.sock ]; do
> +	    sleep 1
> +	done

Known issue: this is a syntax error, due to missing backslashes.  (I
tried to fix this up from a single sleep without a loop in a bit of a
hurry.)

> +.PHONY: start-vhost-user-gpu
> +
>  run-qemu: build/host/appvm/blk/root.img
>  	$(QEMU_KVM) -m 128 -cpu host -machine q35,kernel=$(KERNEL) -vga none \
>  	  -drive file=build/host/appvm/blk/root.img,if=virtio,format=raw,readonly=on \
> @@ -108,13 +121,14 @@ run-qemu: build/host/appvm/blk/root.img
>  	  -device virtconsole,chardev=virtiocon0
>  .PHONY: run-qemu
>  
> -run-cloud-hypervisor: build/host/appvm/blk/root.img
> +run-cloud-hypervisor: build/host/appvm/blk/root.img start-vhost-user-gpu
>  	$(CLOUD_HYPERVISOR) \
>  	    --api-socket path=vmm.sock \
> -	    --memory size=128M \
> +	    --memory size=128M,shared=on \
>  	    --disk path=build/host/appvm/blk/root.img,readonly=on \
>  	           path=$(RUN_IMG),readonly=on \
>  	    --net tap=tap0,mac=0A:B3:EC:00:00:00 \
> +	    --gpu socket=vhost-user-gpu.sock \
>  	    --kernel $(KERNEL) \
>  	    --cmdline "console=ttyS0 root=PARTLABEL=root" \
>  	    --console tty \
> -- 
> 2.37.1

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2022-09-30 13:03 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-30 12:49 [RFC PATCH 00/10] Initial support for VM Wayland Alyssa Ross
2022-09-30 12:49 ` [RFC PATCH 01/10] host/start-vm: use MAP_SHARED memory for VMs Alyssa Ross
2022-09-30 12:49 ` [RFC PATCH 02/10] img/app: don't block app startup on network online Alyssa Ross
2022-09-30 12:49 ` [RFC PATCH 03/10] img/app: add Wayland over virtio-gpu support to kernel Alyssa Ross
2022-09-30 12:49 ` [RFC PATCH 04/10] vm-lib: add mesa drivers to VM Alyssa Ross
2022-09-30 12:49 ` [RFC PATCH 05/10] img/app: add support for testing virtio-gpu Alyssa Ross
2022-09-30 13:03   ` Alyssa Ross
2022-09-30 12:49 ` [RFC PATCH 06/10] img/app: add support for testing in crosvm Alyssa Ross
2022-09-30 12:49 ` [RFC PATCH 07/10] host/start-vm: factor out VM definition path Alyssa Ross
2022-09-30 12:49 ` [RFC PATCH 08/10] host: add support for Wayland in VMs Alyssa Ross
2022-09-30 12:49 ` [RFC PATCH 09/10] vm/app: add hello-wayland demo VM Alyssa Ross
2022-09-30 12:49 ` [RFC PATCH 10/10] host/start-vm: disable cloud-hypervisor sandbox Alyssa Ross

Code repositories for project(s) associated with this public inbox

	https://spectrum-os.org/git/crosvm
	https://spectrum-os.org/git/doc
	https://spectrum-os.org/git/mktuntap
	https://spectrum-os.org/git/nixpkgs
	https://spectrum-os.org/git/spectrum
	https://spectrum-os.org/git/ucspi-vsock
	https://spectrum-os.org/git/www

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).