- Spectrum Discuss - spectrum-os.org

This (Last) Week in Spectrum, 2021-W30

by Alyssa Ross

Hi, it's been a long time since I've done one of these, but so much interesting stuff happened this week, and there's been a bit of renewed interest in the project, that it _really_ needed an update. So here it is. As always when I'm not keeping up with TWiS, this update is limited to only things that happened in the last week, because otherwise it would just be far too long. Wayland ------- A problem I've been thinking about for a long time is Wayland access control. The Linux desktop ecosystem is moving towards access controls for most functionality through xdg-desktop-portal[1] (which sadly doesn't seem to have a website). But it doesn't cover stuff that's part of the core Wayland protocol, like access to the clipboard. And some compositors (wlroots-based ones) provide extra Wayland protocols for things like screenshots. And up to now, Wayland compositors haven't really done any authorization for these potentially dangerous APIs. (All of this is only really meaningful for sandboxed applications -- running in Flatpak or a Spectrum VM or something. Linux isn't really set up to do separation between processes running as the same user without namespaces.) So earlier this week, I posted[2] to the Wayland mailing list with an idea I'd been thinking about for a while[3], which was to place a proxy program in front of the Wayland compositor, that would intercept client->compositor requests and handle access control. I was quickly convinced that a proxy wasn't a good idea, but there was a lot of discussion, and it was really helpful to me figuring out what the right way to do it might be. There are really two problems to be solved here, one of which I hadn't even thought much about. The first is securely identifying a Wayland client. A compositor needs to be able to form the question "Should client X be able to do this?", and to do that it needs to be able to identify a client as client X, and know if it tries to interact with client X later, that it'll be talking to the same client. In a non-virtualized system, the obvious way to do this would be getting the pid of the client from the connection socket and then looking it up in proc(5) to find out executable path, but this approach is fundamentally insecure[4]. The way forward here (and one that would work for Spectrum) appears to be the proposed Security Contexts protocol[5], which would allow a sandbox implementation to provide a security context identifier for a client before handing off the Wayland connection to the client. Once the security context had been set up, it wouldn't be allowed to be modified, so once the sandboxed client was given the connection, it wouldn't be able to change the security context identifier. In Spectrum, the security context identifier here would likely be a unique, user-provided name for the VM, and the security context setup would be done by the virtio wayland implementation in the VMM. The second part of this puzzle is how the compositor should decide whether a client should be allowed to perform a particular operation, like a paste or going fullscreen (which is risky because it might spoof a desktop). There was actually a previous attempt at this a long time ago. libwsm[7] (short for Wayland Security Module) was a library that compositors would have integrated to make authorization decisions. But it wasn't adopted by compositors, and it some things we now know[8] to be bad ideas, like setting policy based on the executable path. It also made compositors responsible for any sort of authorization UI. In my opinion, it's better to have that done by the external piece, so that compositors have as little work to do as possible and therefore authorization is implemented as widely as possible. The compositor could implement an authorization system entirely on its own, but this would be a lot of code for each compositor to write, and it would limit the user to whatever permissions system the compositor came up with, which might not be able to accommodate their needs. (An example of this would be a Spectrum user that wanted to allow pasting between two applications, but not allow pasting between either of those and any other application.) It could also be implemented by having the compositors integrate with something like polkit[6], but compositor authors are reluctant to integrate directly with a single system like that, and even polkit might not support everything that would be desirable in an authorization system. (For example, it might be nice to implement libwsm's concept of a soft allow, where an action is permitted, but a notification is shown so the user is aware it's happened.) So a third solution, suggested by ifreund, a Wayland compositor author, in the Spectrum IRC channel, would be to have a privileged Wayland client that the compositor could ask authorization questions to, with a new protocol. The compositor would know that this particular client should be privileged because it either wouldn't be in a security context, or would be given a special security context identifier known to the compositor ahead of time. Then, implementations of this protocol could do authorization however they wanted, with the only limitation being the questions the compositor was programmed to ask them. I think this is a good way forward, but it'd be important to discuss with more compositor authors before getting to excited about it. Next steps from here with Wayland are: * Figure out what needs to happen to move the security contexts proposal forward. If it needs an experimental implementation, maybe we could help with that? * Inquire about the authorization protocol idea, and see how other compositor offers would feel about it. If there's a generally positive reaction, figure out how to move forward with it. [1]: https://github.com/flatpak/xdg-desktop-portal/ [2]: https://lists.freedesktop.org/archives/wayland-devel/2021-July/041896.html [3]: https://spectrum-os.org/lists/archives/spectrum-discuss/8735ueudel.fsf@alys… [4]: https://gitlab.freedesktop.org/wayland/weston/-/issues/206#note_176699 [5]: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/68 [6]: https://www.freedesktop.org/software/polkit/docs/latest/polkit.8.html [7]: https://github.com/mupuf/libwsm [8]: https://lore.kernel.org/lkml/87r1xplsku.fsf@x220.int.ebiederm.org/ Nixpkgs ------- I created a branch in Spectrum's nixpkgs repository[9] for the long-overdue merge with upstream, did the merge, and posted some patches[10] to spectrum-devel that fix all the builds that broke as a result. I've been letting them sit for a few days hoping for a review. Once that's done, it's time for another chromiumOSPackages upgrade, but that should be pretty easy this time because we're only one version behind. [9]: https://spectrum-os.org/git/nixpkgs [10]: https://spectrum-os.org/lists/archives/spectrum-devel/20210729100928.196534… Thanks for keeping up with Spectrum. :)

3 days, 15 hours

1
0
0 0

July 2021 Get a professional animation video to explain your business , products or ideas.

by Paul Joseph

Hey , I'm an animation ad producer with a lot of experience. I've worked on about 450 unique commercials for a variety of customers. I thought you might like to see a completed sample animation from my portfolio: https://www.youtube.com/watch?v=NaSGqazpMZU As a designer I specialize in innovative content. I would be happy to show what I can do for your organization too. You can find out more about my brand by doing a web search for "Social Lobster Animation". Look forward to hearing from you, Paul Joseph +1-215-352-3354 Oxford Avenue, Philadelphia, PA Senior Developer Please respond with "unsubscribe" if uninterested.

3 weeks, 2 days

1
0
0 0

#spectrum IRC channel has moved to Libera.Chat

by Alyssa Ross

The Spectrum IRC channel is now #spectrum on irc.libera.chat. It can also be joined through the Matrix bridge as #spectrum:libera.chat. I understand there are some teething problems with the Matrix bridge, but hopefully they should be resolved soon. Apologies for the inconvenience.

2 months

1
0
0 0

I'm on a panel at the NGI Forum tomorrow

by Alyssa Ross

Tommorow (2021-05-19) at 7:30 UTC, I'm going to be part of a panel discussion at the NGI Forum. NGI is the EU initiative that funds my work on Spectrum. The topic of the panel is "Internet of Trust", and it'll be streamed here: https://2021.ngiforum.eu/ There's a bit more information about the panel in the agenda here: https://prod5.assets-cdn.io/event/6431/assets/8379007230-30f8d8b6fb.pdf

2 months, 1 week

3
4
1 0

Proxying Wayland for untrusted clients

by Alyssa Ross

I've been thinking a lot about this for a while, thanks to conversations with Thomas and Puck, CCed here. I think it's time to put it into words properly and start working towards making it happen. One of the benefits that Wayland is supposed to have over X11 is security. A Wayland application isn't supposed to be able to record the screen without user permission, for example. But in most compositors, it can, with no restrictions. Existing Wayland compositors are monolithic, and each one would have to implement its own access controls. (Mutter already does this to some extent, at least for screen sharing, I believe.) The popular Wayland compositors are largely focused on being feature-complete reimplementations of their X11 equivalents, and so taking advantage of the security features and access controls the Wayland protocol makes possible hasn't been a priority for them. Additionally, every popular Wayland compositor is written in a memory-unsafe language, and this combined with the complexity of the Wayland protocol, with all the extensions involved, presents a serious concern to applications of Wayland that involve untrusted clients. To solve these problems, I propose a proxy program that sits between Wayland clients and the compositor, in the same privelege domain as the compositor. The proxy would decode and re-encode every Wayland request (client->compositor message), and would discard any request it didn't understand. This would mitigate the problem of a large, privileged program written in a memory-unsafe language being exposed to untrusted inputs. Additionally, the proxy would support a plugin interface, through which the user of the proxy (or their distributor) could configure custom behaviour. This could be used to prompt the user for confirmation before allowing a screen capture request, or even to implement a similar thing for e.g. clipboard access, for which there is no support in the Wayland protocol. It could even be used to modify surfaces, to implement things like Qubes-style unspoofable coloured window borders. This approach would allow permissions systems and other custom Wayland behaviour to be implemented in a compositor-independent manner. Distributions which suppor tseveral compositors could implement customisations in a single place, and users of compositors which lack security features and the assurances memory-safety can provide against untrusted input would gain access to those things. I'd like to hear feedback here, but I think early in the life of this idea we should also reach out to the broader Wayland community. I think there's a lot of potential for this idea beyond Spectrum, and it would be great if it could be something developed with input from a big breadth of Wayland users. If we can do that, it might be sensible for it to live at freedesktop.org? I'm not sure how that works. Let me know what you all think. :)

2 months, 1 week

4
8
0 0

Re: Proxying Wayland for untrusted clients

by Josh DuBois

On May 22, 2021, at 8:05 AM, Alyssa Ross <hi(a)alyssa.is> wrote: > > One of the benefits that Wayland is supposed to have over X11 is > security. A Wayland application isn't supposed to be able to record the > screen without user permission, for example. But in most compositors, > it can, with no restrictions. <snip> > > To solve these problems, I propose a proxy program that sits between > Wayland clients and the compositor, in the same privelege domain as the > compositor. <snip> > If we can do that, it might be sensible for > it to live at freedesktop.org? I'm not sure how that works. I am curious, if you have time, to hear more on why the approach of a proxy vs picking a compositor and implementing security there. If the problem is that the Wayland community so far has not considered security a priority, it seems that a security proxy may suffer from those same forces. Basically, will it be easier to attract developers or gain widespread adoption of a proxy as opposed to getting buy-in to do security directly in a compositor? You mention writing in a memory safe language and having a compositor neutral solution as technical advantages. Do you think a proxy is a good choice primarily because it can achieve a better technical result, or is the choice of a new component more a matter of difficulty getting community buy-in from a popular compositor and doing security there? How would you weigh the upsides of a new project against the difficulties of getting a new thing off the ground and adopted? (This is really just curiosity on my part and my $0.02 from the outside. You may have already had a lot of discussions about that, or even already tried talking to compositor folk and not gotten traction. Seems worth some explicit consideration.)

2 months, 2 weeks

2
1
0 0

[DEMO] virtio-vhost-user between QEMU and crosvm

by Alyssa Ross

Introduction ------------ Virtio-vhost-user[1] is a promising virtualisation technology that allows virtual devices that are exposed to VMs to themselves be implemented in VMs. Let's break down its name a bit to understand how it works: * Virtio[2] is a standard driver interface for virtualisation. Interfaces are available for all sorts of types of virtual devices, e.g. virtio-net, virtio-blk, and virtio-scsi. Typically, virtio devices are implemented by a virtual machine monitor (VMM). * Vhost[3] is kernelspace implementation of virtio virtual devices, created for their performance benefit. Instead of implementing the virtual devices itself, the VMM talks to the kernel implementation of them using a special ioctl protocol. * Vhost-user[4] allows another process to implement the vhost protocol, instead of the kernel, by using a UNIX socket instead of ioctls on a special character device. This doesn't provide the raw performance of vhost, but it serves a different purpose -- it allows virtual devices to be implemented by external programs, in a standardised way so they're portable between VMMs. Virtio-vhost-user allows the program implementing the virtual device to run in a VM of its own, by having the VMM for that VM create the vhost-user socket, and transferring messages over it to its guest using virtio. This is exciting for Spectrum, because it would mean that the host system doesn't have to interact with physical hardware directly beyond the PCI level, and can instead pass it through to a VM, which is responsible for implementing the virtual device backed by that physical hardware, which can be exposed to other VMs. Last year I spent a while looking into virtio-vhost-user[5][6][7]. It's a long way from being ready to use, and it seems to be maturing very slowly. It might be useful to us eventually for driver isolation, or something else might come along. My conclusion from my research was that we should decide later, once the ecosystem has had a chance to develop. But I wanted something to come out of the research I did anyway, and so I've prepared a demonstration. [1]: https://wiki.qemu.org/Features/VirtioVhostUser [2]: https://docs.oasis-open.org/virtio/virtio/v1.1/virtio-v1.1.html [3]: https://blog.vmsplice.net/2011/09/qemu-internals-vhost-architecture.html [4]: https://qemu.readthedocs.io/en/latest/interop/vhost-user.html [5]: https://spectrum-os.org/lists/archives/spectrum-devel/87pn8rezqn.fsf@alyssa… [6]: https://spectrum-os.org/lists/archives/spectrum-devel/87blk1pwph.fsf@alyssa… [7]: https://spectrum-os.org/lists/archives/spectrum-devel/87wo2glkg0.fsf@alyssa… What the demo does ------------------ Using any sort of non-host-based virtual device implementation is going to have to start with taking the virtual device implementation out of the VMM running an application VM, and vhost-user the clear solution to this. Vhost-user isn't supported by crosvm -- its focus is on doing all the virtualisation required by Chromium OS, so there's no need for it to allow other programs to provide virtual device implementations. So another part of the research I did was to try to port the vhost-user implementation from cloud-hypervisor to crosvm, which I was able to do successfully[8]. This has implications beyond vhost-user, and even beyond crosvm, because it demonstrates that it's practical to port features between rust-vmm[9] VMMs, which means we don't have to worry about finding one that provides every feature we need (which is just as well, because there isn't one). So here I demonstrate not just a "standard" virtio-vhost-user setup (to the extent that such a thing can exist at this early stage), but also that my patched crosvm with vhost-user support is capable of interoperating with the experimental virtio-vhost-user implementation for QEMU, because both are speaking the standardised vhost-user protocol. The demo sets up two VMs. One is run with my patched crosvm, and expects a virtual ethernet device to be provided by a vhost-user socket. When it boots, it brings up its network interface, tries to run a DHCP client, and then exits. The other VM is run with Nikos Dragazis and Stefan Hajnoczi's experimental virtio-vhost-user implementation in QEMU[10]. It gets a standard virtual ethernet device (backed by a TAP device on the host), and the virtio-vhost-user device hooked up to the socket that crosvm will be connecting to. Inside the VM, a userspace networking stack (DPDK, again modified to support virtio-vhost-user by Nikos Dragazis[11]) implements the device side of virtio-vhost-user, and forwards packets sent by crosvm's guest to the virtual ethernet device backed by the host TAP. +-----------------------------------------------------------------------------+ | | | +------------------------+ +------------------------+ | | | | | | | | | +----------------+ | | +----------------+ | | | | | | | | | | | | | +-----+ | | +--------+ | | | | | | | | | TAP +------+---+---+ DPDK +---+---+------+---+ | | | | +-----+ | | +--------+ | | | | | | | | | | | | | | | | | | | | Linux | | | | Linux | | | | | +----------------+ | | +----------------+ | | | | | | | | | | QEMU | | crosvm | | | +------------------------+ +------------------------+ | | | | Linux | +-----------------------------------------------------------------------------+ A complicating factor is that the virtio-vhost-user implementation for DPDK only supports outgoing traffic[12]. So packets coming from crosvm will be relayed to the TAP, but not the other way around. This means that we can't just use ping inside the crosvm VM to verify that the connection is working. Instead, we have to tcpdump on the host and verify that the packets the DHCP client inside the crosvm VM is sending are arriving on the TAP. For this to be useful for our intended purpose of isolating drivers for physical devices, we'd pass through the device here rather than using a TAP. It would otherwise work exactly the same, but it's more difficult to test it's working correctly. (I have tested it though -- for the first version of this I got working last year, I verified it worked by checking the logs of my local network's DHCP server.) [8]: https://spectrum-os.org/lists/archives/spectrum-devel/20210512170812.192540… [9]: https://github.com/rust-vmm [10]: https://github.com/ndragazis/qemu/tree/virtio-vhost-user [11]: https://github.com/ndragazis/dpdk-next-virtio/tree/virtio-vhost-user [12]: https://github.com/ndragazis/dpdk-next-virtio/blob/2d60e63/drivers/virtio_v… Running the demo ---------------- First, create a TAP device for QEMU to use: # ip tuntap add qemutap mode tap # ip link set qemutap up Start tcpdump, so we can see if packets arrive on the TAP: # tcpdump -i qemutap Start the QEMU VM: $ $(nix-build -A qemuVm /path/to/demo.nix) When you see "Press enter to exit", DPDK is ready to receive a virtio-vhost-user connection. Start the crosvm VM: $ $(nix-build -A crosvmVm /path/to/demo.nix) Once that VM boots, you should see some "BOOTP/DHCP" lines in the tcpdump output. This demonstrates that traffic from the crosvm guest has been relayed over virtio-vhost-user to DPDK, and then to the TAP on the host over virtio-net. You'll want to press enter to shut down the QEMU VM now, because DPDK pegs a CPU core (for reasons[*] unrelated to virtio-vhost-user that are out of scope here). Then you can remove the TAP device: # ip link delete qemutap Nix expression for the demo --------------------------- # SPDX-License-Identifier: MIT OR Apache-2.0 # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> let pinned = builtins.fetchTarball { url = "https://github.com/NixOS/nixpkgs/tarball/b14062b75c4e8ef4dd4110282f7105be87…"; sha256 = "1hzs0w6pcwwbzl2gkqyk46yrzizzm03mph4kggws02a6vlwphsib"; }; in { pkgs ? import pinned {} }: with pkgs; rec { linux = pkgs.linux.override { structuredExtraConfig = with lib.kernel; { "9P_FS" = yes; NET_9P = yes; NET_9P_VIRTIO = yes; PACKET = yes; VFIO = yes; VFIO_NOIOMMU = yes; VFIO_PCI = yes; VIRTIO_NET = yes; VIRTIO_PCI = yes; }; }; dpdk = stdenv.mkDerivation { name = "dpdk-virtio-vhost-user"; src = fetchFromGitHub { owner = "ndragazis"; repo = "dpdk-next-virtio"; rev = "0a46582dc1d02c0dc5069347ffff1a64239385f2"; sha256 = "169cxdps9k764jj420q44262x3291h2jcqsbrh7038hqjczjkgif"; }; buildInputs = [ numactl ]; configurePhase = '' runHook preConfigure make $makeFlags defconfig runHook postConfigure ''; enableParallelBuilding = true; RTE_KERNELDIR = "${linux.dev}/lib/modules/${linux.modDirVersion}/build"; NIX_CFLAGS_COMPILE = [ "-Wno-error=implicit-fallthrough" "-Wno-error=incompatible-pointer-types" ]; makeFlags = [ "RTE_OUTPUT=$(out)/lib" "kerneldir=$(out)/lib/modules/${linux.modDirVersion}/build" "prefix=$(out)" ]; inherit (pkgs.dpdk) meta; }; # DPDK is huge! We just need one program from it. testpmd = runCommandNoCC "testpmd" {} '' mkdir -p $out/bin cp ${dpdk}/bin/testpmd $out/bin ''; # qemu has changed build system since the virtio-vhost-user branch # was last updated, so it's simpler to just make a new derivation # and inherit the bits that are the same than to override the # existing one. qemu = stdenv.mkDerivation { name = "qemu-virtio-vhost-user"; src = fetchFromGitHub { owner = "ndragazis"; repo = "qemu"; rev = "f9ab08c0c8cfc58036ed95b895f9780397448071"; sha256 = "0p6v4i7gj70d6x7s28x3i3x9z8vlswcbbqdwfbhlx87bbnxjrn3b"; fetchSubmodules = true; }; enableParallelBuilding = true; nativeBuildInputs = lib.subtractLists [ ninja meson ] qemu_kvm.nativeBuildInputs; postPatch = '' sed -i '/$(INSTALL_DIR) "$(DESTDIR)$(qemu_localstatedir)/d' Makefile # The virtio-vhost-user implementation tries to allocate a huge # PCI bar, that's bigger than some CPUs can support! If you see # a kernel panic in vp_reset(), lower this further. substituteInPlace hw/virtio/virtio-vhost-user-pci.c \ --replace '1ULL << 36' '1ULL << 34' ''; inherit (qemu_kvm) buildInputs configureFlags meta; }; qemuInitramfs = makeInitrd { contents = [ { symlink = "/init"; object = writeScript "init" '' #!${busybox}/bin/sh -eux export PATH=${busybox}/bin mkdir -p /nix/store /run /var mount -t sysfs none /sys mount -t proc none /proc mount -t tmpfs none /run mount -t devtmpfs none /dev mkdir /dev/hugepages mount -t hugetlbfs none /dev/hugepages ln -s /run /var # Unbind the virtio-net (host TAP) and virtio-vhost-user devices # from their default drivers, since we'll be passing them # through to DPDK. echo 0000:00:04.0 > /sys/bus/pci/devices/0000:00:04.0/driver/unbind echo 0000:00:05.0 > /sys/bus/pci/devices/0000:00:05.0/driver/unbind # Tell the vfio-pci driver it can support virtio-net and # virtio-vhost-user devices. Since our devices are not # bound to any driver at the moment, doing this will bind # them to vfio-pci automatically. echo 1af4 1000 > /sys/bus/pci/drivers/vfio-pci/new_id echo 1af4 1017 > /sys/bus/pci/drivers/vfio-pci/new_id echo 256 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages ${testpmd}/bin/testpmd \ -l 0-1 \ -w 0000:00:05.0 \ --vdev net_vhost0,iface=0000:00:05.0,virtio-transport=1 \ -w 0000:00:04.0 poweroff -f ''; } ]; }; qemuVm = writeShellScript "qemu-vm" '' exec ${qemu}/bin/qemu-system-x86_64 -enable-kvm -cpu host -smp 2 -m 1G \ -M q35,kernel-irqchip=split \ -initrd ${qemuInitramfs}/initrd \ -netdev tap,id=net0,ifname=qemutap,script=no,downscript=no \ -device virtio-net-pci,netdev=net0,addr=04 \ -chardev socket,id=chardev0,path="$XDG_RUNTIME_DIR/vhost-user0.sock",server,nowait \ -device virtio-vhost-user-pci,addr=05,chardev=chardev0 \ -kernel ${linux}/${stdenv.hostPlatform.linux-kernel.target} \ -append "console=ttyS0 vfio.enable_unsafe_noiommu_mode=1" \ -nographic ''; # Can't use overrideAttrs because of cargoSha256. crosvm = rustPlatform.buildRustPackage rec { name = "crosvm-virtio-vhost-user"; src = fetchFromGitiles { url = "https://chromium.googlesource.com/chromiumos/platform/crosvm"; rev = "8a7e4e902a4950b060ea23b40c0dfce7bfa1b2cb"; sha256 = "1lm6psp0xakb66nhgmmh94valc4wzbb967chk80msk8bcvsfpdn4"; }; unpackPhase = let origSrc = pkgs.crosvm.passthru.src; in builtins.replaceStrings [ "${origSrc}" origSrc.name ] [ "$src" src.name ] pkgs.crosvm.unpackPhase; cargoPatches = [ (fetchpatch { url = "https://spectrum-os.org/lists/archives/spectrum-devel/20210512170812.192540…"; sha256 = "0yzqrpgq35s9wxvbf9s3dgs5cpyxgdc5hr14hsdjr0gd18a6camg"; }) ]; patches = pkgs.crosvm.patches ++ [ (fetchpatch { url = "https://spectrum-os.org/lists/archives/spectrum-devel/20210512170812.192540…"; sha256 = "0g2rvqqa4lvq7bjq0s1ynsjx7lmrxql7lsdv8wyzb7d2z9j6mj13"; }) (fetchpatch { url = "https://spectrum-os.org/lists/archives/spectrum-devel/20210512170812.192540…"; sha256 = "051sz87i8kzc5sbygk2bpiqp4g32y9fxswg2yax1nd3lg4rxh43r"; }) (fetchpatch { url = "https://spectrum-os.org/lists/archives/spectrum-devel/20210512170812.192540…"; sha256 = "1jpas65masn2xg9jxha16vi0y7scarzhl221y9wxh4chi4aa4m3f"; }) ]; cargoSha256 = "07yizbhs64jrb05fq5g7sx812xbz2989bsficacq5l19ziax5164"; passthru = pkgs.crosvm.passthru // { inherit src; }; inherit (pkgs.crosvm) sourceRoot postPatch nativeBuildInputs buildInputs preBuild postInstall CROSVM_CARGO_TEST_KERNEL_BINARY meta; }; crosvmInitramfs = makeInitrd { contents = [ { symlink = "/init"; object = writeScript "init" '' #!${busybox}/bin/sh -eux export PATH=${busybox}/bin mount -t sysfs none /sys mount -t proc none /proc ip link set eth0 up udhcpc -n || : reboot -f ''; } ]; }; crosvmVm = writeShellScript "crosvm-vm" '' # In our patched crosvm, suppling --mac without --host_ip or # --netmask will put it into vhost-user mode. exec ${crosvm}/bin/crosvm run \ --mac 0A:B3:EC:FF:FF:FF \ -i ${crosvmInitramfs}/initrd \ ${linux}/${stdenv.hostPlatform.linux-kernel.target} ''; }

2 months, 3 weeks

1
0
0 0

Git server issues?

by Jamie McClymont

Hello, Cloning the spectrum nixpkgs repo seems to consistently fail, after downloading all the objects, with: fetch-pack: unexpected disconnect while reading sideband packet fatal: early EOF fatal: fetch-pack: invalid index-pack output Have confirmed this across a couple of networks. Relatedly, the git web interface seems to always 504 Gateway Timeout while a clone is in progress. I wonder if the git server is configured to only handle one thread/request at a time, and the clone process involves 2 concurrent requests? (either that, or the server is actually overloaded). fyi, as a workaround: clone nixpkgs from elsewhere, then add the spectrum nixpkgs remote and fetch - it will also be much faster :) Thanks - Jamie McClymont

4 months

2
1
0 0

This Week in Spectrum, 2021-W10

by Alyssa Ross

Surprise! I sent the last This Week in Spectrum in August. Since then, it's been pretty quiet. What happened? Towards then end of last year, I was aggressively pushing to achieve a funding milestone, but was starting to struggle to keep myself going. After that last TWiS, I decided to stop posting them for a while to fully concentrate on the push. I continued like that for two more months, before getting to the point where I was so burned out I just couldn't keep going, with the milestone still not done, the work I had done so far on it unpublished (and not in a good state to be published), and time running out. Fortunately, I was able to talk to NLnet, and they agreed to extend my funding deadlines and renegotiate the milestones. With that sorted out, in late November I began a hiatus from Spectrum to allow myself to recover from the burnout. I'm very grateful to my GitHub Sponsors, thanks to whom I was able to take this time to recover. I tried to come back at the end of January, but after a single day of Spectrum work I was feeling much the same as I had been in November. This week, though, something changed! I've been working on Spectrum again for the last week, and I've been feeling great about it! Now, there were three months of work between the last TWiS in August, and the start of my hiatus. That's a lot of work I haven't written about. But I think the only way this is really going to work is if I leave that largely unwritten about for now, because TWiS tends to be thousands of words long even when it's covering a single week of work. So I'm just going to talk about what I've done in the last week, explaining context from previous work as necessary. With all that in mind, here we go! "Qubes-lite with KVM and Wayland"[1] ------------------------------------ First up is something somebody else did, which is always nice! Thomas has been doing his own great work replacing his Qubes system with a system built on NixOS, using KVM and Wayland. Thomas got in touch on the Spectrum discuss mailing list in January[2] asking some questions about how some things were implemented, and we went back and forth a bit then (although not as much as I'd have liked, due to my burnout :/). Since then, he's come up with some awesome stuff, like an OCaml implementation of a virtio-wayland proxy (equivalent to Sommelier in Chromium OS). In our further conversation prompted by Thomas's article, Thomas raised a further interesting idea[3] -- what if we had a Sommelier-like Wayland proxy, but that ran in the same security domain as the compositor. That way, we could filter to allow only the protocol extensions we want to. We could implement our own permissions when the compositor doesn't. This code would be responsible for securely handling Wayland messages from an untrusted guest, which would be beneficial because it would be a lot less code to audit, and because it could be written in a memory-safe language. This would avoid the need to write or find a secure Wayland compositor, and would in fact allow us to use virtually any compositor without having to worry nearly as much about its security credentials. I'm very excited by that idea, and I've been asking myself for the past few days why I didn't think of it before! I've thought several times "it's quite nice that Sommelier gives us a known subset of Wayland, but it's a shame we can't rely on it since it runs inside a application guest". But for some reason it just never occurred to me to follow that idea one step further, to "what if we had a similar program that run in the compositor's security domain". [1]: https://roscidus.com/blog/blog/2021/03/07/qubes-lite-with-kvm-and-wayland/ [2]: https://spectrum-os.org/lists/archives/spectrum-discuss/CAG4opy_hz_ESEpY0Tq… [3]: https://spectrum-os.org/lists/archives/spectrum-discuss/CAG4opy_JztAH3tD+ZU… uscpi-vsock ----------- ucspi-vsock[4] is a program I started writing in September. VSOCK[5] is a special Linux socket type that allows for communication between VMs, and I'm using it to allow services running in VMs (a USBIP daemon, for example), to notify the host (and by extension other VMs), when the service in the VM becomes ready to accept connections. USCPI[6] is a standard command interface for making it easy to do socket communication using standard IO primitives, without having to teach programs how to do socket connections for every kind of socket. There are several UCSPI implementations for common socket types like Internet and Unix domain sockets, but until now there was not one for VSOCK. So, for example, in a VM we can do: vsockclient 2 $port sh -c 'echo >&7' Which will connect to the host over VSOCK (in VSOCK the host is always address 2), write a newline, and close the connection. UCSPI lets us implement this with standard shell tools. There's an equivalent "vsockserver" program for the other end, that will run a command whenever a connection is received. Anyway, this week I went back to ucspi-vsock to fix some bugs[7][8][9][10]. As usual, (although I'm happy to realise this is still the case after all these months!) Cole was kind enough to review my patches. [4]: https://spectrum-os.org/git/ucspi-vsock/ [5]: https://man7.org/linux/man-pages/man7/vsock.7.html [6]: https://cr.yp.to/proto/ucspi.txt [7]: https://spectrum-os.org/lists/archives/spectrum-devel/20210309154048.14474-… [8]: https://spectrum-os.org/lists/archives/spectrum-devel/20210309171816.8589-1… [9]: https://spectrum-os.org/lists/archives/spectrum-devel/20210310204516.20041-… [10]: https://spectrum-os.org/lists/archives/spectrum-devel/20210310204555.20725-… Interguest networking --------------------- The bulk of my work this week has been in Spectrum's Nixpkgs, which is where (for now, at least) all the VM definitions and stuff live. Mostly, I've been trying to clean up months of frantic work into something I can actually publish as a series of patches, so it's out there instead of just on my computer! My focus at the moment is on doing this with the interguest networking implementation I wrote last year. The plan for this remains the same as last year: have a VM that manages all network hardware access and acts as a router for other VMs that need to talk to the outside world. Eventually, this will hopefully use virtio-vhost-user[11], which will allow us to avoid going through a networking stack on the host at all (or even having one), but until virtio-vhost-user is further along, we'll use bridge devices on the host to connect client VMs to the router. A very basic version of the latter is implemented and working for me locally -- I can run an application VM, and have it connect to another VM which runs the drivers for my ethernet port using VFIO (PCI passthrough). This code was a big ball of mud, and I'd also subtly broken it when I moved on from interguest networking last year to try to get a proof of concept going for another form of interguest communication. So I spent this week trying to page all the context I'd lost in the last few months back into my brain, making lots of partial git commits, writing commit messages, and fixing it up so that it works again. I think what I have is about ready to publish -- hopefully you'll see some patches next week. But what exists at the moment is still very limited -- the biggest limitation is that the router VM doesn't support hotplugging, so a VM that wants to connect to it actually has to be started first, before the router VM. Not good for something that should be run as a system service and likely started on boot! To overcome this limitation, I'll probably have to add support in crosvm for adding network devices at runtime using the control socket. I think this should be easy enough, and I haven't looked at what's happened in crosvm + rust-vmm since I've been away, so it's even possible somebody else will have done this already. [11]: https://wiki.qemu.org/Features/VirtioVhostUser So, the todo list for next week is getting the initial interguest networking PoC patchset posted. That'll come with a nice little demo other people will be able to try out if they want to. And then, I'll work on improving it further to the point where it's actually practical. I don't want to spend /too/ much time on this, since ultimately I do want to be using a virtio-vhost-user stack, which is quite different, but it's important to have an implementation of this so there's something to test further development (that doesn't care about the implementation details) against. My biggest worry at the moment is burning out again. Working in moderation isn't easy for me (I tend to get very sucked into things), but it's important for the project that I find a way to keep my work sustainable, so we don't lose time like this again. I'm still a little hesitant to say "this is it, I'm back for real". But the signs are looking good. Thank you for sticking with me through this.

4 months, 3 weeks

1
0
0 0

New user getting started questions

by Thomas Leonard

Hi, I've been running Qubes for a few years now and I'd like to give Spectrum a try, as I've been having some hardware and performance problems with Qubes. Is there some up-to-date guide I can follow? I found https://alyssa.is/using-virtio-wl/#demo and was able to see the weston terminal. I also tried updating to the latest commit and was able to get a nested wayfire window with: nix-build . -A spectrumPackages && ./result-3/bin/spectrum-vm (I'm fairly new to Nix, so not sure if this is the right way to do things) I managed to change the keyboard layout, mount a tmpfs for home, and increase the memory enough to start firefox, but I haven't managed to get much further. Things I tried so far: - I tried replacing wayfire with weston-terminal, to avoid the nested session. But sommelier segfaults when I do that. - I tried adding `--shared-dir /tmp/ff:ff:type=9p` to share a host directory. Then `mount -t 9p -o trans=virtio,version=9p2000.L ff /tmp` in the VM seemed to work, but `ls /tmp` crashed the VM. - I tried using `-d /dev/mapper/disk` to share an LVM partition, but `mount -t ext4 /dev/vdb /tmp` refused to mount it. - I tried enabling networking with `--host_ip 10.0.0.1`, etc, but it said it couldn't create a tap device. I guess it needs more privileges. Ideally, I'd like to run a VM with each of my old Qubes filesystems, to get back to where I was with my Qubes setup, before investigating new spectrum stuff (e.g. one app per VM). Do you have any advice on this? I see these lists are a bit quiet - I hope someone is still working on this because it sounds great :-) Thanks! -- talex5 (GitHub/Twitter) http://roscidus.com/blog/ GPG: 5DD5 8D70 899C 454A 966D 6A51 7513 3C8F 94F6 E0CC

4 months, 3 weeks

4
18
0 0

2021

2020

2019

Spectrum Discuss