Some deliberately brief updates so I can actually get all this written
down finally:
First, December was my last month working at Unikie. Some annoying
German bureaucratic quirks unfortunately got in the way and made things
difficult for me. My understanding is that they plan to continue
working with and contributing to Spectrum. Former Unikie colleagues,
I'm looking forward to continuing to work with you. :)
Relatedly, I'd like to organise another community call soon, as the last
one was really good. I'd have liked to do one in January too, but too
many non-work things getting in the way meant I wasn't able to make that
happen. But let's keep them going with a call in February. More on
this soon.
This also means that, for the time being, I'm once again funding my work
on Spectrum through community donations. (But as previously promised,
all the money I raised while I was working for Unikie will remain set
aside.) Please consider supporting my work on Spectrum through
https://github.com/sponsors/alyssais or https://liberapay.com/qyliss.
Hopefully I'll have more to share soon on other sources of funding.
What I've been doing recently: I spent December getting the
foundations of virtio-gpu support into upstream rust-vmm[1], and January
doing work on Nixpkgs. It would be a big win for development experience
if I could get rid of all of Spectrum's modifications in upstream
Nixpkgs, so I've been working towards that. I'm taking a bit of a
roundabout route because I need to be able to demonstrate why the
changes are useful in ways that aren't specific to Spectrum — as a
result of this, I've actually been working a bit on improving Nixpkgs'
FreeBSD support, since it shares some characteristics with Spectrum
that most Linux systems wouldn't. (Non-systemd udev implementation, for
example.)
[1]: https://github.com/rust-vmm/vm-virtio/commits/c527b45dada0a81d343aca7f06759…
Upcoming challenges I'm thinking about:
- Way too much of my time at the moment is spent doing QA — making
sure new Nixpkgs updates or kernels aren't going to introduce
regressions in Spectrum. We do some unusual things, so can't rely
on other people to catch problems before they affect us. I want to
get some automated testing against upstreams sorted out, so I can
free up more of my time for working on documentation and features,
which is what I really want to be doing but am fighting for time for
at the moment. Getting to 0 Nixpkgs patches is part of this.
- I want to improve the experience for other contributors — I know it's
lacking in various ways. I think the quickest win here will be to
figure out a way to let people join the Spectrum chat through Matrix
without being kicked after 30 days of inactivity (which is what the
Libera bridge does). I've been told there are various alternative
ways we could have this work. Having reliable real-time chat is
pretty critical for collaboration, and it's become especially clear,
especially after winter holidays, that we don't quite have that at
the moment.
Also, I'll be at FOSDEM this weekend. Get in touch on IRC (qyliss on
libera) or Matrix (@qyliss:fairydust.space) if you want to say hello.
Tomorrow (Wednesday) at 12:00 UTC I will be holding an experimental
Spectrum community call. The idea here is that it gives all
participants in the Spectrum community (interested users, developers,
etc.) the chance to meet and talk about upcoming development work.
So if you would like to hear or ask questions about where Spectrum is
going, or if you're working on Spectrum (or would like to be!) and want
to share what you're doing, please join us tomorrow at the following URL:
https://meet.jit.si/moderated/f20761163cb82478c3cc28dbffc1edcd788d8766232fa…
Since somebody has asked already: I don't plan on recording this call.
If it goes well and we make this a regular thing, we can discuss at that
point whether calls should be recorded.
Hello.
In the spectrum OS, as far as I know, all appvms will connect to outside through netvm.
And each appvm has different subnet.
However, sometimes, a app should be able to access the host network by bridging.
For example, an P2P app needs to send and receive multicast or broadcast to find other peers.
I wonder if it(bridging to host network) is possible in spectrum OS model, and if possible,
I want to know how to do it.
And if there is no such feature, I want to know the plan or opinion to support such app in spectrum OS.
Thanks.
Hi,
We've built the wayland demo branch on aarch64 port and added some other
apps on it to showcase embedded virtualization with Spectrum OS.
The build configuration for the reference device is documented [1] with
the out-of-tree build configuration as agreed in our earlier
discussion[2]. There's also some additional accompanied documentation on
binary cache we have used with aarch64 and could be of general use. I'd
like to see that documentation upstreamed as much as possible.
Would it make sense to link this work from Spectrum documentation? To
indicate work in progress aarch64 support with known issues on a ref.
device and avoid forking this documentation?
The practical benefit is that anyone interested in building Spectrum OS
for an aarch64 device using vendor BSP would have some reference linked
from Spectrum OS documentation - e.g. an initial porting guide as even
the x86_64 does not yet use the build configuration. Additionally, I
don't want to fragment Spectrum OS ports and their documentation further
than necessary.
Best,
-Ville
[1] https://github.com/tiiuae/spectrum-config-imx8/blob/main/README.md
[2]
https://spectrum-os.org/lists/archives/spectrum-discuss/CAP-nJwHTmROzMbyYNt…
Hi,
I've been running Qubes for a few years now and I'd like to give
Spectrum a try, as I've been having some hardware and performance
problems with Qubes. Is there some up-to-date guide I can follow? I
found https://alyssa.is/using-virtio-wl/#demo and was able to see the
weston terminal. I also tried updating to the latest commit and was
able to get a nested wayfire window with:
nix-build . -A spectrumPackages && ./result-3/bin/spectrum-vm
(I'm fairly new to Nix, so not sure if this is the right way to do things)
I managed to change the keyboard layout, mount a tmpfs for home, and
increase the memory enough to start firefox, but I haven't managed to
get much further. Things I tried so far:
- I tried replacing wayfire with weston-terminal, to avoid the nested
session. But sommelier segfaults when I do that.
- I tried adding `--shared-dir /tmp/ff:ff:type=9p` to share a host
directory. Then `mount -t 9p -o trans=virtio,version=9p2000.L ff /tmp`
in the VM seemed to work, but `ls /tmp` crashed the VM.
- I tried using `-d /dev/mapper/disk` to share an LVM partition, but
`mount -t ext4 /dev/vdb /tmp` refused to mount it.
- I tried enabling networking with `--host_ip 10.0.0.1`, etc, but it
said it couldn't create a tap device. I guess it needs more
privileges.
Ideally, I'd like to run a VM with each of my old Qubes filesystems,
to get back to where I was with my Qubes setup, before investigating
new spectrum stuff (e.g. one app per VM). Do you have any advice on
this? I see these lists are a bit quiet - I hope someone is still
working on this because it sounds great :-)
Thanks!
--
talex5 (GitHub/Twitter) http://roscidus.com/blog/
GPG: 5DD5 8D70 899C 454A 966D 6A51 7513 3C8F 94F6 E0CC
Puck has created a video demonstrating the work she's been doing with
the in-development Wayland security-context protocol [1], which allows a
Wayland compositor to distinguish between applications running in
different sandboxes (e.g. in different VMs).
The video is available at
https://diode.zone/w/2n3kKNNjXFkSWUwyjT3hgt
Or alternatively,
magnet:?xt=urn:btih:f340dfd391be0cabbb0638eb8af6659214c5d821&dn=puck%27s%20video%20720p.mp4&tr=https%3A%2F%2Fdiode.zone%2Ftracker%2Fannounce&ws=https%3A%2F%2Fdiode.zone%2Fstatic%2Fstreaming-playlists%2Fhls%2F0b093345-a100-4051-b4c3-37292af48c81%2F176adb94-167a-4cb7-b954-a09b301c4d80-720-fragmented.mp4
As part of this work, she updated the draft wlroots and Sway
implementations to support the latest proposed version of the protocol,
exposed the security context information to Sway configuration hooks,
and created a draft crosvm implementation of exposing security context
information to the compositor.
There's some more information in Puck's post to the Spectrum development
mailing list. [2]
Thanks to NLnet and NGI Zero for funding this project.
[1]: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/68
[2]: https://spectrum-os.org/lists/archives/spectrum-devel/5cf20f6f-9d89-4cf9-91…
The Qubes OS summit will be held in Berlin from tomorrow (Friday) until
Sunday. I'll be attending, and Puck will be giving a talk at 15:20
tomorrow.
Puck's talk is called "Isolating GUIs with the power of Wayland", and
the abstract is:
Could Qubes OS replace its custom GUI isolation protocol with
Wayland while staying as performant and secure? With the advent
of Wayland, many strides have been made in the desktop Linux
space, limiting the effects a malicious application can
have. Gone are the days of every application being able to snoop
every keypress! This presentation will dive into the differences
between X and Wayland, and why it makes for a great fit in
isolating operating systems like Qubes OS and Spectrum.
The talk will be live streamed at:
https://www.youtube.com/channel/UC_djHbyjuJvhVjfT18nyqmQ
iCalendar file for the talk:
https://cfp.3mdeb.com/qubes-os-summit-2022/talk/ZY8KHW.ics
Hi,
Now that we've been developing Spectrum ARM (aarch64) support
with iMX8 boards, I'd like to get back to Spectrum HW configuration design.
On x86 the generic image with kernel supporting most devices as modules can
make sense. On ARM, the vendor specific BSP HW quirks are more common.
As of now, the spectrum fork for aarch64 just adds another config
after rpi configs
and replaces the default config to use that to build. With small
changes this could
be handled like rpi configs. In addition, cloud-hypervisor accepts
kernel only in
EFI format for aarch64[1]. Anyway, this would allow us to build an
aarch64 Spectrum installer
- even make it with a more generic kernel. That takes us to ARM
vendor/device specific HW
quirks which would need to be handled anyway. I'll intentionally leave
device specific
kernel hardening and disabling kernel module loading for security
reasons for now.
As of now the vendor/device specifics are not supported unless one builds device
specific Spectrum image with all configs build-time and skips
installer altogether.
The other option that I see. We discussed earlier nix-hardware and
device specific modules.
That would bring nixos configuration.nix and installation supporting
scripts to Spectrum,
though. Those could be called from the Spectrum installer but it would
change the installer
logic from writing an image to dynamically configuring the device
during install based on user
selections.
Any thoughts which would be the preferred way? Maybe some other way?
In the end, HW specifics are needed also on x86 as we saw with NUCs
and different
Lenovo laptops in the spring. I'm not convinced one image to rule them
all is realistic or secure.
Finally, this is by no means blocking the hardened iMX8 based Spectrum
development
but will keep that work in Spectrum fork until there's an agreed path
to implement this.
Integrating this sooner and making it more generic would make Spectrum
more useful
for a wider audience.
Best regards,
-Ville
[1] https://github.com/tiiuae/spectrum/pull/3#issuecomment-1211834302
Recently I've been working on making it possible for us to use crosvm's
implementation of virtio-gpu (which is necessary for multi-VM Wayland).
The approach I was originally planning on was porting crosvm's
vhost-user-gpu frontend to cloud-hypervisor. That would allow us to run
the crosvm implementation of the device unmodified, with just a small
amount of glue code in cloud-hypervisor.
But then I discovered some things that made me decide to investigate
other approaches:
- crosvm does not implement the standard vhost-user-gpu protocol.
It implements it its own special way. Perhaps ironically, the
crosvm-specific way seems to be closer to how vhost-user works for
other devices (like network and block), which should actually make
it easier to port the frontend to cloud-hypervisor. But it also
changes the potential to upstream that port to cloud-hypervisor
from "a hard sell" to "not going to happen". So if I did that,
it would commit us to carrying a cloud-hypervisor patch indefinitely.
- There's an interesting new protocol called vfio-user, that would be
really helpful to us in this situation. Whereas vhost-user requires
the VMM to still have some basic per-device knowledge (the glue code
I was planning to port), vfio-user operates at the PCI level, so the
VMM only needs to know that the device is PCI. So if we could
somehow provide a virtio-gpu device to cloud-hypervisor over
vfio-user, cloud-hypervisor wouldn't need any GPU-specific code at
all. Everything should just work without any changes to
cloud-hypervisor, as it already implements a vfio-user client.
- The next release of QEMU, 7.1.0, will include support for exporting
any virtual device QEMU can provide over vfio-user.
So with all this in mind, there are three ways we could try to proceed:
1. Port the crosvm-specific vhost-user-gpu frontend to cloud-hypervisor.
2. Make crosvm speak the standard version of vhost-user-gpu, then use
QEMU to act as a bridge between the crosvm GPU device, and
cloud-hypervisor, translating vhost-user to vfio-user, but not doing
anything else. (So we're not using QEMU to run a VM, just to
translate between these two protocols and handle the PCI stuff.)
3. Implement a vfio-user server in crosvm, so crosvm device backends
can be used directly with cloud-hypervisor.
3 is the clear best option, because it doesn't require adding QEMU into
the system, and it would be entirely upstreamable — I spoke to a crosvm
developer about it in their Matrix channel and they said they'd be
interested in patches for it. But it's also quite complicated to
implement, and beyond my ability, at least if I want results any time
soon. 2 would probably be even more complicated as it would require
coordinating between crosvm and QEMU to get them to agree on how the
protocol should work.
So if we want this working soon, 1 is the only feasible option, at least
if it's me doing the work. But it means committing to carrying a
cloud-hypervisor patch until somebody comes along to implement 3. It
gives me bad vibes because it goes against the upstream-first approach I
try to take with Spectrum development, and once timely package updates
are something we have to take more seriously, the patch no longer
applying would block any sort of automatic updates.
So I'm posting this to solicit thoughts on what to do here. The ideal
scenario is that we are able to find somebody else (with more VMM
implementation experience than me) who is able to do 3, and then I would
be totally comfortable with doing 1 as a stopgap until that can happen.
Otherwise, I can either keep trying to chip away at doing it myself,
however long that takes, or we'd have to just accept the consequences of
having the patch indefinitely, and hope that Google or somebody else
also finds themself wishing crosvm had a vfio-user server and implements
it themself.
Thoughts welcome. :)
If you've been paying close attention recently, you'll have seen
patches coming from a few different Unikie[1] email accounts. In
addition to contributing to Spectrum, Unikie has hired me to work on it.
Unikie is interested in Spectrum for both desktop and embedded use.
They're contracting for the TII Secure Systems Research Center[2],
working on developing a Spectrum-based reference system.
One thing they're currently working on is being able to run Spectrum on
the i.MX8 development board, which means we'll hopefully see patches
adding ARM and cross-compilation support to Spectrum in the near future.
The first big thing I'll be working on for Unikie is finally integrating
crosvm's support for graphical application VMs into Spectrum.
While I'm working for Unikie, I plan to set aside any donations I
receive for my Spectrum work, to be used for project expenses and
funding Spectrum work from other people, rather than being used to pay
for my general living expenses as has been the case up to now.
And just to clarify: Spectrum has not been acquired or anything — I'm
still leading the project, and Unikie has the same rights as any other
contributor (copyright ownership of contributions, mostly).
[1]: https://www.unikie.com/en/
[2]: https://www.tii.ae/secure-systems
