* Firefox appVM patches and appVM refactoring @ 2022-12-04 22:42 Vadim Likholetov 2022-12-06 11:42 ` Alyssa Ross 2022-12-06 15:20 ` Alyssa Ross 0 siblings, 2 replies; 18+ messages in thread From: Vadim Likholetov @ 2022-12-04 22:42 UTC (permalink / raw) To: devel [-- Attachment #1: Type: text/plain, Size: 1775 bytes --] I've made an Firefox appVM for wayland using my waypipe patches. To make this appVM I have had to refactor Spectrum OS appvm infrastructure. The main idea of refactoring is enabling appvm to have a user with normal priviledges, not superuser. Running everything from root is not the best idea for secure OS :-) So now the .nix file for appvm has two sections, one that is executed as root and one as user. Here is the sample of this definitions: { config ? import ../../../nix/eval-config.nix {} }: import ../make-vm.nix { inherit config; } { providers.net = [ "netvm" ]; run = config.pkgs.pkgsStatic.callPackage ( { writeScript }: writeScript "run-root-shell" '' #!/bin/execlineb -P /bin/sh '' ) { }; run-as-user = config.pkgs.pkgsStatic.callPackage ( { writeScript, lynx }: writeScript "run-lynx" '' #!/bin/execlineb -P ${lynx}/bin/lynx https://spectrum-os.org '' ) { }; } Cloud-hypervisor has virtual hardware limitations -- it supports only one console device and only one serial device. SpectrumOS is using serial device for kernel logs of appVM and console device as a console. To have access both to root-executed part and to user-executed part of the VM payload, I installed a tmux on console. Now, when you're running vm-console command you get access to the tmux and have the ability to switch between root and user consoles, that can be useful during debugging VM payload. To run Firefox appVM use vm-start-way command: vm-start-way appvm-firefox :) I beleive that as soon as SpectrumOS features will cover basic user needs it's popularity and community will grow and this will make positive impact on SpectrumOS itself. Using appvm-firefox prototype you may build another wayland-enabled appVMs. [-- Attachment #2: Type: text/html, Size: 2098 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Firefox appVM patches and appVM refactoring 2022-12-04 22:42 Firefox appVM patches and appVM refactoring Vadim Likholetov @ 2022-12-06 11:42 ` Alyssa Ross 2022-12-06 13:56 ` Vadim Likholetov 2022-12-06 15:20 ` Alyssa Ross 1 sibling, 1 reply; 18+ messages in thread From: Alyssa Ross @ 2022-12-06 11:42 UTC (permalink / raw) To: Vadim Likholetov; +Cc: devel [-- Attachment #1: Type: text/plain, Size: 3889 bytes --] On Mon, Dec 05, 2022 at 12:42:35AM +0200, Vadim Likholetov wrote: > I've made an Firefox appVM for wayland using my waypipe patches. > To make this appVM I have had to refactor Spectrum OS appvm infrastructure. > The main idea of refactoring is enabling appvm to have a user with normal > priviledges, not superuser. > Running everything from root is not the best idea for secure OS :-) In this case, the application running as root is the only thing running in the VM, so it doesn't _really_ matter, but it is still a good idea to fix, as some applications will refuse to run as root. (cloud-hypervisor also runs on the /host/ as root, and that's something that we'll definitely want to fix, but that's more complicated as just statically assigning c-h processes to uids won't work when we want to dynamically create and destroy VMs. We'll likely want something akin to systemd's DynamicUser=.) > So now the .nix file for appvm has two sections, one that is executed as > root and one as user. > Here is the sample of this definitions: > > { config ? import ../../../nix/eval-config.nix {} }: > > import ../make-vm.nix { inherit config; } { > providers.net = [ "netvm" ]; > > run = config.pkgs.pkgsStatic.callPackage ( > { writeScript }: > writeScript "run-root-shell" '' > #!/bin/execlineb -P > /bin/sh > '' > ) { }; > > run-as-user = config.pkgs.pkgsStatic.callPackage ( > { writeScript, lynx }: > writeScript "run-lynx" '' > #!/bin/execlineb -P > ${lynx}/bin/lynx https://spectrum-os.org > '' > ) { }; > > } I'm not too sure about this part — it seems like quite a lot of complexity in the app VM implementation, when dropping privileges (unless there's something I haven't considered?) should be as simple as putting "s6-applyuidgid -u 1000 -g 1000" in the VM run script at the point where privileges can be dropped. > Cloud-hypervisor has virtual hardware limitations -- it supports only one > console device and only one serial device. > SpectrumOS is using serial device for kernel logs of appVM and console > device as a console. > To have access both to root-executed part and to user-executed part of the > VM payload, I installed a tmux on console. > Now, when you're running vm-console command you get access to the tmux > and have the ability to switch between root and user consoles, > that can be useful during debugging VM payload. > > To run Firefox appVM use vm-start-way command: vm-start-way appvm-firefox :) Would it work with virtio-gpu? I'm still not convinced on Waypipe — where the previous discussion left off, we were talking about VMs over the network. That would be an interesting thing to look at (and it would be really cool if we could make it work!), but doing it would take a lot more than just network-transparent Wayland proxying, so if that's the main thing we'd get out of Waypipe, I think it would only make sense to add Waypipe support as part of that bigger work. (And this point in time, when how VMs work at all in Spectrum is a bit in flux, is probably not the best time to start trying to massively expand their scope!) > i beleive that as soon as spectrumOS features will cover basic user needs > it's popularity and community will grow and this will make positive impact > on SpectrumOS itself. > Using appvm-firefox prototype you may build another wayland-enabled appVMs. Yeah, having a Firefox example VM would be really great for demonstrating how Spectrum works and what it can do, and I'm pleased that we're even getting to a point where it would work! Firefox is a big application that does a lot of stuff, so it would also be useful for testing all sorts of other features, like audio or XDG desktop portals. I'd be very happy to accept a Firefox VM that used virtio-gpu. :) [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Firefox appVM patches and appVM refactoring 2022-12-06 11:42 ` Alyssa Ross @ 2022-12-06 13:56 ` Vadim Likholetov 2022-12-06 15:18 ` Alyssa Ross 0 siblings, 1 reply; 18+ messages in thread From: Vadim Likholetov @ 2022-12-06 13:56 UTC (permalink / raw) To: Alyssa Ross; +Cc: devel > On 6 Dec 2022, at 13:42, Alyssa Ross <alyssa.ross@unikie.com> wrote: > > On Mon, Dec 05, 2022 at 12:42:35AM +0200, Vadim Likholetov wrote: >> I've made an Firefox appVM for wayland using my waypipe patches. >> To make this appVM I have had to refactor Spectrum OS appvm infrastructure. >> The main idea of refactoring is enabling appvm to have a user with normal >> priviledges, not superuser. >> Running everything from root is not the best idea for secure OS :-) > > In this case, the application running as root is the only thing running > in the VM, so it doesn't _really_ matter, but it is still a good idea to > fix, as some applications will refuse to run as root. I don’t agree there — there are many reasons application should not have unnecessary privileges, even assuming the case where someone attacks the application using kernel/processor vulnerabilities to get out of the virtualisation fencing is more fantastic than real :) Also in it is never “the only" application in the VM — we always have those binutils, init stuff and other supplemental software — except the situation we’re running something like bare kernel. So if we have really talking about *secure* OS — drop the <s>gun</s> priviledges ! :-) > (cloud-hypervisor also runs on the /host/ as root, and that's something > that we'll definitely want to fix, but that's more complicated as just > statically assigning c-h processes to uids won't work when we want to > dynamically create and destroy VMs. We'll likely want something akin to > systemd's DynamicUser=.) We can ran C-H as non-root on the host — I don’t see any huge problems there, it’s designed not to be run as root. But the idea to assign an unique UID to each VM is secure enough :-) > >> So now the .nix file for appvm has two sections, one that is executed as >> root and one as user. >> Here is the sample of this definitions: >> >> { config ? import ../../../nix/eval-config.nix {} }: >> >> import ../make-vm.nix { inherit config; } { >> providers.net = [ "netvm" ]; >> >> run = config.pkgs.pkgsStatic.callPackage ( >> { writeScript }: >> writeScript "run-root-shell" '' >> #!/bin/execlineb -P >> /bin/sh >> '' >> ) { }; >> >> run-as-user = config.pkgs.pkgsStatic.callPackage ( >> { writeScript, lynx }: >> writeScript "run-lynx" '' >> #!/bin/execlineb -P >> ${lynx}/bin/lynx https://www.google.com/url?q=https://spectrum-os.org&source=gmail-imap&ust=1670931764000000&usg=AOvVaw3fPCahT573bNMQKMWAGTEW >> '' >> ) { }; >> >> } > > I'm not too sure about this part — it seems like quite a lot of > complexity in the app VM implementation, when dropping privileges > (unless there's something I haven't considered?) should be as simple as > putting "s6-applyuidgid -u 1000 -g 1000" in the VM run script at the > point where privileges can be dropped. I think it’s more flexibility rather than complexity. Also because of immature nature of the Spectrum OS it is good to have some extra tools for developers to debug the stuff and to have extra controls. I can think of refactoring this on the /img/vm level — I’m using su and not s6-tools, if you’re interested in this patchset in general. > >> Cloud-hypervisor has virtual hardware limitations -- it supports only one >> console device and only one serial device. >> SpectrumOS is using serial device for kernel logs of appVM and console >> device as a console. >> To have access both to root-executed part and to user-executed part of the >> VM payload, I installed a tmux on console. >> Now, when you're running vm-console command you get access to the tmux >> and have the ability to switch between root and user consoles, >> that can be useful during debugging VM payload. >> >> To run Firefox appVM use vm-start-way command: vm-start-way appvm-firefox :) > > Would it work with virtio-gpu? I'm still not convinced on Waypipe — > where the previous discussion left off, we were talking about VMs over > the network. That would be an interesting thing to look at (and it > would be really cool if we could make it work!), but doing it would take > a lot more than just network-transparent Wayland proxying, so if that's > the main thing we'd get out of Waypipe, I think it would only make sense > to add Waypipe support as part of that bigger work. (And this point in > time, when how VMs work at all in Spectrum is a bit in flux, is probably > not the best time to start trying to massively expand their scope!) I think it’s the wrong question :-) Both implementations of wayland passthru ( virtio-gpu and waypipe) are ugly right now but they both allow to move the development forward. But someday I believe they’ll become nice. I really believe! :-) The thing is definitely needed in the mainline is abstraction layer for wayland apps — to move all proxying and other system or hardware dependant details out of VM’s descriptions — we define that is an wayland appVM ( like it’s defined it’s networking appVM) and the mechanisms of how the WAYLAND_DISPLAY gets to the is covered by this abstraction layer. >> i beleive that as soon as spectrumOS features will cover basic user needs >> it's popularity and community will grow and this will make positive impact >> on SpectrumOS itself. >> Using appvm-firefox prototype you may build another wayland-enabled appVMs. > > Yeah, having a Firefox example VM would be really great for > demonstrating how Spectrum works and what it can do, and I'm pleased > that we're even getting to a point where it would work! Firefox is a > big application that does a lot of stuff, so it would also be useful for > testing all sorts of other features, like audio or XDG desktop portals. > I'd be very happy to accept a Firefox VM that used virtio-gpu. :) ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Firefox appVM patches and appVM refactoring 2022-12-06 13:56 ` Vadim Likholetov @ 2022-12-06 15:18 ` Alyssa Ross 0 siblings, 0 replies; 18+ messages in thread From: Alyssa Ross @ 2022-12-06 15:18 UTC (permalink / raw) To: Vadim Likholetov; +Cc: devel [-- Attachment #1: Type: text/plain, Size: 4166 bytes --] > >> So now the .nix file for appvm has two sections, one that is executed as > >> root and one as user. > >> Here is the sample of this definitions: > >> > >> { config ? import ../../../nix/eval-config.nix {} }: > >> > >> import ../make-vm.nix { inherit config; } { > >> providers.net = [ "netvm" ]; > >> > >> run = config.pkgs.pkgsStatic.callPackage ( > >> { writeScript }: > >> writeScript "run-root-shell" '' > >> #!/bin/execlineb -P > >> /bin/sh > >> '' > >> ) { }; > >> > >> run-as-user = config.pkgs.pkgsStatic.callPackage ( > >> { writeScript, lynx }: > >> writeScript "run-lynx" '' > >> #!/bin/execlineb -P > >> ${lynx}/bin/lynx https://www.google.com/url?q=https://spectrum-os.org&source=gmail-imap&ust=1670931764000000&usg=AOvVaw3fPCahT573bNMQKMWAGTEW > >> '' > >> ) { }; > >> > >> } > > > > I'm not too sure about this part — it seems like quite a lot of > > complexity in the app VM implementation, when dropping privileges > > (unless there's something I haven't considered?) should be as simple as > > putting "s6-applyuidgid -u 1000 -g 1000" in the VM run script at the > > point where privileges can be dropped. > > I think it’s more flexibility rather than complexity. > Also because of immature nature of the Spectrum OS it is good to have some extra tools for developers to debug the stuff and to have extra controls. > I can think of refactoring this on the /img/vm level — I’m using su and not s6-tools, if you’re interested in this patchset in general. Would you mind explaining what makes run-as-user more flexible? As far as I can tell, run-as-user doesn't make it possible to do anything that wasn't possible before. > >> Cloud-hypervisor has virtual hardware limitations -- it supports only one > >> console device and only one serial device. > >> SpectrumOS is using serial device for kernel logs of appVM and console > >> device as a console. > >> To have access both to root-executed part and to user-executed part of the > >> VM payload, I installed a tmux on console. > >> Now, when you're running vm-console command you get access to the tmux > >> and have the ability to switch between root and user consoles, > >> that can be useful during debugging VM payload. > >> > >> To run Firefox appVM use vm-start-way command: vm-start-way appvm-firefox :) > > > > Would it work with virtio-gpu? I'm still not convinced on Waypipe — > > where the previous discussion left off, we were talking about VMs over > > the network. That would be an interesting thing to look at (and it > > would be really cool if we could make it work!), but doing it would take > > a lot more than just network-transparent Wayland proxying, so if that's > > the main thing we'd get out of Waypipe, I think it would only make sense > > to add Waypipe support as part of that bigger work. (And this point in > > time, when how VMs work at all in Spectrum is a bit in flux, is probably > > not the best time to start trying to massively expand their scope!) > > I think it’s the wrong question :-) > Both implementations of wayland passthru ( virtio-gpu and waypipe) are ugly right now but they both allow to move the development forward. But someday I believe they’ll become nice. I really believe! :-) > > The thing is definitely needed in the mainline is abstraction layer for wayland apps — to move all proxying and other system or hardware dependant details out of VM’s descriptions — we define that is an wayland appVM ( like it’s defined it’s networking appVM) and the mechanisms of how the WAYLAND_DISPLAY gets to the is covered by this abstraction layer. Agreed on that. I want to get to the place soon where we can run arbitrary applications on Spectrum, without having to create custom VM definitions for each one. To be able to do that, we'll need the environment an application it runs in to be set up as common applications expect. So WAYLAND_DISPLAY, etc. will need to be set up. And then if you want to use Waypipe instead of virtio-gpu, you just need to make a custom template that does that. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Firefox appVM patches and appVM refactoring 2022-12-04 22:42 Firefox appVM patches and appVM refactoring Vadim Likholetov 2022-12-06 11:42 ` Alyssa Ross @ 2022-12-06 15:20 ` Alyssa Ross 2022-12-06 15:57 ` Vadim Likholetov 1 sibling, 1 reply; 18+ messages in thread From: Alyssa Ross @ 2022-12-06 15:20 UTC (permalink / raw) To: Vadim Likholetov; +Cc: devel [-- Attachment #1: Type: text/plain, Size: 819 bytes --] On Mon, Dec 05, 2022 at 12:42:35AM +0200, Vadim Likholetov wrote: > Cloud-hypervisor has virtual hardware limitations -- it supports only one > console device and only one serial device. > SpectrumOS is using serial device for kernel logs of appVM and console > device as a console. > To have access both to root-executed part and to user-executed part of the > VM payload, I installed a tmux on console. > Now, when you're running vm-console command you get access to the tmux > and have the ability to switch between root and user consoles, > that can be useful during debugging VM payload. I wonder what it would take to make cloud-hypervisor support multiple virtio-console devices… I suspect it wouldn't be too hard, since it already supports multiples of every other virtio device just fine… [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Firefox appVM patches and appVM refactoring 2022-12-06 15:20 ` Alyssa Ross @ 2022-12-06 15:57 ` Vadim Likholetov 2022-12-06 17:10 ` Alyssa Ross 0 siblings, 1 reply; 18+ messages in thread From: Vadim Likholetov @ 2022-12-06 15:57 UTC (permalink / raw) To: Alyssa Ross; +Cc: devel I’ve done this — I have patched cloud-hypervisor to have three console devices — serial, console and user-console, but than decided that this patches will never go to C-H mainline so got back with more traditional approach with tmux :) > On 6 Dec 2022, at 17:20, Alyssa Ross <alyssa.ross@unikie.com> wrote: > > On Mon, Dec 05, 2022 at 12:42:35AM +0200, Vadim Likholetov wrote: >> Cloud-hypervisor has virtual hardware limitations -- it supports only one >> console device and only one serial device. >> SpectrumOS is using serial device for kernel logs of appVM and console >> device as a console. >> To have access both to root-executed part and to user-executed part of the >> VM payload, I installed a tmux on console. >> Now, when you're running vm-console command you get access to the tmux >> and have the ability to switch between root and user consoles, >> that can be useful during debugging VM payload. > > I wonder what it would take to make cloud-hypervisor support multiple > virtio-console devices… I suspect it wouldn't be too hard, since it > already supports multiples of every other virtio device just fine… ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Firefox appVM patches and appVM refactoring 2022-12-06 15:57 ` Vadim Likholetov @ 2022-12-06 17:10 ` Alyssa Ross 2022-12-06 20:12 ` Vadim Likholetov ` (2 more replies) 0 siblings, 3 replies; 18+ messages in thread From: Alyssa Ross @ 2022-12-06 17:10 UTC (permalink / raw) To: Vadim Likholetov; +Cc: devel [-- Attachment #1: Type: text/plain, Size: 2327 bytes --] On Tue, Dec 06, 2022 at 05:57:19PM +0200, Vadim Likholetov wrote: > I’ve done this — I have patched cloud-hypervisor to have > three console devices — serial, console and user-console, but than > decided that this patches will never go to C-H mainline so got back > with more traditional approach with tmux :) I'd be pretty optimistic about the chances of a patch that just made it so you could provide multiple consoles the same way you can provide multiple block devices. Like "--console pty file=/path/to/console.out" on the command like to make two consoles, one going to a pty and the other to a file. In Spectrum, the way I'd see such an approach working is that by default there'd be a single console that gave you a shell inside the VM, and if you needed other consoles for other reasons (e.g. if you had an application running on the console, like Lynx) you'd configure another console in the VM configuration. But as we move more towards graphical applications, it will probably become rare to need a secondary console for application interaction like that. To get a non-root shell from the root console though, I think it should be enough to just: # s6-applyuidgid -u 1000 -g 1000 sh (Adding tmux to the VM in development to be able to hop between multiple shell sessions would be a reasonable thing to do, of course.) > > On 6 Dec 2022, at 17:20, Alyssa Ross <alyssa.ross@unikie.com> wrote: > > > > On Mon, Dec 05, 2022 at 12:42:35AM +0200, Vadim Likholetov wrote: > >> Cloud-hypervisor has virtual hardware limitations -- it supports only one > >> console device and only one serial device. > >> SpectrumOS is using serial device for kernel logs of appVM and console > >> device as a console. > >> To have access both to root-executed part and to user-executed part of the > >> VM payload, I installed a tmux on console. > >> Now, when you're running vm-console command you get access to the tmux > >> and have the ability to switch between root and user consoles, > >> that can be useful during debugging VM payload. > > > > I wonder what it would take to make cloud-hypervisor support multiple > > virtio-console devices… I suspect it wouldn't be too hard, since it > > already supports multiples of every other virtio device just fine… > > [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Firefox appVM patches and appVM refactoring 2022-12-06 17:10 ` Alyssa Ross @ 2022-12-06 20:12 ` Vadim Likholetov 2022-12-06 20:25 ` Alyssa Ross 2022-12-06 20:19 ` Vadim Likholetov 2022-12-07 8:04 ` Firefox appVM patches and appVM refactoring Ville Ilvonen 2 siblings, 1 reply; 18+ messages in thread From: Vadim Likholetov @ 2022-12-06 20:12 UTC (permalink / raw) To: Alyssa Ross; +Cc: devel I’ll try to explain — running as user is not just dropping the priviledges, it is about preparing the environment - making home directory, fixing permissions, allocating pty-s in case of interactive sessions, setting environment variables. Many system daemons like dbus, pipewire , etc has their system-level and user-level parts and we should manage this separately — and it is our nearest future if we want some complex user environments to run on Spectrum, and possibly the packages for this daemons will extend this environment transparent to the user. So it’s s another form of encapsulation like we discussed about making a separate layer for wayland. I’ve published the cloud-hypervisor patches for extra user console device in my repository under -userconsole branch. I’ll test it (or anyone can) for side effects and then we can discuss if it worth to be used in Spectrum or pushed upstream of C-H. Here is the URL — https://github.com/vadika/cloud-hypervisor/tree/userconsole > On 6 Dec 2022, at 19:10, Alyssa Ross <alyssa.ross@unikie.com> wrote: > > On Tue, Dec 06, 2022 at 05:57:19PM +0200, Vadim Likholetov wrote: >> I’ve done this — I have patched cloud-hypervisor to have >> three console devices — serial, console and user-console, but than >> decided that this patches will never go to C-H mainline so got back >> with more traditional approach with tmux :) > > I'd be pretty optimistic about the chances of a patch that just made it > so you could provide multiple consoles the same way you can provide > multiple block devices. Like "--console pty file=/path/to/console.out" > on the command like to make two consoles, one going to a pty and the > other to a file. > > In Spectrum, the way I'd see such an approach working is that by default > there'd be a single console that gave you a shell inside the VM, and if > you needed other consoles for other reasons (e.g. if you had an > application running on the console, like Lynx) you'd configure another > console in the VM configuration. But as we move more towards graphical > applications, it will probably become rare to need a secondary console > for application interaction like that. > > To get a non-root shell from the root console though, I think it > should be enough to just: > > # s6-applyuidgid -u 1000 -g 1000 sh > > (Adding tmux to the VM in development to be able to hop between multiple > shell sessions would be a reasonable thing to do, of course.) > >>> On 6 Dec 2022, at 17:20, Alyssa Ross <alyssa.ross@unikie.com> wrote: >>> >>> On Mon, Dec 05, 2022 at 12:42:35AM +0200, Vadim Likholetov wrote: >>>> Cloud-hypervisor has virtual hardware limitations -- it supports only one >>>> console device and only one serial device. >>>> SpectrumOS is using serial device for kernel logs of appVM and console >>>> device as a console. >>>> To have access both to root-executed part and to user-executed part of the >>>> VM payload, I installed a tmux on console. >>>> Now, when you're running vm-console command you get access to the tmux >>>> and have the ability to switch between root and user consoles, >>>> that can be useful during debugging VM payload. >>> >>> I wonder what it would take to make cloud-hypervisor support multiple >>> virtio-console devices… I suspect it wouldn't be too hard, since it >>> already supports multiples of every other virtio device just fine… >> >> ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Firefox appVM patches and appVM refactoring 2022-12-06 20:12 ` Vadim Likholetov @ 2022-12-06 20:25 ` Alyssa Ross 2022-12-07 7:41 ` vadik likholetov 0 siblings, 1 reply; 18+ messages in thread From: Alyssa Ross @ 2022-12-06 20:25 UTC (permalink / raw) To: Vadim Likholetov; +Cc: devel [-- Attachment #1: Type: text/plain, Size: 4468 bytes --] On Tue, Dec 06, 2022 at 10:12:49PM +0200, Vadim Likholetov wrote: > I’ll try to explain — running as user is not just dropping the priviledges, it is about preparing the environment - making home directory, fixing permissions, allocating pty-s in case of interactive sessions, setting environment variables. > Many system daemons like dbus, pipewire , etc has their system-level and user-level parts and we should manage this separately — and it is our nearest future if we want some complex user environments to run on Spectrum, and possibly the packages for this daemons will extend this environment transparent to the user. > > So it’s s another form of encapsulation like we discussed about making a separate layer for wayland. Ah, thanks for the explanation. I understand now! You're quite right about running user daemons like PipeWire. I think to manage those, we'd want to run a user-level s6(-rc) instance, so that the services are supervised, and can therefore be restarted if they crash, and dependencies can be taken care of, and logs can be managed however that should work. Let's try adding PipeWire, D-Bus etc. daemons straight to the img/app image? It's intended to be the default image for running applications, so anything that an application is likely to expect to be running should be provided by that image by default. And then later on, when it comes up, we can figure out how to support adding extra services, or whether we should support removing default services from the image, etc. How does that sound? > I’ve published the cloud-hypervisor patches for extra user console device in my repository under -userconsole branch. > I’ll test it (or anyone can) for side effects and then we can discuss if it worth to be used in Spectrum or pushed upstream of C-H. > Here is the URL — https://github.com/vadika/cloud-hypervisor/tree/userconsole > > > On 6 Dec 2022, at 19:10, Alyssa Ross <alyssa.ross@unikie.com> wrote: > > > > On Tue, Dec 06, 2022 at 05:57:19PM +0200, Vadim Likholetov wrote: > >> I’ve done this — I have patched cloud-hypervisor to have > >> three console devices — serial, console and user-console, but than > >> decided that this patches will never go to C-H mainline so got back > >> with more traditional approach with tmux :) > > > > I'd be pretty optimistic about the chances of a patch that just made it > > so you could provide multiple consoles the same way you can provide > > multiple block devices. Like "--console pty file=/path/to/console.out" > > on the command like to make two consoles, one going to a pty and the > > other to a file. > > > > In Spectrum, the way I'd see such an approach working is that by default > > there'd be a single console that gave you a shell inside the VM, and if > > you needed other consoles for other reasons (e.g. if you had an > > application running on the console, like Lynx) you'd configure another > > console in the VM configuration. But as we move more towards graphical > > applications, it will probably become rare to need a secondary console > > for application interaction like that. > > > > To get a non-root shell from the root console though, I think it > > should be enough to just: > > > > # s6-applyuidgid -u 1000 -g 1000 sh > > > > (Adding tmux to the VM in development to be able to hop between multiple > > shell sessions would be a reasonable thing to do, of course.) > > > >>> On 6 Dec 2022, at 17:20, Alyssa Ross <alyssa.ross@unikie.com> wrote: > >>> > >>> On Mon, Dec 05, 2022 at 12:42:35AM +0200, Vadim Likholetov wrote: > >>>> Cloud-hypervisor has virtual hardware limitations -- it supports only one > >>>> console device and only one serial device. > >>>> SpectrumOS is using serial device for kernel logs of appVM and console > >>>> device as a console. > >>>> To have access both to root-executed part and to user-executed part of the > >>>> VM payload, I installed a tmux on console. > >>>> Now, when you're running vm-console command you get access to the tmux > >>>> and have the ability to switch between root and user consoles, > >>>> that can be useful during debugging VM payload. > >>> > >>> I wonder what it would take to make cloud-hypervisor support multiple > >>> virtio-console devices… I suspect it wouldn't be too hard, since it > >>> already supports multiples of every other virtio device just fine… > >> > >> > > [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Firefox appVM patches and appVM refactoring 2022-12-06 20:25 ` Alyssa Ross @ 2022-12-07 7:41 ` vadik likholetov 0 siblings, 0 replies; 18+ messages in thread From: vadik likholetov @ 2022-12-07 7:41 UTC (permalink / raw) To: Alyssa Ross; +Cc: devel [-- Attachment #1: Type: text/plain, Size: 2097 bytes --] On ti, joulu 6 2022 at 20.25.45 +00:00:00, Alyssa Ross <alyssa.ross@unikie.com> wrote: > On Tue, Dec 06, 2022 at 10:12:49PM +0200, Vadim Likholetov wrote: >> I’ll try to explain — running as user is not just dropping the >> priviledges, it is about preparing the environment - making home >> directory, fixing permissions, allocating pty-s in case of >> interactive sessions, setting environment variables. >> Many system daemons like dbus, pipewire , etc has their >> system-level and user-level parts and we should manage this >> separately — and it is our nearest future if we want some complex >> user environments to run on Spectrum, and possibly the packages for >> this daemons will extend this environment transparent to the user. >> >> So it’s s another form of encapsulation like we discussed about >> making a separate layer for wayland. > > Ah, thanks for the explanation. I understand now! > > You're quite right about running user daemons like PipeWire. > I think to manage those, we'd want to run a user-level s6(-rc) > instance, > so that the services are supervised, and can therefore be restarted if > they crash, and dependencies can be taken care of, and logs can be > managed however that should work. > > Let's try adding PipeWire, D-Bus etc. daemons straight to the img/app > image? It's intended to be the default image for running > applications, > so anything that an application is likely to expect to be running > should > be provided by that image by default. And then later on, when it > comes > up, we can figure out how to support adding extra services, or whether > we should support removing default services from the image, etc. How > does that sound? Sounds good if we add both user and system parts of dbus or pipewire subsystem -- the system part when the system starts and the user part when the user "logs in". Adding it only to system part doesn't make sense IMO. I can try to prototype it on the basis of s6 features if we agree about architecure :) -- vadik likholetov [-- Attachment #2: Type: text/html, Size: 2270 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Firefox appVM patches and appVM refactoring 2022-12-06 17:10 ` Alyssa Ross 2022-12-06 20:12 ` Vadim Likholetov @ 2022-12-06 20:19 ` Vadim Likholetov 2022-12-06 20:31 ` Developer-friendliness Alyssa Ross 2022-12-06 20:59 ` Developer-friendliness Michael Raskin 2022-12-07 8:04 ` Firefox appVM patches and appVM refactoring Ville Ilvonen 2 siblings, 2 replies; 18+ messages in thread From: Vadim Likholetov @ 2022-12-06 20:19 UTC (permalink / raw) To: Alyssa Ross; +Cc: devel > On 6 Dec 2022, at 19:10, Alyssa Ross <alyssa.ross@unikie.com> wrote: > > (Adding tmux to the VM in development to be able to hop between multiple > shell sessions would be a reasonable thing to do, of course.) It may sound like religious point of view, but I strongly believe that developers is the number one priority to support with Spectrum OS now — as easier and friendlier the OS will be to the developer as faster it will be delivered to the common users. It, from my point of view, includes two main aspects — ease of development giving the developer proper “bricks” (well encapsulated OS services) and examples, that coveres main use cases/use patterns of the operating system. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Developer-friendliness 2022-12-06 20:19 ` Vadim Likholetov @ 2022-12-06 20:31 ` Alyssa Ross 2022-12-07 7:46 ` Developer-friendliness vadik likholetov 2022-12-06 20:59 ` Developer-friendliness Michael Raskin 1 sibling, 1 reply; 18+ messages in thread From: Alyssa Ross @ 2022-12-06 20:31 UTC (permalink / raw) To: Vadim Likholetov; +Cc: devel [-- Attachment #1: Type: text/plain, Size: 1559 bytes --] On Tue, Dec 06, 2022 at 10:19:58PM +0200, Vadim Likholetov wrote: > > > > On 6 Dec 2022, at 19:10, Alyssa Ross <alyssa.ross@unikie.com> wrote: > > > > (Adding tmux to the VM in development to be able to hop between multiple > > shell sessions would be a reasonable thing to do, of course.) > > It may sound like religious point of view, but I strongly believe > that developers is the number one priority to support with Spectrum OS > now — as easier and friendlier the OS will be to the developer as > faster it will be delivered to the common users. It, from my point of > view, includes two main aspects — ease of development giving the > developer proper “bricks” (well encapsulated OS services) and > examples, that coveres main use cases/use patterns of the operating > system. Yes, that's a compelling argument. I wonder if it would make sense to add some special configuration option for development, that would enable extra development tools and other goodies? (I often find myself adding strace…) (I think they probably shouldn't be present by default, because then it could be difficult to be sure nothing was depending on them in future when it came time to remove them. But something that any given developer could set once and then always have developer stuff available, and toggle off when they wanted to test a production build, could make things a lot nicer.) And BTW, thanks so much for spending all this time engaging with this stuff! The project will be much better off for your efforts. :) [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Developer-friendliness 2022-12-06 20:31 ` Developer-friendliness Alyssa Ross @ 2022-12-07 7:46 ` vadik likholetov 0 siblings, 0 replies; 18+ messages in thread From: vadik likholetov @ 2022-12-07 7:46 UTC (permalink / raw) To: Alyssa Ross; +Cc: devel [-- Attachment #1: Type: text/plain, Size: 1053 bytes --] On ti, joulu 6 2022 at 20.31.48 +00:00:00, Alyssa Ross <alyssa.ross@unikie.com> wrote: > Yes, that's a compelling argument. I wonder if it would make sense to > add some special configuration option for development, that would > enable > extra development tools and other goodies? (I often find myself > adding > strace…) > > (I think they probably shouldn't be present by default, because then > it > could be difficult to be sure nothing was depending on them in future > when it came time to remove them. But something that any given > developer could set once and then always have developer stuff > available, > and toggle off when they wanted to test a production build, could make > things a lot nicer.) I often add strace and ltrace :-) Logging and extra access to the internals of host/VM are also good. I agree about thati it's good idea to have developer-support features in place and have an ability to make production and development builds with a simple switch between them. -- vadik likholetov [-- Attachment #2: Type: text/html, Size: 1275 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Developer-friendliness 2022-12-06 20:19 ` Vadim Likholetov 2022-12-06 20:31 ` Developer-friendliness Alyssa Ross @ 2022-12-06 20:59 ` Michael Raskin 2022-12-06 21:14 ` Developer-friendliness Alyssa Ross 2022-12-06 22:05 ` Developer-friendliness Michael Raskin 1 sibling, 2 replies; 18+ messages in thread From: Michael Raskin @ 2022-12-06 20:59 UTC (permalink / raw) To: alyssa.ross, vadim.likholetov; +Cc: devel >> It may sound like religious point of view, but I strongly believe >> that developers is the number one priority to support with Spectrum OS >> now — as easier and friendlier the OS will be to the developer as >> faster it will be delivered to the common users. It, from my point of >> view, includes two main aspects — ease of development giving the >> developer proper “bricks” (well encapsulated OS services) and >> examples, that coveres main use cases/use patterns of the operating >> system. > >Yes, that's a compelling argument. I wonder if it would make sense to >add some special configuration option for development, that would enable >extra development tools and other goodies? (I often find myself adding >strace…) > >(I think they probably shouldn't be present by default, because then it >could be difficult to be sure nothing was depending on them in future >when it came time to remove them. But something that any given >developer could set once and then always have developer stuff available, >and toggle off when they wanted to test a production build, could make >things a lot nicer.) Well, one could have VM generator functions, so that you can easily generate a development-like VM or a reasonable-setup like VM or minimal VM (without D-Bus etc.), and so the developers can just use the dev version and always get the good stuff included? >And BTW, thanks so much for spending all this time engaging with this >stuff! The project will be much better off for your efforts. :) ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Developer-friendliness 2022-12-06 20:59 ` Developer-friendliness Michael Raskin @ 2022-12-06 21:14 ` Alyssa Ross 2022-12-06 22:05 ` Developer-friendliness Michael Raskin 1 sibling, 0 replies; 18+ messages in thread From: Alyssa Ross @ 2022-12-06 21:14 UTC (permalink / raw) To: Michael Raskin; +Cc: vadim.likholetov, devel [-- Attachment #1: Type: text/plain, Size: 3135 bytes --] On Tue, Dec 06, 2022 at 09:59:22PM +0100, Michael Raskin wrote: > >> It may sound like religious point of view, but I strongly believe > >> that developers is the number one priority to support with Spectrum OS > >> now — as easier and friendlier the OS will be to the developer as > >> faster it will be delivered to the common users. It, from my point of > >> view, includes two main aspects — ease of development giving the > >> developer proper “bricks” (well encapsulated OS services) and > >> examples, that coveres main use cases/use patterns of the operating > >> system. > > > >Yes, that's a compelling argument. I wonder if it would make sense to > >add some special configuration option for development, that would enable > >extra development tools and other goodies? (I often find myself adding > >strace…) > > > >(I think they probably shouldn't be present by default, because then it > >could be difficult to be sure nothing was depending on them in future > >when it came time to remove them. But something that any given > >developer could set once and then always have developer stuff available, > >and toggle off when they wanted to test a production build, could make > >things a lot nicer.) > > Well, one could have VM generator functions, so that you can easily > generate a development-like VM or a reasonable-setup like VM or minimal > VM (without D-Bus etc.), and so the developers can just use the dev > version and always get the good stuff included? Yes, that's true. Just as a quick refresher, currently if you want to make a Spectrum VM, these are your options: - Lay out the VM configuration directory manually in the user data partition, and bring your own kernel/rootfs. - As above, but use the rootfs and kernel in /usr/img/appvm. These aim to provide a "standard" environment that can be shared between application VMs, so it comes with mesa, and in future virtio-gpu set up, as well as D-Bus, PipeWire, etc. It mounts a second block device that contains your application, as well as any dependencies not included in the base image, and then runs your application from there. This way, big stuff like mesa only has to exist once on disk. - Create an application VM that uses /usr/img/appvm with Nix, using vm-lib/make-vm.nix. So what we're missing is a way to create, with Nix, a VM that isn't based on the /usr/img/appvm "template". The reason we don't currently have such a function is that I don't think it's clear yet how it should work — what should be customisable, what should be part of the base image vs. not, etc.? For a while, how make-vm.nix should work was similarly unclear, which is why it only recently came into existence. After I'd manually created my third or so application VM, it became clear what could be part of the base image, and what had to be customisable. Currently, we only have one VM in the tree that doesn't use the /usr/img/appvm template, netvm. Once we have more, I think it'll be easier to evaluate how a more minimal VM builder function should work. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Developer-friendliness 2022-12-06 20:59 ` Developer-friendliness Michael Raskin 2022-12-06 21:14 ` Developer-friendliness Alyssa Ross @ 2022-12-06 22:05 ` Michael Raskin 1 sibling, 0 replies; 18+ messages in thread From: Michael Raskin @ 2022-12-06 22:05 UTC (permalink / raw) To: alyssa.ross; +Cc: vadim.likholetov, devel >> Well, one could have VM generator functions, so that you can easily >> generate a development-like VM or a reasonable-setup like VM or minimal >> VM (without D-Bus etc.), and so the developers can just use the dev >> version and always get the good stuff included? > >Yes, that's true. > >Just as a quick refresher, currently if you want to make a Spectrum VM, >these are your options: > > - Lay out the VM configuration directory manually in the user data > partition, and bring your own kernel/rootfs. > > - As above, but use the rootfs and kernel in /usr/img/appvm. These aim > to provide a "standard" environment that can be shared between > application VMs, so it comes with mesa, and in future virtio-gpu set > up, as well as D-Bus, PipeWire, etc. It mounts a second block device > that contains your application, as well as any dependencies not > included in the base image, and then runs your application from > there. This way, big stuff like mesa only has to exist once on disk. > > - Create an application VM that uses /usr/img/appvm with Nix, using > vm-lib/make-vm.nix. Yes, sorry, I have still not tried all this (I am going to avoid Wayland as the main UI as long as practical, and I am not going to commit enough time to Spectrum-adjacent things to make it worth your time to ask how would I set up things for Spectrum-like isolation of single things into Weston-inside-Xorg or whatever on host). >So what we're missing is a way to create, with Nix, a VM that isn't >based on the /usr/img/appvm "template". The reason we don't currently >have such a function is that I don't think it's clear yet how it should >work — what should be customisable, what should be part of the base >image vs. not, etc.? For a while, how make-vm.nix should work was >similarly unclear, which is why it only recently came into existence. >After I'd manually created my third or so application VM, it became >clear what could be part of the base image, and what had to be >customisable. Currently, we only have one VM in the tree that doesn't >use the /usr/img/appvm template, netvm. Once we have more, I think >it'll be easier to evaluate how a more minimal VM builder function >should work. Hm, I would say that I have a good experience solving such a problem with a Nixpkgs-overlay-like structure, i.e. a makeExtensible with all possible tunables defined as attributes, and then more complex data structures built out of them, as well as packages for the things. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Firefox appVM patches and appVM refactoring 2022-12-06 17:10 ` Alyssa Ross 2022-12-06 20:12 ` Vadim Likholetov 2022-12-06 20:19 ` Vadim Likholetov @ 2022-12-07 8:04 ` Ville Ilvonen 2022-12-07 11:09 ` vadik likholetov 2 siblings, 1 reply; 18+ messages in thread From: Ville Ilvonen @ 2022-12-07 8:04 UTC (permalink / raw) To: Alyssa Ross, Vadim Likholetov; +Cc: devel On 12/6/22 19:10, Alyssa Ross wrote: > On Tue, Dec 06, 2022 at 05:57:19PM +0200, Vadim Likholetov wrote: >> I’ve done this — I have patched cloud-hypervisor to have >> three console devices — serial, console and user-console, but than >> decided that this patches will never go to C-H mainline so got back >> with more traditional approach with tmux :) > > I'd be pretty optimistic about the chances of a patch that just made it > so you could provide multiple consoles the same way you can provide > multiple block devices. Like "--console pty file=/path/to/console.out" > on the command like to make two consoles, one going to a pty and the > other to a file. Vadim, please go ahead and create a PR to C-H upstream. There's good chance your work would get accepted. Let's see their comments and and feedback. -Ville ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Firefox appVM patches and appVM refactoring 2022-12-07 8:04 ` Firefox appVM patches and appVM refactoring Ville Ilvonen @ 2022-12-07 11:09 ` vadik likholetov 0 siblings, 0 replies; 18+ messages in thread From: vadik likholetov @ 2022-12-07 11:09 UTC (permalink / raw) To: Ville Ilvonen; +Cc: Alyssa Ross, devel [-- Attachment #1: Type: text/plain, Size: 345 bytes --] On ke, joulu 7 2022 at 10.04.28 +02:00:00, Ville Ilvonen <ville.ilvonen@unikie.com> wrote: > > Vadim, please go ahead and create a PR to C-H upstream. There's good > chance your work would get accepted. Let's see their comments and and > feedback. > > -Ville Ok, I will. It will take couple of days to clean them up. -- vadik likholetov [-- Attachment #2: Type: text/html, Size: 557 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2022-12-07 11:09 UTC | newest] Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-12-04 22:42 Firefox appVM patches and appVM refactoring Vadim Likholetov 2022-12-06 11:42 ` Alyssa Ross 2022-12-06 13:56 ` Vadim Likholetov 2022-12-06 15:18 ` Alyssa Ross 2022-12-06 15:20 ` Alyssa Ross 2022-12-06 15:57 ` Vadim Likholetov 2022-12-06 17:10 ` Alyssa Ross 2022-12-06 20:12 ` Vadim Likholetov 2022-12-06 20:25 ` Alyssa Ross 2022-12-07 7:41 ` vadik likholetov 2022-12-06 20:19 ` Vadim Likholetov 2022-12-06 20:31 ` Developer-friendliness Alyssa Ross 2022-12-07 7:46 ` Developer-friendliness vadik likholetov 2022-12-06 20:59 ` Developer-friendliness Michael Raskin 2022-12-06 21:14 ` Developer-friendliness Alyssa Ross 2022-12-06 22:05 ` Developer-friendliness Michael Raskin 2022-12-07 8:04 ` Firefox appVM patches and appVM refactoring Ville Ilvonen 2022-12-07 11:09 ` vadik likholetov
Code repositories for project(s) associated with this public inbox https://spectrum-os.org/git/crosvm https://spectrum-os.org/git/doc https://spectrum-os.org/git/mktuntap https://spectrum-os.org/git/nixpkgs https://spectrum-os.org/git/spectrum https://spectrum-os.org/git/ucspi-vsock https://spectrum-os.org/git/www This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).