Michael Raskin <7c6f434c@mail.ru> writes:

>>> Given that it is sometimes convenient to spawn multiple VMs with almost 
>>> identical settings on the fly (e.g. Firefox instances but with different
>>> state directories for downloads/uploads), should we commit to define
>>> a CLI layer for starting VMs (probably similar to the underlying VMM
>>> software CLI, but adapted to the notions we want to use), with the 
>>> Nix-based management of VM fleet built on top of this CLI?
>>
>>Yeah, I've been thinking about this too and agree.  We should have a nix
>>run analogue for temporary VMs.  It would be nice for scripts that use
>>multiple VMs to be able to start them on the fly as well.  (But we
>>should definitely not have a nix-env -i analogue, because there should
>>be no hidden state on a host.)>
>
> I think it is better to avoid evaluation, actually.
>
> So I want a level where there is a Spectrum Profile Generation with 
> named prebuilt VMs complexity levels (console-only, for stuff like 
> ImageMagick on images downloaded from 4chan; GUI-but-no-HW, to remove 
> GPU attack surface and guarantee lack of sound; full-GUI, which is still
> not guaranteed to get access to everything, of course), and prebuilt 
> application sets (just buildEnv stuff, no fancy stuff).
>
> Then I say:
>
> spectrum-vm --base full-gui --net nat --app firefox-esr-js-profile \
> --app gvim --dri card0 --rw-directory user:conf/nixcon-2020 \
> -- firefox https://nixos.org/
>
> This should not do anything in run time that can be safely cached. VM is
> prebuilt, application bunches are prebuilt and just mounted into store,
> even Firefox profile is prebuilt and changes are going to be discarded.
> No builds, no Nix evaluations.
>
> (yes, there is also the annoying question of RAM amount management, not
> sure how to do it well, ballooning would be the nicest thing if it is 
> feasible)

I think I had imagined you building a "firefox" VM, but yeah, maybe
there's really no point to that.  In which case, I'd be happy for this
sort of thing to be handled outside of Nix.

>>> I suspect that building things the other way round is not much simpler,
>>> but building a single VM image usable with slightly different settings
>>> will have smaller overhead than evaluating a Nix expression for each
>>> extra VM.
>>
>>Maybe.  I'll have to think more about that.  tbh I think we can assume
>>that the evaluator will get faster at some point (because there's big
>>potential for optimisations and caching in safe mode).  It might be nice
>
> Oh well, in the safe mode there is even a potential for precompiling Nix
> functions. It's another question whether we want to experiment with
> workflows without having access to such speedups.
>
> And even a pure evaluation of a checkout needs to stat all the relevant
> files (even if we accept the risk of not checksumming them).

Another good point.

>>to invent yet another way to do options, and just keep that in Nix.  But
>>I could be swayed either way.
>
> I assume there are good arguments for the config to be structured (and
> in my current system there are some interface boundaries with structured
> config and some wrapper scripts that manage to generate almost all the 
> usecases using command-line arguments, but sometimes parts of the 
> structure need to be passed to them).
>
> On the other hand, JSON can be parsed and written easily in all
> languages we might want to use, even in Bash using jq. And we need some
> code to setup the namespaces, but nsjail/firejail/bubblewrap are C code
> so in the long term there would be some rewrite, probably in Rust??? And
> having Nix preprocess arguments for Rust code sounds strange.

Not sure about this -- I think it's something we'll have to experiment
with.  Maybe structured configuration isn't necessary -- it's a pain to
do in a CLI...

> And whatever you do, Nix evaluation is always just another layer before
> running the actual code for setting up the isolated environments that 
> still needs to interpret its arguments.
>
> Also, I might want to use some binary cache, but when I am offline Nix
> builds wait for a long time for a reply from a binary cache. It's not so
> bad if I am intentionally building something and can pass an empty value
> for binary-caches via the command line, but doing it for each new 
> command I execute sounds excessive.

Yes, this is true, although I consider it a Nix bug.  I shouldn't have
to remember to say --option substitute false for trivial offline
rebuilds.

> Also, it might be convenient to pass entire Nix store inside some VMs,
> but if all the _environments_ of all the currently running VMs are 
> visible in the Nix store??? I really shouldn't.
>
> I guess if I mention multiuser systems you would say that we need to
> allow chmod a-x on /nix/store?

I don't plan on doing anything to accomodate multi-user systems, at
least for now.