January 2023 - Spectrum Development

[PATCH 00/22] Implement managing VMs with Nix
by Alyssa Ross 26 Feb '23

26 Feb '23

IMPORTANT NOTE: this series should be applied on top of v2 of my previous series "Introduce a shared base for application VMs" [1]. I'm much happier with v2 of that series, but I only posted it yesterday so I still want to leave a little more opportunity for comment before applying it. [1]: https://spectrum-os.org/lists/archives/spectrum-devel/20221009114036.463071… This series contains the final big chunk of work I had left to do on Spectrum's original NLnet grant. It adds support for managing Spectrum VMs from the Spectrum system itself using Nix. Nix is optional, and can co-exist with VMs provided in some other way. More information is included in the new documentation. Most of this work was done earlier this year, but I got stuck on some implementation details that prevented me from getting over the last hurdle until I came up with a solution. That's explained in more detail in patch 15. Patches 1–10 add support for configuring VMs with read/write access to host directories using virtiofs. Then, in patches 11–14, come various changes that make the default user data partition more suitable as a mutable filesystem, which we haven't actually used it for before. And then the remaining patches actually implement support for a VM that can run Nix and easily build VMs that are available on the host. Alyssa Ross (22): host/start-vm: use MAP_SHARED memory for VMs host/start-vm: implement shared directories host/rootfs: generate virtiofsd services Documentation: explain VM shared directories vm-lib/make-vm.nix: support shared directories img/app: add support for testing virtiofs img/app: don't block app startup on network online img/app: auto-mount virtiofs0 filesystem vm/app/mg.nix: init vm/app/mg.nix: open virtio filesystem in dired host/rootfs: move ext mounting to s6-rc service host/rootfs: automatically grow user partition host/rootfs: use a bigger test ext partition host/initramfs/extfs.nix: tar2ext4 -> mkfs.ext4 -d host/start-vm: resolve VM symlinks with /ext root host/rootfs: resolve VM symlinks with /ext root Documentation: explain /ext symlink resolution host/start-vm: increase memory size to 512M vm/app/nix: add vm-lib/make-vms.nix: add host/initramfs/extfs.nix: add example Nix-built VM Documentation: add how-to guide for Nix-built VMs .gitignore | 5 +- Documentation/_sass/custom/custom.scss | 22 ++++++ Documentation/creating-vms.adoc | 12 ++- Documentation/nix-vms.adoc | 22 ++++++ host/initramfs/extfs.nix | 29 +++++-- host/rootfs/Makefile | 13 +++- host/rootfs/default.nix | 16 +++- host/rootfs/etc/mdev/block/add | 1 - host/rootfs/etc/s6-rc/ext-rc-init/up | 19 ++++- host/rootfs/etc/s6-rc/ext/up | 5 +- host/rootfs/etc/template/fs/notification-fd | 1 + .../etc/template/fs/notification-fd.license | 2 + host/rootfs/etc/template/fs/run | 11 +++ host/rootfs/etc/template/fs/type | 1 + host/rootfs/etc/template/fs/type.license | 2 + host/start-vm/fs.c | 17 +++++ host/start-vm/fs.rs | 68 +++++++++++++++++ host/start-vm/lib.rs | 54 ++++++++++--- host/start-vm/meson.build | 2 +- host/start-vm/start-vm.rs | 15 ++-- host/start-vm/tests/meson.build | 4 + host/start-vm/tests/vm_command-basic.rs | 6 +- .../tests/vm_command-config-symlink.rs | 30 ++++++++ host/start-vm/tests/vm_command-shared-dir.rs | 43 +++++++++++ img/app/Makefile | 24 +++++- img/app/etc/mdev.conf | 3 +- img/app/etc/mdev/iface | 4 +- img/app/etc/mdev/listen | 12 +++ img/app/etc/mdev/virtiofs | 10 +++ img/app/etc/mdev/wait | 15 ++++ img/app/etc/s6-rc/ok-all/contents | 1 + img/app/shell.nix | 6 +- tools/resolve_in_root/default.nix | 23 ++++++ tools/resolve_in_root/meson.build | 10 +++ tools/resolve_in_root/resolve_in_root.c | 76 +++++++++++++++++++ tools/resolve_in_root/test.sh | 11 +++ vm-lib/make-vm.nix | 20 ++++- vm-lib/make-vms.nix | 19 +++++ vm/app/catgirl.nix | 1 + vm/app/lynx.nix | 1 + vm/app/{lynx.nix => mg.nix} | 10 ++- vm/app/nix/bin/vm-rebuild | 25 ++++++ vm/app/nix/default.nix | 43 +++++++++++ vm/app/nix/example.nix | 13 ++++ 44 files changed, 673 insertions(+), 54 deletions(-) create mode 100644 Documentation/_sass/custom/custom.scss create mode 100644 Documentation/nix-vms.adoc create mode 100644 host/rootfs/etc/template/fs/notification-fd create mode 100644 host/rootfs/etc/template/fs/notification-fd.license create mode 100755 host/rootfs/etc/template/fs/run create mode 100644 host/rootfs/etc/template/fs/type create mode 100644 host/rootfs/etc/template/fs/type.license create mode 100644 host/start-vm/fs.c create mode 100644 host/start-vm/fs.rs create mode 100644 host/start-vm/tests/vm_command-config-symlink.rs create mode 100644 host/start-vm/tests/vm_command-shared-dir.rs create mode 100755 img/app/etc/mdev/listen create mode 100755 img/app/etc/mdev/virtiofs create mode 100755 img/app/etc/mdev/wait create mode 100644 tools/resolve_in_root/default.nix create mode 100644 tools/resolve_in_root/meson.build create mode 100644 tools/resolve_in_root/resolve_in_root.c create mode 100755 tools/resolve_in_root/test.sh create mode 100644 vm-lib/make-vms.nix copy vm/app/{lynx.nix => mg.nix} (52%) create mode 100755 vm/app/nix/bin/vm-rebuild create mode 100644 vm/app/nix/default.nix create mode 100644 vm/app/nix/example.nix -- 2.37.1

2 44

[PATCH] Documentation: add contributing information
by Jenni Nikolaenko 20 Jan '23

20 Jan '23

Hello there! I decided to create a patch from my drafts so this information will not be lost. It still needs some attention. It is a good idea to have the documentation in one place. For us this means that information from these pages https://spectrum-os.org/contributing.html, https://spectrum-os.org/participating.html -- should be moved here https://spectrum-os.org/participating.html. What I did particularly: 1. Added a new chapter "Contributing". Copied here the information from here https://spectrum-os.org/contributing.html, adapting a bit to Spectrum Docs. 2. Moved the "Building Documentation" section from "Development" to "Contributing". 3. In the "Contributing" chapter you can find the following structure: - Communication in Spectrum (copied from https://spectrum-os.org/participating.html) - Documentation Style Guide - Building Documentation I tried to fix all cross-reference links, so this new content can be used right now. Any further updates are very welcome. Signed-off-by: Jenni Nikolaenko <evgeniia.nikolaenko(a)unikie.com> --- .../building-documentation.adoc | 6 +- Documentation/contributing/communication.adoc | 89 +++++++++++++++ Documentation/contributing/index.adoc | 95 ++++++++++++++++ .../contributing/writing_documentation.adoc | 104 ++++++++++++++++++ Documentation/development/index.adoc | 7 +- Documentation/development/uuid-reference.adoc | 2 +- Documentation/index.adoc | 4 +- 7 files changed, 299 insertions(+), 8 deletions(-) rename Documentation/{development => contributing}/building-documentation.adoc (90%) create mode 100644 Documentation/contributing/communication.adoc create mode 100644 Documentation/contributing/index.adoc create mode 100644 Documentation/contributing/writing_documentation.adoc diff --git a/Documentation/development/building-documentation.adoc b/Documentation/contributing/building-documentation.adoc similarity index 90% rename from Documentation/development/building-documentation.adoc rename to Documentation/contributing/building-documentation.adoc index 4f464f5..63a3038 100644 --- a/Documentation/development/building-documentation.adoc +++ b/Documentation/contributing/building-documentation.adoc @@ -1,6 +1,6 @@ = Building Documentation -:page-parent: Development -:page-nav_order: 5 +:page-parent: Contributing +:page-nav_order: 3 // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -49,4 +49,4 @@ modify any of those, or add new ones, run `scripts/build.sh` again to do a full rebuild of the site. After making changes to the documentation, see how to -xref:first-patch.adoc[send your patch] and submit the changes for review. +xref:../development/first-patch.adoc[send your patch] and submit the changes for review. diff --git a/Documentation/contributing/communication.adoc b/Documentation/contributing/communication.adoc new file mode 100644 index 0000000..8b92d56 --- /dev/null +++ b/Documentation/contributing/communication.adoc @@ -0,0 +1,89 @@ += Communication in Spectrum +:description: Channels, announcements and so on. +:page-nav_order: 1 +:page-parent: Contributing + +// SPDX-FileCopyrightText: 2023 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +There are several places online in which announcements, +discussion, and collaboration happen. Rather than tell +you about them all at once, though, it is best to tell you +which ones are relevant to you, given how you would like +to participate (or not) in the project. + +Our primary methods of communication are mailing lists and IRC. +If this is not how you are used to interacting with free +software projects, *DON'T PANIC*. + +We provide instructions for how to get started, and several +different interfaces for you to choose from depending on what workflow +(web, mail, Atom, etc.) makes you most comfortable. Details of these +will be explained further on, but first, you should figure out what +communication channels are actually relevant for you. + +If you are interested in keeping up to date with announcements from +Spectrum's developers, subscribe to *announce(a)spectrum-os.org*. +You can discuss Spectrum and related topics in real time on IRC, or +more long-form on spectrum-announce (_discuss(a)spectrum-os.org_). +Finally, development work happens in Spectrum's +xref:https://spectrum-os.org/git/[source repositories], and is +discussed on spectrum-devel (_devel(a)spectrum-os.org_). + + +== Internet Relay Chat + +There is a single Internet Relay Chat (IRC) channel, +xref:ircs://irc.libera.chat:6697/spectrum[spectrum:libera.chat] +on xref:https://libera.chat/[Libera.Chat], for all kinds of Spectrum +discussions.You can also join through xref:https://matrix.org/[Matrix] as +xref:https://matrix.to/#/#spectrum:libera.chat[spectrum:libera.chat]. +If you are new to IRC, you might find Matrix the easiest way to get started. +The channel is *logged*, but the logs are not available online yet. + + +== Mailing Lists + +If you are interested in participating in the project, or even just +following along with it, the best thing to do is subscribe to one or +more of the Spectrum mailing lists. + +If mailing lists are not your thing, that is okay! Try the +xref:https://spectrum-os.org/lists/hyperkitty/[Web UI], which allows +you to interact with the mailing lists much like a modern web forum. + +Each list has xref:https://spectrum-os.org/lists/archives/[archives] +that can also be accessed via the web, NNTP or Atom. You are strongly +encouraged to mirror them. + +[cols="1,1"] +|=== + +|announce(a)spectrum-os.org +|This is a one-way list where announcements +from Spectrum's developers will be posted. + +You can subscribe +xref:https://spectrum-os.org/lists/mailman3/lists/announce.spectrum-os.org/[on the web], +or by sending mail to *announce-subscribe(a)spectrum-os.org*. + +|discuss(a)spectrum-os.org +|This list contains high-level discussion about the project. + +You can subscribe +xref:https://spectrum-os.org/lists/mailman3/lists/discuss.spectrum-os.org/[on the web], +or by sending mail to *discuss-subscribe(a)spectrum-os.org*. + +|devel(a)spectrum-os.org +|This list is where low-level technical discussions happen, +and where patches are sent. + +There is a separate guide for the Spectrum development process, +including how patches are sent to this list and discussed. +For more information, see +xref:development/working-with-patches.adoc[Working with Patches]. + +You can subscribe +xref:https://spectrum-os.org/lists/mailman3/lists/devel.spectrum-os.org/[on the web], or by sending mail to *devel-subscribe(a)spectrum-os.org*. + +|=== \ No newline at end of file diff --git a/Documentation/contributing/index.adoc b/Documentation/contributing/index.adoc new file mode 100644 index 0000000..71ffe0e --- /dev/null +++ b/Documentation/contributing/index.adoc @@ -0,0 +1,95 @@ += Contributing +:description: How to contribute changes to the Spectrum source repository. +:page-nav_order: 5 +:page-has_children: true + +// SPDX-FileCopyrightText: 2023 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Thank you so much for your interest in contributing to Spectrum. +Having a healthy community of people who feel empowered to work +on the project is vital to its success. + +If you have any questions about the project, please +xref:../contributing/communication.adoc[get in touch] with +the community. We will gladly help you out! + + +== Contributing Code + +If you are interested in contributing code to Spectrum, +you can find the source repositories online: https://spectrum-os.org/git/. + +If you have made changes you would like to be included +in the official Spectrum distribution, please send patches +to the spectrum-devel mailing list (devel(a)spectrum-os.org). +For more information, see +xref:../development/working-with-patches.adoc[Working with Patches]. + +Sending patches to the list is easy — you can do it with Git, +straight from the command line. You do not have to set anything up +in your email client, and you do not have to subscribe to the list. +The only thing you need to do is probably to set up Git so it knows +how to send email using your provider's SMTP server. +These instructions https://git-send-email.io/ should be enough +to get you going. If you are still stuck, send an email +to *devel(a)spectrum-os.org* and we will help you out. + +Once you have Git configured correctly, sending your changes to the +list should be as simple as something like: + +[listing] +[source,shell] +git send-email origin/master --to devel(a)spectrum-os.org + +You will be CCed on replies to your posts, so you do not even have to be +subscribed to the mailing list if you are worried about too much traffic. +There is also a web interface https://spectrum-os.org/lists/hyperkitty/ +that you can use to browse or even post to the mailing list. + +Add a Signed-off-by line to each patch you submit, to indicate your +certification of the xref:https://spectrum-os.org/git/spectrum/tree/DCO-1.1.txt[Developer's Certificate of Origin] for that patch. +The easiest way to do this is with `git commit's -s` flag. + +Do not be too afraid of getting it wrong the first couple of times. +This might well be a new way of collaborating for you, and everybody +understands that. Hopefully, you will soon realise just how effective +this style of collaboration can be compared to the alternatives. + +== Contributing Documentation + +If you notice something that can be fixed or improved: + +. xref:https://spectrum-os.org/doc/development/first-patch.html#making-change… or add any relevant documentation]. ++ +TIP: Make sure your changes are formatted correctly and consistently +with the rest of the documentation. For information on writing guidelines, +see xref:../contributing/writing_documentation.adoc[Documentation Style Guide]. ++ +. xref:../contributing/building-documentation.adoc[Test your documentation]. +Reread what you wrote and run a spellchecker on it to make sure you did not miss anything. +. xref:https://spectrum-os.org/doc/development/first-patch.html#submitting-ch… changes]. + + +=== Documentation To-Do List + +Spectrum documentation needs are: + +* Reporting Bugs +* Development conventions/style (code review, naming) +* Integration, code philosophy + +The list will be updated. Improvements to the documentation are welcome! + +== Contributing Other Things + +At this stage, it is difficult to say what things would be useful +to the project beyond code and documentation. Artwork will be +extremely important and valuable. But, do not let that dissuade you! +At some point, we will need all sorts of things, and we might as well +get the work done early if people are willing to do it. + +If you think you have skills to offer the project beyond writing code, +we would love to hear from you. The best thing to do is to either talk +with us on spectrum-discuss, or in real time on IRC. +For more information, see xref:../contributing/communication.adoc[Communication in Spectrum]. \ No newline at end of file diff --git a/Documentation/contributing/writing_documentation.adoc b/Documentation/contributing/writing_documentation.adoc new file mode 100644 index 0000000..9c1d5fb --- /dev/null +++ b/Documentation/contributing/writing_documentation.adoc @@ -0,0 +1,104 @@ += Documentation Style Guide +:description: Channels, announcements and so on. +:page-nav_order: 2 +:page-parent: Contributing + +// SPDX-FileCopyrightText: 2023 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Here you can find the standards we follow for writing, +formatting, and organizing the Spectrum documentation. +Please follow these guidelines and conventions +when editing the documentation. + +We use xref:https://nixos.org/manual/nix/stable/introduction.html[Nix] +and xref:https://github.com/just-the-docs/just-the-docs[Just the Docs] +for building the documentation. Sources are written in AsciiDoc +If you are new with it, +see xref:https://docs.asciidoctor.org/asciidoc/latest/[AsciiDoc Language Documentation]. + +If you want to test your changes, +see xref:../contributing/building-documentation.adoc[Building Documentation]. + +== Philosophy + +* 1 change = 1 patch +* Update the documentation with the code. +* Organize the AsciiDoc files into subdirectories. +* Keep separate documentation for every different process +to avoid confusion. +* Do not assume that readers know everything you currently know. +* Avoid jargon and acronyms, if you can. +* Avoid multiple variations or spellings to refer to the same +function, element, and so on. +* Write short and useful documents. Cut out everything unnecessary, +while also making a habit of continually improving every doc +to suit your changing needs. + + +== Document Layout + +* xref:https://docs.asciidoctor.org/asciidoc/latest/document/header/[Document header]: +include your name and a revision line following the author line. Example: + + = Document Title + :description: Some words about the content. + Author Name <author(a)example.org> + v1.0, 2012-02-10 + //empty line that ends the document header + +* Licensing: include license information after a document header. For example: + + // SPDX-FileCopyrightText: year / Name, surname / email address or the company name + // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +* Double space: the double space after the period is deliberate. +Some people find it improves readability, and others do not like it, +but the reason we use it in Spectrum documentation is that it allows +text editors (Emacs) to differentiate between the ends of sentences +and abbreviations (like "etc."), which means that you can make commands +like "move to the next sentence" work accurately even if those +abbreviations are used. + +* Character line limit: +** Keep text hard-wrapped at 70-80 characters. (Most editors should be able +to do this automatically.) This makes it easier to pick out specific parts +in review, and also makes it easier to read in some editors. (It does not make a difference to the result as rendered in the web browser.) +** Keep the text in the description attribute hard-wrapped as well. For more information, see the xref:https://docs.asciidoctor.org/asciidoc/latest/document/metadata/#descri… Metadata] section of the Asciidoctor Docs. + +* Put one sentence on each line. This makes it easy to move content around, and also easy to spot (too) long sentences. For more information, see xref:https://asciidoctor.org/docs/asciidoc-recommended-practices/#one-sente… Recommended Practices]. + +== Structure + +* Use subheading to organize information. +* One topic per article. +* To make accents, use xref:https://docs.asciidoctor.org/asciidoc/latest/blocks/admonitions/[admon…. +* Use numbered lists for actions that happen in sequence. +* Do not use parenthesis for additional information, make a separate sentence instead. + + +== Language + +=== Plain English + +Plain language is a universal language that makes information clear and better to understand. + +* Short, sharp sentences. +* Use simple tenses. +* Write in an active voice. +* Understandable language, fewer gerunds. +* Do not contract the words: use cannot instead of can’t. +* Write the full name when first mentioned with the acronym in brackets. +* Use numerals for numbers. +* Do not use Latin words. For example: +** perform operations, *etc.* => perform operations, *and so on* +** *e.g.* a Microsoft SQL Server => *for example,* a Microsoft SQL Server +** *via* the system => *through* the system + +=== Spelling + +* Use standard United States (U.S.) English. In cases where US spelling differs +from UK spelling, use the US spelling. There is no need to fix one by +replacing it with the other. +* When you use filenames, URLs, and data parameters in examples, try to avoid words that are spelled differently by different groups of English speakers. +* Use serial (Oxford) commas. diff --git a/Documentation/development/index.adoc b/Documentation/development/index.adoc index 1fccdd5..6b48418 100644 --- a/Documentation/development/index.adoc +++ b/Documentation/development/index.adoc @@ -15,13 +15,16 @@ Spectrum is free software, currently under active development. * https://nixos.org/manual/nix/stable/introduction.html[Nix package manager] * https://docs.asciidoctor.org/[AsciiDoc] for writing the documentation +TIP: For information on writing guidelines, +see xref:../contributing/writing_documentation.adoc[Documentation Style Guide]. + == Mailing Lists The Spectrum project runs several -https://spectrum-os.org/participating.html#mailing-lists[mailing lists] on +xref:../contributing/communication.adoc[mailing lists] on which you can ask your questions or help other people with the questions they have. All the Spectrum developers as well as many long time Linux and Spectrum users are on the lists. For real-time feedback, use -https://spectrum-os.org/participating.html#irc[IRC/Matrix channel]. +xref:../contributing/communication.adoc[IRC/Matrix channel]. diff --git a/Documentation/development/uuid-reference.adoc b/Documentation/development/uuid-reference.adoc index 81300ee..5247726 100644 --- a/Documentation/development/uuid-reference.adoc +++ b/Documentation/development/uuid-reference.adoc @@ -55,6 +55,6 @@ The Spectrum installer system. == Finding Undocumented UUIDs -Running the Documentation/scripts/undocumented-uuids.sh script from +Running the `Documentation/scripts/undocumented-uuids.sh` script from the root of the Spectrum repository will scan the entire tree for UUIDs not mentioned in this document. diff --git a/Documentation/index.adoc b/Documentation/index.adoc index 51a6e7d..cc25fd3 100644 --- a/Documentation/index.adoc +++ b/Documentation/index.adoc @@ -28,5 +28,5 @@ Once you are up and running, see xref:development/index.adoc[Development] to understand how to work with Spectrum's code and participate in the collaborative development process. -If you are thinking of contributing to the Spectrum docs, see - xref:development/building-documentation.adoc[Building Documentation]. +If you are thinking of contributing to the Spectrum code or docs, see + xref:contributing/index.adoc[Contributing]. -- 2.34.1

3 2

[PATCH] rootfs: make rootfs kernel configurable
by Ivan Nikolaenko 10 Jan '23

10 Jan '23

This patch makes it possible to use custom linux kernel. This is needed in case of porting Spectrum OS to the other hardware than generic-x86. To override rootfs kernel, define "spectrum_kernel" in the customization overlay. Signed-off-by: Ivan Nikolaenko <ivan.nikolaenko(a)unikie.com> --- host/rootfs/default.nix | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index 0a84f55..b853250 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -81,7 +81,7 @@ let imports = [ (modulesPath + "/profiles/all-hardware.nix") ]; }); - kernel = pkgs.linux_latest; + kernel = pkgs.spectrum_kernel or pkgs.linux_latest; appvm = import ../../img/app { inherit config; -- 2.34.1

2 2

[PATCH v2 1/2] Add flakes support
by Valentin Kharin 10 Jan '23

10 Jan '23

Signed-off-by: Valentin Kharin <valentin.kharin(a)unikie.com> --- flake.lock | 43 ++++++++++++++++++++++++++++++++ flake.lock.license | 3 +++ flake.nix | 62 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 108 insertions(+) create mode 100644 flake.lock create mode 100644 flake.lock.license create mode 100644 flake.nix diff --git a/flake.lock b/flake.lock new file mode 100644 index 0000000..aa4ee5e --- /dev/null +++ b/flake.lock @@ -0,0 +1,43 @@ +{ + "nodes": { + "flake-utils": { + "locked": { + "lastModified": 1667395993, + "narHash": "sha256-nuEHfE/LcWyuSWnS8t12N1wc105Qtau+/OdUAjtQ0rA=", + "owner": "numtide", + "repo": "flake-utils", + "rev": "5aed5285a952e0b949eb3ba02c12fa4fcfef535f", + "type": "github" + }, + "original": { + "owner": "numtide", + "repo": "flake-utils", + "type": "github" + } + }, + "nixpkgs": { + "locked": { + "lastModified": 1669635185, + "narHash": "sha256-vYg6GjnsEWNWt/4TmfFN9WtQmSXb4S796J2UOfyTcW0=", + "ref": "refs/heads/rootfs", + "rev": "3176ddef4b4cec85faa2f49d29ce74816d452dc0", + "revCount": 429673, + "type": "git", + "url": "https://spectrum-os.org/git/nixpkgs/" + }, + "original": { + "ref": "refs/heads/rootfs", + "type": "git", + "url": "https://spectrum-os.org/git/nixpkgs/" + } + }, + "root": { + "inputs": { + "flake-utils": "flake-utils", + "nixpkgs": "nixpkgs" + } + } + }, + "root": "root", + "version": 7 +} diff --git a/flake.lock.license b/flake.lock.license new file mode 100644 index 0000000..a8a7980 --- /dev/null +++ b/flake.lock.license @@ -0,0 +1,3 @@ +SPDX-License-Identifier: CC0-1.0 +SPDX-FileCopyrightText: 2022 Unikie + diff --git a/flake.nix b/flake.nix new file mode 100644 index 0000000..ab54fed --- /dev/null +++ b/flake.nix @@ -0,0 +1,62 @@ +# SPDX-License-Identifier: MIT +# SPDX-FileCopyrightText: 2022 Unikie + +{ + description = "A compartmentalized operating system"; + + # NOTE: Revision specification format is ?ref=refs%2fheads%2f<BRANCH>&rev=<COMMIT_REVISION> + inputs.nixpkgs.url = + "git+https://spectrum-os.org/git/nixpkgs/?ref=refs%2fheads%2frootfs"; + inputs.flake-utils.url = "github:numtide/flake-utils"; + + nixConfig = { + extra-substituters = [ "https://cache.dataaturservice.se/spectrum/" ]; + trusted-public-keys = [ + "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" + "spectrum-os.org-1:rnnSumz3+Dbs5uewPlwZSTP0k3g/5SRG4hD7Wbr9YuQ=" + ]; + }; + + outputs = { self, nixpkgs, flake-utils }: + let + supportedSystems = with flake-utils.lib.system; [ x86_64-linux aarch64-linux ]; + in flake-utils.lib.eachSystem supportedSystems (system: + let + pkgs = nixpkgs.legacyPackages.${system}; + config = { inherit pkgs; }; + lib = pkgs.lib; + + mkEntryPoint = { name ? builtins.baseNameOf path, path + , enableShell ? true, enablePackage ? true }: + let + shell = { + # NOTE: https://stackoverflow.com/a/43850372 + devShells.${name} = + import (path + "/shell.nix") { inherit config; }; + }; + package = { packages.${name} = import path { inherit config; }; }; + in (if enableShell then shell else { }) + // (if enablePackage then package else { }); + + # Entry point is a directory with shell.nix and default.nix + # This function maps every entry point to corresponding devShell and package + mapEntryPoints = epoints: + builtins.foldl' lib.recursiveUpdate { } (map mkEntryPoint epoints); + in lib.recursiveUpdate (mapEntryPoints [ + { + path = ./.; + enablePackage = false; + } + { path = ./host/initramfs; } + { path = ./host/rootfs; } + { path = ./host/start-vm; } + { path = ./img/app; } + { path = ./release/live; } + { path = ./vm/sys/net; } + ]) { + # Add some other flake schema related stuff here. + # NOTE: flake-utils.lib.eachDefaultSystem automagically adds ${system}. + devShells.documentation = import ./Documentation { inherit config; }; + packages.documentation = import ./Documentation { inherit config; }; + }); +} -- 2.38.1

2 2

[PATCH] rootfs: support default package list extension
by Ivan Nikolaenko 09 Jan '23

09 Jan '23

This patch will allow users to add their packages to the Spectrum OS rootfs. To do so, the user needs to declare a variable "additionalPackages" in his overlay with the list of required packages. Please, note that static packages need to be taken from pkgsStatic list, since additionalPackages are added from pkgs list. Signed-off-by: Ivan Nikolaenko <ivan.nikolaenko(a)unikie.com> --- host/rootfs/default.nix | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index 0a84f55..37c820b 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -75,7 +75,8 @@ let CONFIG_RMMOD n ''; }) - ] ++ (with pkgsGui; [ foot westonLite ]); + ] ++ (with pkgsGui; [ foot westonLite ]) + ++ (pkgs.additionalPackages or []); nixosAllHardware = nixos ({ modulesPath, ... }: { imports = [ (modulesPath + "/profiles/all-hardware.nix") ]; -- 2.34.1

1 0

[PATCH] Documentation: Architecture Decision Record
by Ville Ilvonen 06 Jan '23

06 Jan '23

* ADRs based on discussions with Alyssa * A note on ADRs to architecture.adoc Signed-off-by: Ville Ilvonen <ville.ilvonen(a)unikie.com> --- Documentation/architecture.adoc | 13 ++++++++++ ...architecture-decision-record-template.adoc | 13 ++++++++++ .../decisions/001-host-update-mechanism.adoc | 19 ++++++++++++++ .../decisions/002-install-options.adoc | 18 +++++++++++++ Documentation/decisions/003-partitioning.adoc | 25 +++++++++++++++++++ .../004-data-at-rest-encryption.adoc | 16 ++++++++++++ .../005-virtual-machine-manager.adoc | 24 ++++++++++++++++++ .../decisions/006-drivers-on-host.adoc | 17 +++++++++++++ .../decisions/007-USB-virtual-machine.adoc | 14 +++++++++++ ...008-Inter-VM-communication-mechanisms.adoc | 18 +++++++++++++ 10 files changed, 177 insertions(+) create mode 100644 Documentation/decisions/000-lightweight-architecture-decision-record-template.adoc create mode 100644 Documentation/decisions/001-host-update-mechanism.adoc create mode 100644 Documentation/decisions/002-install-options.adoc create mode 100644 Documentation/decisions/003-partitioning.adoc create mode 100644 Documentation/decisions/004-data-at-rest-encryption.adoc create mode 100644 Documentation/decisions/005-virtual-machine-manager.adoc create mode 100644 Documentation/decisions/006-drivers-on-host.adoc create mode 100644 Documentation/decisions/007-USB-virtual-machine.adoc create mode 100644 Documentation/decisions/008-Inter-VM-communication-mechanisms.adoc diff --git a/Documentation/architecture.adoc b/Documentation/architecture.adoc index 157907f..185740c 100644 --- a/Documentation/architecture.adoc +++ b/Documentation/architecture.adoc @@ -20,6 +20,19 @@ devices and provides network services to application VMs). Refer to xref:creating-vms.adoc[Creating VMs] and xref:running-vms.adoc[Running VMs] for more information about using VMs in Spectrum. +== Architecture Decision Record (ADR) + +https://adr.github.io/[Architecturally significant decisions] are +recorded as https://github.com/joelparkerhenderson/architecture-decision-record/blob/ma… ADRs] + +Status of Spectrum OS ADRs: +Accepted - Implemented and likely not to change. +Proposed - Designed and possibly partially implmented. May change. +Other - Not yet in use. + +Comments and contributions to ADRs are welcome. ADRs can be found at +Documentation/decisions + == The Spectrum host system Compartmentalization is implemented using diff --git a/Documentation/decisions/000-lightweight-architecture-decision-record-template.adoc b/Documentation/decisions/000-lightweight-architecture-decision-record-template.adoc new file mode 100644 index 0000000..087ec44 --- /dev/null +++ b/Documentation/decisions/000-lightweight-architecture-decision-record-template.adoc @@ -0,0 +1,13 @@ +# Title + +## Status +What is the status, such as proposed, accepted, rejected, deprecated, superseded, etc.? + +## Context +What is the issue that we're seeing that is motivating this decision or change? + +## Decision +What is the change that we're proposing and/or doing? + +## Consequences +What becomes easier or more difficult to do because of this change? diff --git a/Documentation/decisions/001-host-update-mechanism.adoc b/Documentation/decisions/001-host-update-mechanism.adoc new file mode 100644 index 0000000..03bbae2 --- /dev/null +++ b/Documentation/decisions/001-host-update-mechanism.adoc @@ -0,0 +1,19 @@ +# Host update mechanism + +## Status +Proposed + +## Context +Spectrum OS has no implementation for software update. The host - consisting of +Linux kernel, KVM, cloud-hypervisor and minimal user space tools - software +updates are required to support feature development and security fixes. + +## Decision +A-B partitioning created by Spectrum installer Installer sets up the system on +partition A of the block device A-B update scheme where user (or installer) +writes the update image to partition B Bootloader provides four boot options: +A, A mutable, B, B mutable + +## Consequences +Default boot selection, incremental updates (e.g. overlays), network update +postponed for later. diff --git a/Documentation/decisions/002-install-options.adoc b/Documentation/decisions/002-install-options.adoc new file mode 100644 index 0000000..f5857c5 --- /dev/null +++ b/Documentation/decisions/002-install-options.adoc @@ -0,0 +1,18 @@ +# Install options + +## Status +Proposed + +## Context +Based on identified different audiences for the Spectrum OS release it is +proposed we support three base configurations to use with Spectrum OS in the +first boot. + +## Decision +* Minimal - Spectrum OS host + system VMs: netvm, guivm, usbvm + home-directory +(optionally encrypted - see 004-disk-encryption.md) +* Common - Minimal + browser app VM + 2-3 selected app VMs +* Power - Common + NixOS VM + +## Consequences +Requires first-boot-vm (like wizard) to support user to get started. diff --git a/Documentation/decisions/003-partitioning.adoc b/Documentation/decisions/003-partitioning.adoc new file mode 100644 index 0000000..345619f --- /dev/null +++ b/Documentation/decisions/003-partitioning.adoc @@ -0,0 +1,25 @@ +# Partitioning + +## Status +Proposed + +## Context +Partitions are required to install the Spectrum OS, VMs and store user data. + +## Decision +---- +<blockdevice> # EFI system partition +<blockdevice> # XBOOTLDR +<blockdevice> # A +<blockdevice> # B +# first 32 GB are reserved for Spectrum system +# rest of the disk is reserved for user data +<blockdevice>n-1 # bootstrap user data +<blockdevice>n to the end of disk # user data +---- + +## Consequences +LVM may support resizing - both increasing and decreasing with some limitation +when there's alreay data on volume(s). Does LVM work with all disk types? We +have to implement XBOOTLDR to support EFI system partition created by Windows - +to support dual boot diff --git a/Documentation/decisions/004-data-at-rest-encryption.adoc b/Documentation/decisions/004-data-at-rest-encryption.adoc new file mode 100644 index 0000000..3ed9abb --- /dev/null +++ b/Documentation/decisions/004-data-at-rest-encryption.adoc @@ -0,0 +1,16 @@ +# Data at rest encryption + +## Status +Proposed + +## Context +To support user data and privacy protection, encryption of data at rest is +required. + +## Decision +User data is encrypted. + +## Consequences +Spectrum OS needs to come with enough SW to get the encryption key via different +methods (password, usb, fido, etc.) Can we use dm-crypt for everything instead +of LUKS? diff --git a/Documentation/decisions/005-virtual-machine-manager.adoc b/Documentation/decisions/005-virtual-machine-manager.adoc new file mode 100644 index 0000000..b4af595 --- /dev/null +++ b/Documentation/decisions/005-virtual-machine-manager.adoc @@ -0,0 +1,24 @@ +# Virtual Machine Manager + +## Status +Accepted + +## Context +rust-vmm-based VMM provides memory and concurrency safe solution. +cloud-hypervisor was chosen because firecrack does not support other +virtio-devices than net or block. crosvm was not chosen because cloud-hypervisor +has more flexible IPC mechanisms, more engaging community as LF-project. +cloud-hypervisor has more core features - such as snapshotting, live migration +and more general hot plugging. crosvm supports more devices we will also need. +It was seen easier to port devices from crosvm to cloud-hypervisor than to port +core features from cloud-hypervisor to crosvm. + +## Decision +Spectrum OS design and implementation decision is to use cloud-hypervisor as the +primary VMM. + +## Consequences +We gotta port some stuff from crosvm to cloud-hypervisor. It's easier for +Spectrum to handle virtualization dynamically with cloud-hypervisor. If the +primary VMM, cloud-hypervisor, is exchanged for trials etc. functionality is +expected to break or not supported. diff --git a/Documentation/decisions/006-drivers-on-host.adoc b/Documentation/decisions/006-drivers-on-host.adoc new file mode 100644 index 0000000..052b596 --- /dev/null +++ b/Documentation/decisions/006-drivers-on-host.adoc @@ -0,0 +1,17 @@ +# Drivers on host + +## Status +Accepted + +## Context +To harden the trusted computing base and make it more minimal, the target is to +minimize the amount of drivers on the Spectrum host kernel. + +## Decision +We are aiming to have as few drivers as possible on the host. + +## Consequences +No networking on the host. Responsibilities of the host are expected to get +smaller over time. More flexible management of devices. We need to decouple +device classes - like net, usb, bluetooth and gui - from host to their +respective VMs. diff --git a/Documentation/decisions/007-USB-virtual-machine.adoc b/Documentation/decisions/007-USB-virtual-machine.adoc new file mode 100644 index 0000000..2072427 --- /dev/null +++ b/Documentation/decisions/007-USB-virtual-machine.adoc @@ -0,0 +1,14 @@ +# USB Virtual Machine + +## Status +Proposed + +## Context +To support specific USB devices on specific VMs + +## Decision +The decision is to pass-through USB controller to a VM with authorization +controls inside the VMs to forward a specific USB device using USBIP. + +## Consequences +We need to modify the upstream USBIP daemon to support authorization. diff --git a/Documentation/decisions/008-Inter-VM-communication-mechanisms.adoc b/Documentation/decisions/008-Inter-VM-communication-mechanisms.adoc new file mode 100644 index 0000000..1007037 --- /dev/null +++ b/Documentation/decisions/008-Inter-VM-communication-mechanisms.adoc @@ -0,0 +1,18 @@ +### Inter VM communication mechanisms + +### Status +Proposed + +### Context +Guest VM software needs to communicate with software in other guest VMs + +### Decision +Spectrum provides two mechanism +- TCP/IP with virtio-net +- Wayland with virtio-gpu (nevermind the semantics) for streamed IPC protocol to + send references to shared memory + + +### Consequences +- Examples required on how to write applications which communicate over + virtio-gpu -- 2.36.0

4 9