diff options
Diffstat (limited to 'nixos/doc/manual/development')
24 files changed, 2601 insertions, 0 deletions
diff --git a/nixos/doc/manual/development/activation-script.section.md b/nixos/doc/manual/development/activation-script.section.md new file mode 100644 index 00000000000..df683662404 --- /dev/null +++ b/nixos/doc/manual/development/activation-script.section.md @@ -0,0 +1,72 @@ +# Activation script {#sec-activation-script} + +The activation script is a bash script called to activate the new +configuration which resides in a NixOS system in `$out/activate`. Since its +contents depend on your system configuration, the contents may differ. +This chapter explains how the script works in general and some common NixOS +snippets. Please be aware that the script is executed on every boot and system +switch, so tasks that can be performed in other places should be performed +there (for example letting a directory of a service be created by systemd using +mechanisms like `StateDirectory`, `CacheDirectory`, ... or if that's not +possible using `preStart` of the service). + +Activation scripts are defined as snippets using +[](#opt-system.activationScripts). They can either be a simple multiline string +or an attribute set that can depend on other snippets. The builder for the +activation script will take these dependencies into account and order the +snippets accordingly. As a simple example: + +```nix +system.activationScripts.my-activation-script = { + deps = [ "etc" ]; + # supportsDryActivation = true; + text = '' + echo "Hallo i bims" + ''; +}; +``` + +This example creates an activation script snippet that is run after the `etc` +snippet. The special variable `supportsDryActivation` can be set so the snippet +is also run when `nixos-rebuild dry-activate` is run. To differentiate between +real and dry activation, the `$NIXOS_ACTION` environment variable can be +read which is set to `dry-activate` when a dry activation is done. + +An activation script can write to special files instructing +`switch-to-configuration` to restart/reload units. The script will take these +requests into account and will incorperate the unit configuration as described +above. This means that the activation script will "fake" a modified unit file +and `switch-to-configuration` will act accordingly. By doing so, configuration +like [systemd.services.\<name\>.restartIfChanged](#opt-systemd.services) is +respected. Since the activation script is run **after** services are already +stopped, [systemd.services.\<name\>.stopIfChanged](#opt-systemd.services) +cannot be taken into account anymore and the unit is always restarted instead +of being stopped and started afterwards. + +The files that can be written to are `/run/nixos/activation-restart-list` and +`/run/nixos/activation-reload-list` with their respective counterparts for +dry activation being `/run/nixos/dry-activation-restart-list` and +`/run/nixos/dry-activation-reload-list`. Those files can contain +newline-separated lists of unit names where duplicates are being ignored. These +files are not create automatically and activation scripts must take the +possiblility into account that they have to create them first. + +## NixOS snippets {#sec-activation-script-nixos-snippets} + +There are some snippets NixOS enables by default because disabling them would +most likely break you system. This section lists a few of them and what they +do: + +- `binsh` creates `/bin/sh` which points to the runtime shell +- `etc` sets up the contents of `/etc`, this includes systemd units and + excludes `/etc/passwd`, `/etc/group`, and `/etc/shadow` (which are managed by + the `users` snippet) +- `hostname` sets the system's hostname in the kernel (not in `/etc`) +- `modprobe` sets the path to the `modprobe` binary for module auto-loading +- `nix` prepares the nix store and adds a default initial channel +- `specialfs` is responsible for mounting filesystems like `/proc` and `sys` +- `users` creates and removes users and groups by managing `/etc/passwd`, + `/etc/group` and `/etc/shadow`. This also creates home directories +- `usrbinenv` creates `/usr/bin/env` +- `var` creates some directories in `/var` that are not service-specific +- `wrappers` creates setuid wrappers like `ping` and `sudo` diff --git a/nixos/doc/manual/development/assertions.section.md b/nixos/doc/manual/development/assertions.section.md new file mode 100644 index 00000000000..cc6d81e5699 --- /dev/null +++ b/nixos/doc/manual/development/assertions.section.md @@ -0,0 +1,40 @@ +# Warnings and Assertions {#sec-assertions} + +When configuration problems are detectable in a module, it is a good idea to write an assertion or warning. Doing so provides clear feedback to the user and prevents errors after the build. + +Although Nix has the `abort` and `builtins.trace` [functions](https://nixos.org/nix/manual/#ssec-builtins) to perform such tasks, they are not ideally suited for NixOS modules. Instead of these functions, you can declare your warnings and assertions using the NixOS module system. + +## Warnings {#sec-assertions-warnings} + +This is an example of using `warnings`. + +```nix +{ config, lib, ... }: +{ + config = lib.mkIf config.services.foo.enable { + warnings = + if config.services.foo.bar + then [ ''You have enabled the bar feature of the foo service. + This is known to cause some specific problems in certain situations. + '' ] + else []; + } +} +``` + +## Assertions {#sec-assertions-assetions} + +This example, extracted from the [`syslogd` module](https://github.com/NixOS/nixpkgs/blob/release-17.09/nixos/modules/services/logging/syslogd.nix) shows how to use `assertions`. Since there can only be one active syslog daemon at a time, an assertion is useful to prevent such a broken system from being built. + +```nix +{ config, lib, ... }: +{ + config = lib.mkIf config.services.syslogd.enable { + assertions = + [ { assertion = !config.services.rsyslogd.enable; + message = "rsyslogd conflicts with syslogd"; + } + ]; + } +} +``` diff --git a/nixos/doc/manual/development/building-nixos.chapter.md b/nixos/doc/manual/development/building-nixos.chapter.md new file mode 100644 index 00000000000..3310dee98f9 --- /dev/null +++ b/nixos/doc/manual/development/building-nixos.chapter.md @@ -0,0 +1,46 @@ +# Building a NixOS (Live) ISO {#sec-building-image} + +Default live installer configurations are available inside `nixos/modules/installer/cd-dvd`. +For building other system images, [nixos-generators] is a good place to start looking at. + +You have two options: + +- Use any of those default configurations as is +- Combine them with (any of) your host config(s) + +System images, such as the live installer ones, know how to enforce configuration settings +on wich they immediately depend in order to work correctly. + +However, if you are confident, you can opt to override those +enforced values with `mkForce`. + +[nixos-generators]: https://github.com/nix-community/nixos-generators + +## Practical Instructions {#sec-building-image-instructions} + +```ShellSession +$ git clone https://github.com/NixOS/nixpkgs.git +$ cd nixpkgs/nixos +$ nix-build -A config.system.build.isoImage -I nixos-config=modules/installer/cd-dvd/installation-cd-minimal.nix default.nix +``` + +To check the content of an ISO image, mount it like so: + +```ShellSession +# mount -o loop -t iso9660 ./result/iso/cd.iso /mnt/iso +``` + +## Technical Notes {#sec-building-image-tech-notes} + +The config value enforcement is implemented via `mkImageMediaOverride = mkOverride 60;` +and therefore primes over simple value assignments, but also yields to `mkForce`. + +This property allows image designers to implement in semantically correct ways those +configuration values upon which the correct functioning of the image depends. + +For example, the iso base image overrides those file systems which it needs at a minimum +for correct functioning, while the installer base image overrides the entire file system +layout because there can't be any other guarantees on a live medium than those given +by the live medium itself. The latter is especially true befor formatting the target +block device(s). On the other hand, the netboot iso only overrides its minimum dependencies +since netboot images are always made-to-target. diff --git a/nixos/doc/manual/development/building-parts.chapter.md b/nixos/doc/manual/development/building-parts.chapter.md new file mode 100644 index 00000000000..79ddaa37140 --- /dev/null +++ b/nixos/doc/manual/development/building-parts.chapter.md @@ -0,0 +1,74 @@ +# Building Specific Parts of NixOS {#sec-building-parts} + +With the command `nix-build`, you can build specific parts of your NixOS +configuration. This is done as follows: + +```ShellSession +$ cd /path/to/nixpkgs/nixos +$ nix-build -A config.option +``` + +where `option` is a NixOS option with type "derivation" (i.e. something +that can be built). Attributes of interest include: + +`system.build.toplevel` + +: The top-level option that builds the entire NixOS system. Everything + else in your configuration is indirectly pulled in by this option. + This is what `nixos-rebuild` builds and what `/run/current-system` + points to afterwards. + + A shortcut to build this is: + + ```ShellSession + $ nix-build -A system + ``` + +`system.build.manual.manualHTML` + +: The NixOS manual. + +`system.build.etc` + +: A tree of symlinks that form the static parts of `/etc`. + +`system.build.initialRamdisk` , `system.build.kernel` + +: The initial ramdisk and kernel of the system. This allows a quick + way to test whether the kernel and the initial ramdisk boot + correctly, by using QEMU's `-kernel` and `-initrd` options: + + ```ShellSession + $ nix-build -A config.system.build.initialRamdisk -o initrd + $ nix-build -A config.system.build.kernel -o kernel + $ qemu-system-x86_64 -kernel ./kernel/bzImage -initrd ./initrd/initrd -hda /dev/null + ``` + +`system.build.nixos-rebuild` , `system.build.nixos-install` , `system.build.nixos-generate-config` + +: These build the corresponding NixOS commands. + +`systemd.units.unit-name.unit` + +: This builds the unit with the specified name. Note that since unit + names contain dots (e.g. `httpd.service`), you need to put them + between quotes, like this: + + ```ShellSession + $ nix-build -A 'config.systemd.units."httpd.service".unit' + ``` + + You can also test individual units, without rebuilding the whole + system, by putting them in `/run/systemd/system`: + + ```ShellSession + $ cp $(nix-build -A 'config.systemd.units."httpd.service".unit')/httpd.service \ + /run/systemd/system/tmp-httpd.service + # systemctl daemon-reload + # systemctl start tmp-httpd.service + ``` + + Note that the unit must not have the same name as any unit in + `/etc/systemd/system` since those take precedence over + `/run/systemd/system`. That's why the unit is installed as + `tmp-httpd.service` here. diff --git a/nixos/doc/manual/development/development.xml b/nixos/doc/manual/development/development.xml new file mode 100644 index 00000000000..21286cdbd2b --- /dev/null +++ b/nixos/doc/manual/development/development.xml @@ -0,0 +1,20 @@ +<part xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + version="5.0" + xml:id="ch-development"> + <title>Development</title> + <partintro xml:id="ch-development-intro"> + <para> + This chapter describes how you can modify and extend NixOS. + </para> + </partintro> + <xi:include href="../from_md/development/sources.chapter.xml" /> + <xi:include href="../from_md/development/writing-modules.chapter.xml" /> + <xi:include href="../from_md/development/building-parts.chapter.xml" /> + <xi:include href="../from_md/development/what-happens-during-a-system-switch.chapter.xml" /> + <xi:include href="../from_md/development/writing-documentation.chapter.xml" /> + <xi:include href="../from_md/development/building-nixos.chapter.xml" /> + <xi:include href="../from_md/development/nixos-tests.chapter.xml" /> + <xi:include href="../from_md/development/testing-installer.chapter.xml" /> +</part> diff --git a/nixos/doc/manual/development/freeform-modules.section.md b/nixos/doc/manual/development/freeform-modules.section.md new file mode 100644 index 00000000000..10e876b96d5 --- /dev/null +++ b/nixos/doc/manual/development/freeform-modules.section.md @@ -0,0 +1,79 @@ +# Freeform modules {#sec-freeform-modules} + +Freeform modules allow you to define values for option paths that have +not been declared explicitly. This can be used to add attribute-specific +types to what would otherwise have to be `attrsOf` options in order to +accept all attribute names. + +This feature can be enabled by using the attribute `freeformType` to +define a freeform type. By doing this, all assignments without an +associated option will be merged using the freeform type and combined +into the resulting `config` set. Since this feature nullifies name +checking for entire option trees, it is only recommended for use in +submodules. + +::: {#ex-freeform-module .example} +::: {.title} +**Example: Freeform submodule** +::: +The following shows a submodule assigning a freeform type that allows +arbitrary attributes with `str` values below `settings`, but also +declares an option for the `settings.port` attribute to have it +type-checked and assign a default value. See +[Example: Declaring a type-checked `settings` attribute](#ex-settings-typed-attrs) +for a more complete example. + +```nix +{ lib, config, ... }: { + + options.settings = lib.mkOption { + type = lib.types.submodule { + + freeformType = with lib.types; attrsOf str; + + # We want this attribute to be checked for the correct type + options.port = lib.mkOption { + type = lib.types.port; + # Declaring the option also allows defining a default value + default = 8080; + }; + + }; + }; +} +``` + +And the following shows what such a module then allows + +```nix +{ + # Not a declared option, but the freeform type allows this + settings.logLevel = "debug"; + + # Not allowed because the the freeform type only allows strings + # settings.enable = true; + + # Allowed because there is a port option declared + settings.port = 80; + + # Not allowed because the port option doesn't allow strings + # settings.port = "443"; +} +``` +::: + +::: {.note} +Freeform attributes cannot depend on other attributes of the same set +without infinite recursion: + +```nix +{ + # This throws infinite recursion encountered + settings.logLevel = lib.mkIf (config.settings.port == 80) "debug"; +} +``` + +To prevent this, declare options for all attributes that need to depend +on others. For above example this means to declare `logLevel` to be an +option. +::: diff --git a/nixos/doc/manual/development/importing-modules.section.md b/nixos/doc/manual/development/importing-modules.section.md new file mode 100644 index 00000000000..65d78959b8e --- /dev/null +++ b/nixos/doc/manual/development/importing-modules.section.md @@ -0,0 +1,46 @@ +# Importing Modules {#sec-importing-modules} + +Sometimes NixOS modules need to be used in configuration but exist +outside of Nixpkgs. These modules can be imported: + +```nix +{ config, lib, pkgs, ... }: + +{ + imports = + [ # Use a locally-available module definition in + # ./example-module/default.nix + ./example-module + ]; + + services.exampleModule.enable = true; +} +``` + +The environment variable `NIXOS_EXTRA_MODULE_PATH` is an absolute path +to a NixOS module that is included alongside the Nixpkgs NixOS modules. +Like any NixOS module, this module can import additional modules: + +```nix +# ./module-list/default.nix +[ + ./example-module1 + ./example-module2 +] +``` + +```nix +# ./extra-module/default.nix +{ imports = import ./module-list.nix; } +``` + +```nix +# NIXOS_EXTRA_MODULE_PATH=/absolute/path/to/extra-module +{ config, lib, pkgs, ... }: + +{ + # No `imports` needed + + services.exampleModule1.enable = true; +} +``` diff --git a/nixos/doc/manual/development/linking-nixos-tests-to-packages.section.md b/nixos/doc/manual/development/linking-nixos-tests-to-packages.section.md new file mode 100644 index 00000000000..38a64027f7c --- /dev/null +++ b/nixos/doc/manual/development/linking-nixos-tests-to-packages.section.md @@ -0,0 +1,6 @@ +# Linking NixOS tests to packages {#sec-linking-nixos-tests-to-packages} + +You can link NixOS module tests to the packages that they exercised, +so that the tests can be run automatically during code review when the package gets changed. +This is +[described in the nixpkgs manual](https://nixos.org/manual/nixpkgs/stable/#ssec-nixos-tests-linking). diff --git a/nixos/doc/manual/development/meta-attributes.section.md b/nixos/doc/manual/development/meta-attributes.section.md new file mode 100644 index 00000000000..946c08efd0a --- /dev/null +++ b/nixos/doc/manual/development/meta-attributes.section.md @@ -0,0 +1,66 @@ +# Meta Attributes {#sec-meta-attributes} + +Like Nix packages, NixOS modules can declare meta-attributes to provide +extra information. Module meta attributes are defined in the `meta.nix` +special module. + +`meta` is a top level attribute like `options` and `config`. Available +meta-attributes are `maintainers`, `doc`, and `buildDocsInSandbox`. + +Each of the meta-attributes must be defined at most once per module +file. + +```nix +{ config, lib, pkgs, ... }: +{ + options = { + ... + }; + + config = { + ... + }; + + meta = { + maintainers = with lib.maintainers; [ ericsagnes ]; + doc = ./default.xml; + buildDocsInSandbox = true; + }; +} +``` + +- `maintainers` contains a list of the module maintainers. + +- `doc` points to a valid DocBook file containing the module + documentation. Its contents is automatically added to + [](#ch-configuration). Changes to a module documentation have to + be checked to not break building the NixOS manual: + + ```ShellSession + $ nix-build nixos/release.nix -A manual.x86_64-linux + ``` + +- `buildDocsInSandbox` indicates whether the option documentation for the + module can be built in a derivation sandbox. This option is currently only + honored for modules shipped by nixpkgs. User modules and modules taken from + `NIXOS_EXTRA_MODULE_PATH` are always built outside of the sandbox, as has + been the case in previous releases. + + Building NixOS option documentation in a sandbox allows caching of the built + documentation, which greatly decreases the amount of time needed to evaluate + a system configuration that has NixOS documentation enabled. The sandbox also + restricts which attributes may be referenced by documentation attributes + (such as option descriptions) to the `options` and `lib` module arguments and + the `pkgs.formats` attribute of the `pkgs` argument, `config` and the rest of + `pkgs` are disallowed and will cause doc build failures when used. This + restriction is necessary because we cannot reproduce the full nixpkgs + instantiation with configuration and overlays from a system configuration + inside the sandbox. The `options` argument only includes options of modules + that are also built inside the sandbox, referencing an option of a module + that isn't built in the sandbox is also forbidden. + + The default is `true` and should usually not be changed; set it to `false` + only if the module requires access to `pkgs` in its documentation (e.g. + because it loads information from a linked package to build an option type) + or if its documentation depends on other modules that also aren't sandboxed + (e.g. by using types defined in the other module). diff --git a/nixos/doc/manual/development/nixos-tests.chapter.md b/nixos/doc/manual/development/nixos-tests.chapter.md new file mode 100644 index 00000000000..2a4fdddeaa6 --- /dev/null +++ b/nixos/doc/manual/development/nixos-tests.chapter.md @@ -0,0 +1,13 @@ +# NixOS Tests {#sec-nixos-tests} + +When you add some feature to NixOS, you should write a test for it. +NixOS tests are kept in the directory `nixos/tests`, and are executed +(using Nix) by a testing framework that automatically starts one or more +virtual machines containing the NixOS system(s) required for the test. + +```{=docbook} +<xi:include href="writing-nixos-tests.section.xml" /> +<xi:include href="running-nixos-tests.section.xml" /> +<xi:include href="running-nixos-tests-interactively.section.xml" /> +<xi:include href="linking-nixos-tests-to-packages.section.xml" /> +``` diff --git a/nixos/doc/manual/development/option-declarations.section.md b/nixos/doc/manual/development/option-declarations.section.md new file mode 100644 index 00000000000..53ecb9b3a62 --- /dev/null +++ b/nixos/doc/manual/development/option-declarations.section.md @@ -0,0 +1,221 @@ +# Option Declarations {#sec-option-declarations} + +An option declaration specifies the name, type and description of a +NixOS configuration option. It is invalid to define an option that +hasn't been declared in any module. An option declaration generally +looks like this: + +```nix +options = { + name = mkOption { + type = type specification; + default = default value; + example = example value; + description = "Description for use in the NixOS manual."; + }; +}; +``` + +The attribute names within the `name` attribute path must be camel +cased in general but should, as an exception, match the [ package +attribute name](https://nixos.org/nixpkgs/manual/#sec-package-naming) +when referencing a Nixpkgs package. For example, the option +`services.nix-serve.bindAddress` references the `nix-serve` Nixpkgs +package. + +The function `mkOption` accepts the following arguments. + +`type` + +: The type of the option (see [](#sec-option-types)). This + argument is mandatory for nixpkgs modules. Setting this is highly + recommended for the sake of documentation and type checking. In case it is + not set, a fallback type with unspecified behavior is used. + +`default` + +: The default value used if no value is defined by any module. A + default is not required; but if a default is not given, then users + of the module will have to define the value of the option, otherwise + an error will be thrown. + +`defaultText` + +: A textual representation of the default value to be rendered verbatim in + the manual. Useful if the default value is a complex expression or depends + on other values or packages. + Use `lib.literalExpression` for a Nix expression, `lib.literalDocBook` for + a plain English description in DocBook format. + +`example` + +: An example value that will be shown in the NixOS manual. + You can use `lib.literalExpression` and `lib.literalDocBook` in the same way + as in `defaultText`. + +`description` + +: A textual description of the option, in DocBook format, that will be + included in the NixOS manual. + +## Utility functions for common option patterns {#sec-option-declarations-util} + +### `mkEnableOption` {#sec-option-declarations-util-mkEnableOption} + +Creates an Option attribute set for a boolean value option i.e an +option to be toggled on or off. + +This function takes a single string argument, the name of the thing to be toggled. + +The option's description is "Whether to enable \<name\>.". + +For example: + +::: {#ex-options-declarations-util-mkEnableOption-magic .example} +```nix +lib.mkEnableOption "magic" +# is like +lib.mkOption { + type = lib.types.bool; + default = false; + example = true; + description = "Whether to enable magic."; +} +``` + +### `mkPackageOption` {#sec-option-declarations-util-mkPackageOption} + +Usage: + +```nix +mkPackageOption pkgs "name" { default = [ "path" "in" "pkgs" ]; example = "literal example"; } +``` + +Creates an Option attribute set for an option that specifies the package a module should use for some purpose. + +**Note**: You shouldn’t necessarily make package options for all of your modules. You can always overwrite a specific package throughout nixpkgs by using [nixpkgs overlays](https://nixos.org/manual/nixpkgs/stable/#chap-overlays). + +The default package is specified as a list of strings representing its attribute path in nixpkgs. Because of this, you need to pass nixpkgs itself as the first argument. + +The second argument is the name of the option, used in the description "The \<name\> package to use.". You can also pass an example value, either a literal string or a package's attribute path. + +You can omit the default path if the name of the option is also attribute path in nixpkgs. + +::: {#ex-options-declarations-util-mkPackageOption .title} +Examples: + +::: {#ex-options-declarations-util-mkPackageOption-hello .example} +```nix +lib.mkPackageOption pkgs "hello" { } +# is like +lib.mkOption { + type = lib.types.package; + default = pkgs.hello; + defaultText = lib.literalExpression "pkgs.hello"; + description = "The hello package to use."; +} +``` + +::: {#ex-options-declarations-util-mkPackageOption-ghc .example} +```nix +lib.mkPackageOption pkgs "GHC" { + default = [ "ghc" ]; + example = "pkgs.haskell.package.ghc921.ghc.withPackages (hkgs: [ hkgs.primes ])"; +} +# is like +lib.mkOption { + type = lib.types.package; + default = pkgs.ghc; + defaultText = lib.literalExpression "pkgs.ghc"; + example = lib.literalExpression "pkgs.haskell.package.ghc921.ghc.withPackages (hkgs: [ hkgs.primes ])"; + description = "The GHC package to use."; +} +``` + +## Extensible Option Types {#sec-option-declarations-eot} + +Extensible option types is a feature that allow to extend certain types +declaration through multiple module files. This feature only work with a +restricted set of types, namely `enum` and `submodules` and any composed +forms of them. + +Extensible option types can be used for `enum` options that affects +multiple modules, or as an alternative to related `enable` options. + +As an example, we will take the case of display managers. There is a +central display manager module for generic display manager options and a +module file per display manager backend (sddm, gdm \...). + +There are two approaches we could take with this module structure: + +- Configuring the display managers independently by adding an enable + option to every display manager module backend. (NixOS) + +- Configuring the display managers in the central module by adding + an option to select which display manager backend to use. + +Both approaches have problems. + +Making backends independent can quickly become hard to manage. For +display managers, there can only be one enabled at a time, but the +type system cannot enforce this restriction as there is no relation +between each backend's `enable` option. As a result, this restriction +has to be done explicitly by adding assertions in each display manager +backend module. + +On the other hand, managing the display manager backends in the +central module will require changing the central module option every +time a new backend is added or removed. + +By using extensible option types, it is possible to create a placeholder +option in the central module +([Example: Extensible type placeholder in the service module](#ex-option-declaration-eot-service)), +and to extend it in each backend module +([Example: Extending `services.xserver.displayManager.enable` in the `gdm` module](#ex-option-declaration-eot-backend-gdm), +[Example: Extending `services.xserver.displayManager.enable` in the `sddm` module](#ex-option-declaration-eot-backend-sddm)). + +As a result, `displayManager.enable` option values can be added without +changing the main service module file and the type system automatically +enforces that there can only be a single display manager enabled. + +::: {#ex-option-declaration-eot-service .example} +::: {.title} +**Example: Extensible type placeholder in the service module** +::: +```nix +services.xserver.displayManager.enable = mkOption { + description = "Display manager to use"; + type = with types; nullOr (enum [ ]); +}; +``` +::: + +::: {#ex-option-declaration-eot-backend-gdm .example} +::: {.title} +**Example: Extending `services.xserver.displayManager.enable` in the `gdm` module** +::: +```nix +services.xserver.displayManager.enable = mkOption { + type = with types; nullOr (enum [ "gdm" ]); +}; +``` +::: + +::: {#ex-option-declaration-eot-backend-sddm .example} +::: {.title} +**Example: Extending `services.xserver.displayManager.enable` in the `sddm` module** +::: +```nix +services.xserver.displayManager.enable = mkOption { + type = with types; nullOr (enum [ "sddm" ]); +}; +``` +::: + +The placeholder declaration is a standard `mkOption` declaration, but it +is important that extensible option declarations only use the `type` +argument. + +Extensible option types work with any of the composed variants of `enum` +such as `with types; nullOr (enum [ "foo" "bar" ])` or `with types; +listOf (enum [ "foo" "bar" ])`. diff --git a/nixos/doc/manual/development/option-def.section.md b/nixos/doc/manual/development/option-def.section.md new file mode 100644 index 00000000000..91b24cd4a3a --- /dev/null +++ b/nixos/doc/manual/development/option-def.section.md @@ -0,0 +1,91 @@ +# Option Definitions {#sec-option-definitions} + +Option definitions are generally straight-forward bindings of values to +option names, like + +```nix +config = { + services.httpd.enable = true; +}; +``` + +However, sometimes you need to wrap an option definition or set of +option definitions in a *property* to achieve certain effects: + +## Delaying Conditionals {#sec-option-definitions-delaying-conditionals .unnumbered} + +If a set of option definitions is conditional on the value of another +option, you may need to use `mkIf`. Consider, for instance: + +```nix +config = if config.services.httpd.enable then { + environment.systemPackages = [ ... ]; + ... +} else {}; +``` + +This definition will cause Nix to fail with an "infinite recursion" +error. Why? Because the value of `config.services.httpd.enable` depends +on the value being constructed here. After all, you could also write the +clearly circular and contradictory: + +```nix +config = if config.services.httpd.enable then { + services.httpd.enable = false; +} else { + services.httpd.enable = true; +}; +``` + +The solution is to write: + +```nix +config = mkIf config.services.httpd.enable { + environment.systemPackages = [ ... ]; + ... +}; +``` + +The special function `mkIf` causes the evaluation of the conditional to +be "pushed down" into the individual definitions, as if you had written: + +```nix +config = { + environment.systemPackages = if config.services.httpd.enable then [ ... ] else []; + ... +}; +``` + +## Setting Priorities {#sec-option-definitions-setting-priorities .unnumbered} + +A module can override the definitions of an option in other modules by +setting a *priority*. All option definitions that do not have the lowest +priority value are discarded. By default, option definitions have +priority 1000. You can specify an explicit priority by using +`mkOverride`, e.g. + +```nix +services.openssh.enable = mkOverride 10 false; +``` + +This definition causes all other definitions with priorities above 10 to +be discarded. The function `mkForce` is equal to `mkOverride 50`. + +## Merging Configurations {#sec-option-definitions-merging .unnumbered} + +In conjunction with `mkIf`, it is sometimes useful for a module to +return multiple sets of option definitions, to be merged together as if +they were declared in separate modules. This can be done using +`mkMerge`: + +```nix +config = mkMerge + [ # Unconditional stuff. + { environment.systemPackages = [ ... ]; + } + # Conditional stuff. + (mkIf config.services.bla.enable { + environment.systemPackages = [ ... ]; + }) + ]; +``` diff --git a/nixos/doc/manual/development/option-types.section.md b/nixos/doc/manual/development/option-types.section.md new file mode 100644 index 00000000000..00f1d85bdb6 --- /dev/null +++ b/nixos/doc/manual/development/option-types.section.md @@ -0,0 +1,583 @@ +# Options Types {#sec-option-types} + +Option types are a way to put constraints on the values a module option +can take. Types are also responsible of how values are merged in case of +multiple value definitions. + +## Basic Types {#sec-option-types-basic} + +Basic types are the simplest available types in the module system. Basic +types include multiple string types that mainly differ in how definition +merging is handled. + +`types.bool` + +: A boolean, its values can be `true` or `false`. + +`types.path` + +: A filesystem path is anything that starts with a slash when + coerced to a string. Even if derivations can be considered as + paths, the more specific `types.package` should be preferred. + +`types.package` + +: A top-level store path. This can be an attribute set pointing + to a store path, like a derivation or a flake input. + +`types.anything` + +: A type that accepts any value and recursively merges attribute sets + together. This type is recommended when the option type is unknown. + + ::: {#ex-types-anything .example} + ::: {.title} + **Example: `types.anything` Example** + ::: + Two definitions of this type like + + ```nix + { + str = lib.mkDefault "foo"; + pkg.hello = pkgs.hello; + fun.fun = x: x + 1; + } + ``` + + ```nix + { + str = lib.mkIf true "bar"; + pkg.gcc = pkgs.gcc; + fun.fun = lib.mkForce (x: x + 2); + } + ``` + + will get merged to + + ```nix + { + str = "bar"; + pkg.gcc = pkgs.gcc; + pkg.hello = pkgs.hello; + fun.fun = x: x + 2; + } + ``` + ::: + +`types.raw` + +: A type which doesn't do any checking, merging or nested evaluation. It + accepts a single arbitrary value that is not recursed into, making it + useful for values coming from outside the module system, such as package + sets or arbitrary data. Options of this type are still evaluated according + to priorities and conditionals, so `mkForce`, `mkIf` and co. still work on + the option value itself, but not for any value nested within it. This type + should only be used when checking, merging and nested evaluation are not + desirable. + +`types.optionType` + +: The type of an option's type. Its merging operation ensures that nested + options have the correct file location annotated, and that if possible, + multiple option definitions are correctly merged together. The main use + case is as the type of the `_module.freeformType` option. + +`types.attrs` + +: A free-form attribute set. + + ::: {.warning} + This type will be deprecated in the future because it doesn\'t + recurse into attribute sets, silently drops earlier attribute + definitions, and doesn\'t discharge `lib.mkDefault`, `lib.mkIf` + and co. For allowing arbitrary attribute sets, prefer + `types.attrsOf types.anything` instead which doesn\'t have these + problems. + ::: + +Integer-related types: + +`types.int` + +: A signed integer. + +`types.ints.{s8, s16, s32}` + +: Signed integers with a fixed length (8, 16 or 32 bits). They go from + −2^n/2 to + 2^n/2−1 respectively (e.g. `−128` to + `127` for 8 bits). + +`types.ints.unsigned` + +: An unsigned integer (that is >= 0). + +`types.ints.{u8, u16, u32}` + +: Unsigned integers with a fixed length (8, 16 or 32 bits). They go + from 0 to 2^n−1 respectively (e.g. `0` + to `255` for 8 bits). + +`types.ints.positive` + +: A positive integer (that is > 0). + +`types.port` + +: A port number. This type is an alias to + `types.ints.u16`. + +String-related types: + +`types.str` + +: A string. Multiple definitions cannot be merged. + +`types.lines` + +: A string. Multiple definitions are concatenated with a new line + `"\n"`. + +`types.commas` + +: A string. Multiple definitions are concatenated with a comma `","`. + +`types.envVar` + +: A string. Multiple definitions are concatenated with a collon `":"`. + +`types.strMatching` + +: A string matching a specific regular expression. Multiple + definitions cannot be merged. The regular expression is processed + using `builtins.match`. + +## Value Types {#sec-option-types-value} + +Value types are types that take a value parameter. + +`types.enum` *`l`* + +: One element of the list *`l`*, e.g. `types.enum [ "left" "right" ]`. + Multiple definitions cannot be merged. + +`types.separatedString` *`sep`* + +: A string with a custom separator *`sep`*, e.g. + `types.separatedString "|"`. + +`types.ints.between` *`lowest highest`* + +: An integer between *`lowest`* and *`highest`* (both inclusive). Useful + for creating types like `types.port`. + +`types.submodule` *`o`* + +: A set of sub options *`o`*. *`o`* can be an attribute set, a function + returning an attribute set, or a path to a file containing such a + value. Submodules are used in composed types to create modular + options. This is equivalent to + `types.submoduleWith { modules = toList o; shorthandOnlyDefinesConfig = true; }`. + Submodules are detailed in [Submodule](#section-option-types-submodule). + +`types.submoduleWith` { *`modules`*, *`specialArgs`* ? {}, *`shorthandOnlyDefinesConfig`* ? false } + +: Like `types.submodule`, but more flexible and with better defaults. + It has parameters + + - *`modules`* A list of modules to use by default for this + submodule type. This gets combined with all option definitions + to build the final list of modules that will be included. + + ::: {.note} + Only options defined with this argument are included in rendered + documentation. + ::: + + - *`specialArgs`* An attribute set of extra arguments to be passed + to the module functions. The option `_module.args` should be + used instead for most arguments since it allows overriding. + *`specialArgs`* should only be used for arguments that can\'t go + through the module fixed-point, because of infinite recursion or + other problems. An example is overriding the `lib` argument, + because `lib` itself is used to define `_module.args`, which + makes using `_module.args` to define it impossible. + + - *`shorthandOnlyDefinesConfig`* Whether definitions of this type + should default to the `config` section of a module (see + [Example: Structure of NixOS Modules](#ex-module-syntax)) + if it is an attribute set. Enabling this only has a benefit + when the submodule defines an option named `config` or `options`. + In such a case it would allow the option to be set with + `the-submodule.config = "value"` instead of requiring + `the-submodule.config.config = "value"`. This is because + only when modules *don\'t* set the `config` or `options` + keys, all keys are interpreted as option definitions in the + `config` section. Enabling this option implicitly puts all + attributes in the `config` section. + + With this option enabled, defining a non-`config` section + requires using a function: + `the-submodule = { ... }: { options = { ... }; }`. + +## Composed Types {#sec-option-types-composed} + +Composed types are types that take a type as parameter. `listOf + int` and `either int str` are examples of composed types. + +`types.listOf` *`t`* + +: A list of *`t`* type, e.g. `types.listOf + int`. Multiple definitions are merged with list concatenation. + +`types.attrsOf` *`t`* + +: An attribute set of where all the values are of *`t`* type. Multiple + definitions result in the joined attribute set. + + ::: {.note} + This type is *strict* in its values, which in turn means attributes + cannot depend on other attributes. See ` + types.lazyAttrsOf` for a lazy version. + ::: + +`types.lazyAttrsOf` *`t`* + +: An attribute set of where all the values are of *`t`* type. Multiple + definitions result in the joined attribute set. This is the lazy + version of `types.attrsOf + `, allowing attributes to depend on each other. + + ::: {.warning} + This version does not fully support conditional definitions! With an + option `foo` of this type and a definition + `foo.attr = lib.mkIf false 10`, evaluating `foo ? attr` will return + `true` even though it should be false. Accessing the value will then + throw an error. For types *`t`* that have an `emptyValue` defined, + that value will be returned instead of throwing an error. So if the + type of `foo.attr` was `lazyAttrsOf (nullOr int)`, `null` would be + returned instead for the same `mkIf false` definition. + ::: + +`types.nullOr` *`t`* + +: `null` or type *`t`*. Multiple definitions are merged according to + type *`t`*. + +`types.uniq` *`t`* + +: Ensures that type *`t`* cannot be merged. It is used to ensure option + definitions are declared only once. + +`types.unique` `{ message = m }` *`t`* + +: Ensures that type *`t`* cannot be merged. Prints the message *`m`*, after + the line `The option <option path> is defined multiple times.` and before + a list of definition locations. + +`types.either` *`t1 t2`* + +: Type *`t1`* or type *`t2`*, e.g. `with types; either int str`. + Multiple definitions cannot be merged. + +`types.oneOf` \[ *`t1 t2`* \... \] + +: Type *`t1`* or type *`t2`* and so forth, e.g. + `with types; oneOf [ int str bool ]`. Multiple definitions cannot be + merged. + +`types.coercedTo` *`from f to`* + +: Type *`to`* or type *`from`* which will be coerced to type *`to`* using + function *`f`* which takes an argument of type *`from`* and return a + value of type *`to`*. Can be used to preserve backwards compatibility + of an option if its type was changed. + +## Submodule {#section-option-types-submodule} + +`submodule` is a very powerful type that defines a set of sub-options +that are handled like a separate module. + +It takes a parameter *`o`*, that should be a set, or a function returning +a set with an `options` key defining the sub-options. Submodule option +definitions are type-checked accordingly to the `options` declarations. +Of course, you can nest submodule option definitons for even higher +modularity. + +The option set can be defined directly +([Example: Directly defined submodule](#ex-submodule-direct)) or as reference +([Example: Submodule defined as a reference](#ex-submodule-reference)). + +::: {#ex-submodule-direct .example} +::: {.title} +**Example: Directly defined submodule** +::: +```nix +options.mod = mkOption { + description = "submodule example"; + type = with types; submodule { + options = { + foo = mkOption { + type = int; + }; + bar = mkOption { + type = str; + }; + }; + }; +}; +``` +::: + +::: {#ex-submodule-reference .example} +::: {.title} +**Example: Submodule defined as a reference** +::: +```nix +let + modOptions = { + options = { + foo = mkOption { + type = int; + }; + bar = mkOption { + type = int; + }; + }; + }; +in +options.mod = mkOption { + description = "submodule example"; + type = with types; submodule modOptions; +}; +``` +::: + +The `submodule` type is especially interesting when used with composed +types like `attrsOf` or `listOf`. When composed with `listOf` +([Example: Declaration of a list of submodules](#ex-submodule-listof-declaration)), `submodule` allows +multiple definitions of the submodule option set +([Example: Definition of a list of submodules](#ex-submodule-listof-definition)). + +::: {#ex-submodule-listof-declaration .example} +::: {.title} +**Example: Declaration of a list of submodules** +::: +```nix +options.mod = mkOption { + description = "submodule example"; + type = with types; listOf (submodule { + options = { + foo = mkOption { + type = int; + }; + bar = mkOption { + type = str; + }; + }; + }); +}; +``` +::: + +::: {#ex-submodule-listof-definition .example} +::: {.title} +**Example: Definition of a list of submodules** +::: +```nix +config.mod = [ + { foo = 1; bar = "one"; } + { foo = 2; bar = "two"; } +]; +``` +::: + +When composed with `attrsOf` +([Example: Declaration of attribute sets of submodules](#ex-submodule-attrsof-declaration)), `submodule` allows +multiple named definitions of the submodule option set +([Example: Definition of attribute sets of submodules](#ex-submodule-attrsof-definition)). + +::: {#ex-submodule-attrsof-declaration .example} +::: {.title} +**Example: Declaration of attribute sets of submodules** +::: +```nix +options.mod = mkOption { + description = "submodule example"; + type = with types; attrsOf (submodule { + options = { + foo = mkOption { + type = int; + }; + bar = mkOption { + type = str; + }; + }; + }); +}; +``` +::: + +::: {#ex-submodule-attrsof-definition .example} +::: {.title} +**Example: Definition of attribute sets of submodules** +::: +```nix +config.mod.one = { foo = 1; bar = "one"; }; +config.mod.two = { foo = 2; bar = "two"; }; +``` +::: + +## Extending types {#sec-option-types-extending} + +Types are mainly characterized by their `check` and `merge` functions. + +`check` + +: The function to type check the value. Takes a value as parameter and + return a boolean. It is possible to extend a type check with the + `addCheck` function ([Example: Adding a type check](#ex-extending-type-check-1)), + or to fully override the check function + ([Example: Overriding a type check](#ex-extending-type-check-2)). + + ::: {#ex-extending-type-check-1 .example} + ::: {.title} + **Example: Adding a type check** + ::: + ```nix + byte = mkOption { + description = "An integer between 0 and 255."; + type = types.addCheck types.int (x: x >= 0 && x <= 255); + }; + ``` + ::: + + ::: {#ex-extending-type-check-2 .example} + ::: {.title} + **Example: Overriding a type check** + ::: + ```nix + nixThings = mkOption { + description = "words that start with 'nix'"; + type = types.str // { + check = (x: lib.hasPrefix "nix" x) + }; + }; + ``` + ::: + +`merge` + +: Function to merge the options values when multiple values are set. + The function takes two parameters, `loc` the option path as a list + of strings, and `defs` the list of defined values as a list. It is + possible to override a type merge function for custom needs. + +## Custom Types {#sec-option-types-custom} + +Custom types can be created with the `mkOptionType` function. As type +creation includes some more complex topics such as submodule handling, +it is recommended to get familiar with `types.nix` code before creating +a new type. + +The only required parameter is `name`. + +`name` + +: A string representation of the type function name. + +`definition` + +: Description of the type used in documentation. Give information of + the type and any of its arguments. + +`check` + +: A function to type check the definition value. Takes the definition + value as a parameter and returns a boolean indicating the type check + result, `true` for success and `false` for failure. + +`merge` + +: A function to merge multiple definitions values. Takes two + parameters: + + *`loc`* + + : The option path as a list of strings, e.g. `["boot" "loader + "grub" "enable"]`. + + *`defs`* + + : The list of sets of defined `value` and `file` where the value + was defined, e.g. `[ { + file = "/foo.nix"; value = 1; } { file = "/bar.nix"; value = 2 } + ]`. The `merge` function should return the merged value + or throw an error in case the values are impossible or not meant + to be merged. + +`getSubOptions` + +: For composed types that can take a submodule as type parameter, this + function generate sub-options documentation. It takes the current + option prefix as a list and return the set of sub-options. Usually + defined in a recursive manner by adding a term to the prefix, e.g. + `prefix: + elemType.getSubOptions (prefix ++ + ["prefix"])` where *`"prefix"`* is the newly added prefix. + +`getSubModules` + +: For composed types that can take a submodule as type parameter, this + function should return the type parameters submodules. If the type + parameter is called `elemType`, the function should just recursively + look into submodules by returning `elemType.getSubModules;`. + +`substSubModules` + +: For composed types that can take a submodule as type parameter, this + function can be used to substitute the parameter of a submodule + type. It takes a module as parameter and return the type with the + submodule options substituted. It is usually defined as a type + function call with a recursive call to `substSubModules`, e.g for a + type `composedType` that take an `elemtype` type parameter, this + function should be defined as `m: + composedType (elemType.substSubModules m)`. + +`typeMerge` + +: A function to merge multiple type declarations. Takes the type to + merge `functor` as parameter. A `null` return value means that type + cannot be merged. + + *`f`* + + : The type to merge `functor`. + + Note: There is a generic `defaultTypeMerge` that work with most of + value and composed types. + +`functor` + +: An attribute set representing the type. It is used for type + operations and has the following keys: + + `type` + + : The type function. + + `wrapped` + + : Holds the type parameter for composed types. + + `payload` + + : Holds the value parameter for value types. The types that have a + `payload` are the `enum`, `separatedString` and `submodule` + types. + + `binOp` + + : A binary operation that can merge the payloads of two same + types. Defined as a function that take two payloads as + parameters and return the payloads merged. diff --git a/nixos/doc/manual/development/replace-modules.section.md b/nixos/doc/manual/development/replace-modules.section.md new file mode 100644 index 00000000000..0700a82004c --- /dev/null +++ b/nixos/doc/manual/development/replace-modules.section.md @@ -0,0 +1,64 @@ +# Replace Modules {#sec-replace-modules} + +Modules that are imported can also be disabled. The option declarations, +config implementation and the imports of a disabled module will be +ignored, allowing another to take it\'s place. This can be used to +import a set of modules from another channel while keeping the rest of +the system on a stable release. + +`disabledModules` is a top level attribute like `imports`, `options` and +`config`. It contains a list of modules that will be disabled. This can +either be the full path to the module or a string with the filename +relative to the modules path (eg. \<nixpkgs/nixos/modules> for nixos). + +This example will replace the existing postgresql module with the +version defined in the nixos-unstable channel while keeping the rest of +the modules and packages from the original nixos channel. This only +overrides the module definition, this won\'t use postgresql from +nixos-unstable unless explicitly configured to do so. + +```nix +{ config, lib, pkgs, ... }: + +{ + disabledModules = [ "services/databases/postgresql.nix" ]; + + imports = + [ # Use postgresql service from nixos-unstable channel. + # sudo nix-channel --add https://nixos.org/channels/nixos-unstable nixos-unstable + <nixos-unstable/nixos/modules/services/databases/postgresql.nix> + ]; + + services.postgresql.enable = true; +} +``` + +This example shows how to define a custom module as a replacement for an +existing module. Importing this module will disable the original module +without having to know it\'s implementation details. + +```nix +{ config, lib, pkgs, ... }: + +with lib; + +let + cfg = config.programs.man; +in + +{ + disabledModules = [ "services/programs/man.nix" ]; + + options = { + programs.man.enable = mkOption { + type = types.bool; + default = true; + description = "Whether to enable manual pages."; + }; + }; + + config = mkIf cfg.enabled { + warnings = [ "disabled manpages for production deployments." ]; + }; +} +``` diff --git a/nixos/doc/manual/development/running-nixos-tests-interactively.section.md b/nixos/doc/manual/development/running-nixos-tests-interactively.section.md new file mode 100644 index 00000000000..a1431859ff5 --- /dev/null +++ b/nixos/doc/manual/development/running-nixos-tests-interactively.section.md @@ -0,0 +1,35 @@ +# Running Tests interactively {#sec-running-nixos-tests-interactively} + +The test itself can be run interactively. This is particularly useful +when developing or debugging a test: + +```ShellSession +$ nix-build . -A nixosTests.login.driverInteractive +$ ./result/bin/nixos-test-driver +[...] +>>> +``` + +You can then take any Python statement, e.g. + +```py +>>> start_all() +>>> test_script() +>>> machine.succeed("touch /tmp/foo") +>>> print(machine.succeed("pwd")) # Show stdout of command +``` + +The function `test_script` executes the entire test script and drops you +back into the test driver command line upon its completion. This allows +you to inspect the state of the VMs after the test (e.g. to debug the +test script). + +You can re-use the VM states coming from a previous run by setting the +`--keep-vm-state` flag. + +```ShellSession +$ ./result/bin/nixos-test-driver --keep-vm-state +``` + +The machine state is stored in the `$TMPDIR/vm-state-machinename` +directory. diff --git a/nixos/doc/manual/development/running-nixos-tests.section.md b/nixos/doc/manual/development/running-nixos-tests.section.md new file mode 100644 index 00000000000..1bec023b613 --- /dev/null +++ b/nixos/doc/manual/development/running-nixos-tests.section.md @@ -0,0 +1,31 @@ +# Running Tests {#sec-running-nixos-tests} + +You can run tests using `nix-build`. For example, to run the test +[`login.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix), +you just do: + +```ShellSession +$ nix-build '<nixpkgs/nixos/tests/login.nix>' +``` + +or, if you don't want to rely on `NIX_PATH`: + +```ShellSession +$ cd /my/nixpkgs/nixos/tests +$ nix-build login.nix +… +running the VM test script +machine: QEMU running (pid 8841) +… +6 out of 6 tests succeeded +``` + +After building/downloading all required dependencies, this will perform +a build that starts a QEMU/KVM virtual machine containing a NixOS +system. The virtual machine mounts the Nix store of the host; this makes +VM creation very fast, as no disk image needs to be created. Afterwards, +you can view a log of the test: + +```ShellSession +$ nix-store --read-log result +``` diff --git a/nixos/doc/manual/development/settings-options.section.md b/nixos/doc/manual/development/settings-options.section.md new file mode 100644 index 00000000000..f9bb6ff9cc4 --- /dev/null +++ b/nixos/doc/manual/development/settings-options.section.md @@ -0,0 +1,237 @@ +# Options for Program Settings {#sec-settings-options} + +Many programs have configuration files where program-specific settings +can be declared. File formats can be separated into two categories: + +- Nix-representable ones: These can trivially be mapped to a subset of + Nix syntax. E.g. JSON is an example, since its values like + `{"foo":{"bar":10}}` can be mapped directly to Nix: + `{ foo = { bar = 10; }; }`. Other examples are INI, YAML and TOML. + The following section explains the convention for these settings. + +- Non-nix-representable ones: These can\'t be trivially mapped to a + subset of Nix syntax. Most generic programming languages are in this + group, e.g. bash, since the statement `if true; then echo hi; fi` + doesn\'t have a trivial representation in Nix. + + Currently there are no fixed conventions for these, but it is common + to have a `configFile` option for setting the configuration file + path directly. The default value of `configFile` can be an + auto-generated file, with convenient options for controlling the + contents. For example an option of type `attrsOf str` can be used + for representing environment variables which generates a section + like `export FOO="foo"`. Often it can also be useful to also include + an `extraConfig` option of type `lines` to allow arbitrary text + after the autogenerated part of the file. + +## Nix-representable Formats (JSON, YAML, TOML, INI, \...) {#sec-settings-nix-representable} + +By convention, formats like this are handled with a generic `settings` +option, representing the full program configuration as a Nix value. The +type of this option should represent the format. The most common formats +have a predefined type and string generator already declared under +`pkgs.formats`: + +`pkgs.formats.json` { } + +: A function taking an empty attribute set (for future extensibility) + and returning a set with JSON-specific attributes `type` and + `generate` as specified [below](#pkgs-formats-result). + +`pkgs.formats.yaml` { } + +: A function taking an empty attribute set (for future extensibility) + and returning a set with YAML-specific attributes `type` and + `generate` as specified [below](#pkgs-formats-result). + +`pkgs.formats.ini` { *`listsAsDuplicateKeys`* ? false, *`listToValue`* ? null, \... } + +: A function taking an attribute set with values + + `listsAsDuplicateKeys` + + : A boolean for controlling whether list values can be used to + represent duplicate INI keys + + `listToValue` + + : A function for turning a list of values into a single value. + + It returns a set with INI-specific attributes `type` and `generate` + as specified [below](#pkgs-formats-result). + +`pkgs.formats.toml` { } + +: A function taking an empty attribute set (for future extensibility) + and returning a set with TOML-specific attributes `type` and + `generate` as specified [below](#pkgs-formats-result). + +`pkgs.formats.elixirConf { elixir ? pkgs.elixir }` + +: A function taking an attribute set with values + + `elixir` + + : The Elixir package which will be used to format the generated output + + It returns a set with Elixir-Config-specific attributes `type`, `lib`, and + `generate` as specified [below](#pkgs-formats-result). + + The `lib` attribute contains functions to be used in settings, for + generating special Elixir values: + + `mkRaw elixirCode` + + : Outputs the given string as raw Elixir code + + `mkGetEnv { envVariable, fallback ? null }` + + : Makes the configuration fetch an environment variable at runtime + + `mkAtom atom` + + : Outputs the given string as an Elixir atom, instead of the default + Elixir binary string. Note: lowercase atoms still needs to be prefixed + with `:` + + `mkTuple array` + + : Outputs the given array as an Elixir tuple, instead of the default + Elixir list + + `mkMap attrset` + + : Outputs the given attribute set as an Elixir map, instead of the + default Elixir keyword list + + +::: {#pkgs-formats-result} +These functions all return an attribute set with these values: +::: + +`type` + +: A module system type representing a value of the format + +`lib` + +: Utility functions for convenience, or special interactions with the format. + This attribute is optional. It may contain inside a `types` attribute + containing types specific to this format. + +`generate` *`filename jsonValue`* + +: A function that can render a value of the format to a file. Returns + a file path. + + ::: {.note} + This function puts the value contents in the Nix store. So this + should be avoided for secrets. + ::: + +::: {#ex-settings-nix-representable .example} +::: {.title} +**Example: Module with conventional `settings` option** +::: +The following shows a module for an example program that uses a JSON +configuration file. It demonstrates how above values can be used, along +with some other related best practices. See the comments for +explanations. + +```nix +{ options, config, lib, pkgs, ... }: +let + cfg = config.services.foo; + # Define the settings format used for this program + settingsFormat = pkgs.formats.json {}; +in { + + options.services.foo = { + enable = lib.mkEnableOption "foo service"; + + settings = lib.mkOption { + # Setting this type allows for correct merging behavior + type = settingsFormat.type; + default = {}; + description = '' + Configuration for foo, see + <link xlink:href="https://example.com/docs/foo"/> + for supported settings. + ''; + }; + }; + + config = lib.mkIf cfg.enable { + # We can assign some default settings here to make the service work by just + # enabling it. We use `mkDefault` for values that can be changed without + # problems + services.foo.settings = { + # Fails at runtime without any value set + log_level = lib.mkDefault "WARN"; + + # We assume systemd's `StateDirectory` is used, so we require this value, + # therefore no mkDefault + data_path = "/var/lib/foo"; + + # Since we use this to create a user we need to know the default value at + # eval time + user = lib.mkDefault "foo"; + }; + + environment.etc."foo.json".source = + # The formats generator function takes a filename and the Nix value + # representing the format value and produces a filepath with that value + # rendered in the format + settingsFormat.generate "foo-config.json" cfg.settings; + + # We know that the `user` attribute exists because we set a default value + # for it above, allowing us to use it without worries here + users.users.${cfg.settings.user} = { isSystemUser = true; }; + + # ... + }; +} +``` +::: + +### Option declarations for attributes {#sec-settings-attrs-options} + +Some `settings` attributes may deserve some extra care. They may need a +different type, default or merging behavior, or they are essential +options that should show their documentation in the manual. This can be +done using [](#sec-freeform-modules). + +We extend above example using freeform modules to declare an option for +the port, which will enforce it to be a valid integer and make it show +up in the manual. + +::: {#ex-settings-typed-attrs .example} +::: {.title} +**Example: Declaring a type-checked `settings` attribute** +::: +```nix +settings = lib.mkOption { + type = lib.types.submodule { + + freeformType = settingsFormat.type; + + # Declare an option for the port such that the type is checked and this option + # is shown in the manual. + options.port = lib.mkOption { + type = lib.types.port; + default = 8080; + description = '' + Which port this service should listen on. + ''; + }; + + }; + default = {}; + description = '' + Configuration for Foo, see + <link xlink:href="https://example.com/docs/foo"/> + for supported values. + ''; +}; +``` +::: diff --git a/nixos/doc/manual/development/sources.chapter.md b/nixos/doc/manual/development/sources.chapter.md new file mode 100644 index 00000000000..88173f7135b --- /dev/null +++ b/nixos/doc/manual/development/sources.chapter.md @@ -0,0 +1,77 @@ +# Getting the Sources {#sec-getting-sources} + +By default, NixOS's `nixos-rebuild` command uses the NixOS and Nixpkgs +sources provided by the `nixos` channel (kept in +`/nix/var/nix/profiles/per-user/root/channels/nixos`). To modify NixOS, +however, you should check out the latest sources from Git. This is as +follows: + +```ShellSession +$ git clone https://github.com/NixOS/nixpkgs +$ cd nixpkgs +$ git remote update origin +``` + +This will check out the latest Nixpkgs sources to `./nixpkgs` the NixOS +sources to `./nixpkgs/nixos`. (The NixOS source tree lives in a +subdirectory of the Nixpkgs repository.) The `nixpkgs` repository has +branches that correspond to each Nixpkgs/NixOS channel (see +[](#sec-upgrading) for more information about channels). Thus, the +Git branch `origin/nixos-17.03` will contain the latest built and tested +version available in the `nixos-17.03` channel. + +It's often inconvenient to develop directly on the master branch, since +if somebody has just committed (say) a change to GCC, then the binary +cache may not have caught up yet and you'll have to rebuild everything +from source. So you may want to create a local branch based on your +current NixOS version: + +```ShellSession +$ nixos-version +17.09pre104379.6e0b727 (Hummingbird) + +$ git checkout -b local 6e0b727 +``` + +Or, to base your local branch on the latest version available in a NixOS +channel: + +```ShellSession +$ git remote update origin +$ git checkout -b local origin/nixos-17.03 +``` + +(Replace `nixos-17.03` with the name of the channel you want to use.) +You can use `git merge` or `git + rebase` to keep your local branch in sync with the channel, e.g. + +```ShellSession +$ git remote update origin +$ git merge origin/nixos-17.03 +``` + +You can use `git cherry-pick` to copy commits from your local branch to +the upstream branch. + +If you want to rebuild your system using your (modified) sources, you +need to tell `nixos-rebuild` about them using the `-I` flag: + +```ShellSession +# nixos-rebuild switch -I nixpkgs=/my/sources/nixpkgs +``` + +If you want `nix-env` to use the expressions in `/my/sources`, use +`nix-env -f + /my/sources/nixpkgs`, or change the default by adding a symlink in +`~/.nix-defexpr`: + +```ShellSession +$ ln -s /my/sources/nixpkgs ~/.nix-defexpr/nixpkgs +``` + +You may want to delete the symlink `~/.nix-defexpr/channels_root` to +prevent root's NixOS channel from clashing with your own tree (this may +break the command-not-found utility though). If you want to go back to +the default state, you may just remove the `~/.nix-defexpr` directory +completely, log out and log in again and it should have been recreated +with a link to the root channels. diff --git a/nixos/doc/manual/development/testing-installer.chapter.md b/nixos/doc/manual/development/testing-installer.chapter.md new file mode 100644 index 00000000000..2eaa0161492 --- /dev/null +++ b/nixos/doc/manual/development/testing-installer.chapter.md @@ -0,0 +1,18 @@ +# Testing the Installer {#ch-testing-installer} + +Building, burning, and booting from an installation CD is rather +tedious, so here is a quick way to see if the installer works properly: + +```ShellSession +# mount -t tmpfs none /mnt +# nixos-generate-config --root /mnt +$ nix-build '<nixpkgs/nixos>' -A config.system.build.nixos-install +# ./result/bin/nixos-install +``` + +To start a login shell in the new NixOS installation in `/mnt`: + +```ShellSession +$ nix-build '<nixpkgs/nixos>' -A config.system.build.nixos-enter +# ./result/bin/nixos-enter +``` diff --git a/nixos/doc/manual/development/unit-handling.section.md b/nixos/doc/manual/development/unit-handling.section.md new file mode 100644 index 00000000000..a7ccb3dbd04 --- /dev/null +++ b/nixos/doc/manual/development/unit-handling.section.md @@ -0,0 +1,62 @@ +# Unit handling {#sec-unit-handling} + +To figure out what units need to be started/stopped/restarted/reloaded, the +script first checks the current state of the system, similar to what `systemctl +list-units` shows. For each of the units, the script goes through the following +checks: + +- Is the unit file still in the new system? If not, **stop** the service unless + it sets `X-StopOnRemoval` in the `[Unit]` section to `false`. + +- Is it a `.target` unit? If so, **start** it unless it sets + `RefuseManualStart` in the `[Unit]` section to `true` or `X-OnlyManualStart` + in the `[Unit]` section to `true`. Also **stop** the unit again unless it + sets `X-StopOnReconfiguration` to `false`. + +- Are the contents of the unit files different? They are compared by parsing + them and comparing their contents. If they are different but only + `X-Reload-Triggers` in the `[Unit]` section is changed, **reload** the unit. + The NixOS module system allows setting these triggers with the option + [systemd.services.\<name\>.reloadTriggers](#opt-systemd.services). There are + some additional keys in the `[Unit]` section that are ignored as well. If the + unit files differ in any way, the following actions are performed: + + - `.path` and `.slice` units are ignored. There is no need to restart them + since changes in their values are applied by systemd when systemd is + reloaded. + + - `.mount` units are **reload**ed. These mostly come from the `/etc/fstab` + parser. + + - `.socket` units are currently ignored. This is to be fixed at a later + point. + + - The rest of the units (mostly `.service` units) are then **reload**ed if + `X-ReloadIfChanged` in the `[Service]` section is set to `true` (exposed + via [systemd.services.\<name\>.reloadIfChanged](#opt-systemd.services)). + A little exception is done for units that were deactivated in the meantime, + for example because they require a unit that got stopped before. These + are **start**ed instead of reloaded. + + - If the reload flag is not set, some more flags decide if the unit is + skipped. These flags are `X-RestartIfChanged` in the `[Service]` section + (exposed via + [systemd.services.\<name\>.restartIfChanged](#opt-systemd.services)), + `RefuseManualStop` in the `[Unit]` section, and `X-OnlyManualStart` in the + `[Unit]` section. + + - Further behavior depends on the unit having `X-StopIfChanged` in the + `[Service]` section set to `true` (exposed via + [systemd.services.\<name\>.stopIfChanged](#opt-systemd.services)). This is + set to `true` by default and must be explicitly turned off if not wanted. + If the flag is enabled, the unit is **stop**ped and then **start**ed. If + not, the unit is **restart**ed. The goal of the flag is to make sure that + the new unit never runs in the old environment which is still in place + before the activation script is run. This behavior is different when the + service is socket-activated, as outlined in the following steps. + + - The last thing that is taken into account is whether the unit is a service + and socket-activated. If `X-StopIfChanged` is **not** set, the service + is **restart**ed with the others. If it is set, both the service and the + socket are **stop**ped and the socket is **start**ed, leaving socket + activation to start the service when it's needed. diff --git a/nixos/doc/manual/development/what-happens-during-a-system-switch.chapter.md b/nixos/doc/manual/development/what-happens-during-a-system-switch.chapter.md new file mode 100644 index 00000000000..aad82831a3c --- /dev/null +++ b/nixos/doc/manual/development/what-happens-during-a-system-switch.chapter.md @@ -0,0 +1,53 @@ +# What happens during a system switch? {#sec-switching-systems} + +Running `nixos-rebuild switch` is one of the more common tasks under NixOS. +This chapter explains some of the internals of this command to make it simpler +for new module developers to configure their units correctly and to make it +easier to understand what is happening and why for curious administrators. + +`nixos-rebuild`, like many deployment solutions, calls `switch-to-configuration` +which resides in a NixOS system at `$out/bin/switch-to-configuration`. The +script is called with the action that is to be performed like `switch`, `test`, +`boot`. There is also the `dry-activate` action which does not really perform +the actions but rather prints what it would do if you called it with `test`. +This feature can be used to check what service states would be changed if the +configuration was switched to. + +If the action is `switch` or `boot`, the bootloader is updated first so the +configuration will be the next one to boot. Unless `NIXOS_NO_SYNC` is set to +`1`, `/nix/store` is synced to disk. + +If the action is `switch` or `test`, the currently running system is inspected +and the actions to switch to the new system are calculated. This process takes +two data sources into account: `/etc/fstab` and the current systemd status. +Mounts and swaps are read from `/etc/fstab` and the corresponding actions are +generated. If a new mount is added, for example, the proper `.mount` unit is +marked to be started. The current systemd state is inspected, the difference +between the current system and the desired configuration is calculated and +actions are generated to get to this state. There are a lot of nuances that can +be controlled by the units which are explained here. + +After calculating what should be done, the actions are carried out. The order +of actions is always the same: +- Stop units (`systemctl stop`) +- Run activation script (`$out/activate`) +- See if the activation script requested more units to restart +- Restart systemd if needed (`systemd daemon-reexec`) +- Forget about the failed state of units (`systemctl reset-failed`) +- Reload systemd (`systemctl daemon-reload`) +- Reload systemd user instances (`systemctl --user daemon-reload`) +- Set up tmpfiles (`systemd-tmpfiles --create`) +- Reload units (`systemctl reload`) +- Restart units (`systemctl restart`) +- Start units (`systemctl start`) +- Inspect what changed during these actions and print units that failed and + that were newly started + +Most of these actions are either self-explaining but some of them have to do +with our units or the activation script. For this reason, these topics are +explained in the next sections. + +```{=docbook} +<xi:include href="unit-handling.section.xml" /> +<xi:include href="activation-script.section.xml" /> +``` diff --git a/nixos/doc/manual/development/writing-documentation.chapter.md b/nixos/doc/manual/development/writing-documentation.chapter.md new file mode 100644 index 00000000000..7c29f600d70 --- /dev/null +++ b/nixos/doc/manual/development/writing-documentation.chapter.md @@ -0,0 +1,93 @@ +# Writing NixOS Documentation {#sec-writing-documentation} + +As NixOS grows, so too does the need for a catalogue and explanation of +its extensive functionality. Collecting pertinent information from +disparate sources and presenting it in an accessible style would be a +worthy contribution to the project. + +## Building the Manual {#sec-writing-docs-building-the-manual} + +The DocBook sources of the [](#book-nixos-manual) are in the +[`nixos/doc/manual`](https://github.com/NixOS/nixpkgs/tree/master/nixos/doc/manual) +subdirectory of the Nixpkgs repository. + +You can quickly validate your edits with `make`: + +```ShellSession +$ cd /path/to/nixpkgs/nixos/doc/manual +$ nix-shell +nix-shell$ make +``` + +Once you are done making modifications to the manual, it\'s important to +build it before committing. You can do that as follows: + +```ShellSession +nix-build nixos/release.nix -A manual.x86_64-linux +``` + +When this command successfully finishes, it will tell you where the +manual got generated. The HTML will be accessible through the `result` +symlink at `./result/share/doc/nixos/index.html`. + +## Editing DocBook XML {#sec-writing-docs-editing-docbook-xml} + +For general information on how to write in DocBook, see [DocBook 5: The +Definitive Guide](http://www.docbook.org/tdg5/en/html/docbook.html). + +Emacs nXML Mode is very helpful for editing DocBook XML because it +validates the document as you write, and precisely locates errors. To +use it, see [](#sec-emacs-docbook-xml). + +[Pandoc](http://pandoc.org) can generate DocBook XML from a multitude of +formats, which makes a good starting point. Here is an example of Pandoc +invocation to convert GitHub-Flavoured MarkDown to DocBook 5 XML: + +```ShellSession +pandoc -f markdown_github -t docbook5 docs.md -o my-section.md +``` + +Pandoc can also quickly convert a single `section.xml` to HTML, which is +helpful when drafting. + +Sometimes writing valid DocBook is simply too difficult. In this case, +submit your documentation updates in a [GitHub +Issue](https://github.com/NixOS/nixpkgs/issues/new) and someone will +handle the conversion to XML for you. + +## Creating a Topic {#sec-writing-docs-creating-a-topic} + +You can use an existing topic as a basis for the new topic or create a +topic from scratch. + +Keep the following guidelines in mind when you create and add a topic: + +- The NixOS [`book`](http://www.docbook.org/tdg5/en/html/book.html) + element is in `nixos/doc/manual/manual.xml`. It includes several + [`parts`](http://www.docbook.org/tdg5/en/html/book.html) which are in + subdirectories. + +- Store the topic file in the same directory as the `part` to which it + belongs. If your topic is about configuring a NixOS module, then the + XML file can be stored alongside the module definition `nix` file. + +- If you include multiple words in the file name, separate the words + with a dash. For example: `ipv6-config.xml`. + +- Make sure that the `xml:id` value is unique. You can use abbreviations + if the ID is too long. For example: `nixos-config`. + +- Determine whether your topic is a chapter or a section. If you are + unsure, open an existing topic file and check whether the main + element is chapter or section. + +## Adding a Topic to the Book {#sec-writing-docs-adding-a-topic} + +Open the parent XML file and add an `xi:include` element to the list of +chapters with the file name of the topic that you created. If you +created a `section`, you add the file to the `chapter` file. If you created +a `chapter`, you add the file to the `part` file. + +If the topic is about configuring a NixOS module, it can be +automatically included in the manual by using the `meta.doc` attribute. +See [](#sec-meta-attributes) for an explanation. diff --git a/nixos/doc/manual/development/writing-modules.chapter.md b/nixos/doc/manual/development/writing-modules.chapter.md new file mode 100644 index 00000000000..0c41cbd3cb7 --- /dev/null +++ b/nixos/doc/manual/development/writing-modules.chapter.md @@ -0,0 +1,208 @@ +# Writing NixOS Modules {#sec-writing-modules} + +NixOS has a modular system for declarative configuration. This system +combines multiple *modules* to produce the full system configuration. +One of the modules that constitute the configuration is +`/etc/nixos/configuration.nix`. Most of the others live in the +[`nixos/modules`](https://github.com/NixOS/nixpkgs/tree/master/nixos/modules) +subdirectory of the Nixpkgs tree. + +Each NixOS module is a file that handles one logical aspect of the +configuration, such as a specific kind of hardware, a service, or +network settings. A module configuration does not have to handle +everything from scratch; it can use the functionality provided by other +modules for its implementation. Thus a module can *declare* options that +can be used by other modules, and conversely can *define* options +provided by other modules in its own implementation. For example, the +module +[`pam.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/security/pam.nix) +declares the option `security.pam.services` that allows other modules (e.g. +[`sshd.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/networking/ssh/sshd.nix)) +to define PAM services; and it defines the option `environment.etc` (declared by +[`etc.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/system/etc/etc.nix)) +to cause files to be created in `/etc/pam.d`. + +In [](#sec-configuration-syntax), we saw the following structure of +NixOS modules: + +```nix +{ config, pkgs, ... }: + +{ option definitions +} +``` + +This is actually an *abbreviated* form of module that only defines +options, but does not declare any. The structure of full NixOS modules +is shown in [Example: Structure of NixOS Modules](#ex-module-syntax). + +::: {#ex-module-syntax .example} +::: {.title} +**Example: Structure of NixOS Modules** +::: +```nix +{ config, pkgs, ... }: + +{ + imports = + [ paths of other modules + ]; + + options = { + option declarations + }; + + config = { + option definitions + }; +} +``` +::: + +The meaning of each part is as follows. + +- The first line makes the current Nix expression a function. The variable + `pkgs` contains Nixpkgs (by default, it takes the `nixpkgs` entry of + `NIX_PATH`, see the [Nix manual](https://nixos.org/manual/nix/stable/#sec-common-env) + for further details), while `config` contains the full system + configuration. This line can be omitted if there is no reference to + `pkgs` and `config` inside the module. + +- This `imports` list enumerates the paths to other NixOS modules that + should be included in the evaluation of the system configuration. A + default set of modules is defined in the file `modules/module-list.nix`. + These don\'t need to be added in the import list. + +- The attribute `options` is a nested set of *option declarations* + (described below). + +- The attribute `config` is a nested set of *option definitions* (also + described below). + +[Example: NixOS Module for the "locate" Service](#locate-example) +shows a module that handles the regular update of the "locate" database, +an index of all files in the file system. This module declares two +options that can be defined by other modules (typically the user's +`configuration.nix`): `services.locate.enable` (whether the database should +be updated) and `services.locate.interval` (when the update should be done). +It implements its functionality by defining two options declared by other +modules: `systemd.services` (the set of all systemd services) and +`systemd.timers` (the list of commands to be executed periodically by +`systemd`). + +Care must be taken when writing systemd services using `Exec*` directives. By +default systemd performs substitution on `%<char>` specifiers in these +directives, expands environment variables from `$FOO` and `${FOO}`, splits +arguments on whitespace, and splits commands on `;`. All of these must be escaped +to avoid unexpected substitution or splitting when interpolating into an `Exec*` +directive, e.g. when using an `extraArgs` option to pass additional arguments to +the service. The functions `utils.escapeSystemdExecArg` and +`utils.escapeSystemdExecArgs` are provided for this, see [Example: Escaping in +Exec directives](#exec-escaping-example) for an example. When using these +functions system environment substitution should *not* be disabled explicitly. + +::: {#locate-example .example} +::: {.title} +**Example: NixOS Module for the "locate" Service** +::: +```nix +{ config, lib, pkgs, ... }: + +with lib; + +let + cfg = config.services.locate; +in { + options.services.locate = { + enable = mkOption { + type = types.bool; + default = false; + description = '' + If enabled, NixOS will periodically update the database of + files used by the locate command. + ''; + }; + + interval = mkOption { + type = types.str; + default = "02:15"; + example = "hourly"; + description = '' + Update the locate database at this interval. Updates by + default at 2:15 AM every day. + + The format is described in + systemd.time(7). + ''; + }; + + # Other options omitted for documentation + }; + + config = { + systemd.services.update-locatedb = + { description = "Update Locate Database"; + path = [ pkgs.su ]; + script = + '' + mkdir -m 0755 -p $(dirname ${toString cfg.output}) + exec updatedb \ + --localuser=${cfg.localuser} \ + ${optionalString (!cfg.includeStore) "--prunepaths='/nix/store'"} \ + --output=${toString cfg.output} ${concatStringsSep " " cfg.extraFlags} + ''; + }; + + systemd.timers.update-locatedb = mkIf cfg.enable + { description = "Update timer for locate database"; + partOf = [ "update-locatedb.service" ]; + wantedBy = [ "timers.target" ]; + timerConfig.OnCalendar = cfg.interval; + }; + }; +} +``` +::: + +::: {#exec-escaping-example .example} +::: {.title} +**Example: Escaping in Exec directives** +::: +```nix +{ config, lib, pkgs, utils, ... }: + +with lib; + +let + cfg = config.services.echo; + echoAll = pkgs.writeScript "echo-all" '' + #! ${pkgs.runtimeShell} + for s in "$@"; do + printf '%s\n' "$s" + done + ''; + args = [ "a%Nything" "lang=\${LANG}" ";" "/bin/sh -c date" ]; +in { + systemd.services.echo = + { description = "Echo to the journal"; + wantedBy = [ "multi-user.target" ]; + serviceConfig.Type = "oneshot"; + serviceConfig.ExecStart = '' + ${echoAll} ${utils.escapeSystemdExecArgs args} + ''; + }; +} +``` +::: + +```{=docbook} +<xi:include href="option-declarations.section.xml" /> +<xi:include href="option-types.section.xml" /> +<xi:include href="option-def.section.xml" /> +<xi:include href="assertions.section.xml" /> +<xi:include href="meta-attributes.section.xml" /> +<xi:include href="importing-modules.section.xml" /> +<xi:include href="replace-modules.section.xml" /> +<xi:include href="freeform-modules.section.xml" /> +<xi:include href="settings-options.section.xml" /> +``` diff --git a/nixos/doc/manual/development/writing-nixos-tests.section.md b/nixos/doc/manual/development/writing-nixos-tests.section.md new file mode 100644 index 00000000000..7de57d0d2a3 --- /dev/null +++ b/nixos/doc/manual/development/writing-nixos-tests.section.md @@ -0,0 +1,366 @@ +# Writing Tests {#sec-writing-nixos-tests} + +A NixOS test is a Nix expression that has the following structure: + +```nix +import ./make-test-python.nix { + + # Either the configuration of a single machine: + machine = + { config, pkgs, ... }: + { configuration… + }; + + # Or a set of machines: + nodes = + { machine1 = + { config, pkgs, ... }: { … }; + machine2 = + { config, pkgs, ... }: { … }; + … + }; + + testScript = + '' + Python code… + ''; +} +``` + +The attribute `testScript` is a bit of Python code that executes the +test (described below). During the test, it will start one or more +virtual machines, the configuration of which is described by the +attribute `machine` (if you need only one machine in your test) or by +the attribute `nodes` (if you need multiple machines). For instance, +[`login.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix) +only needs a single machine to test whether users can log in +on the virtual console, whether device ownership is correctly maintained +when switching between consoles, and so on. On the other hand, +[`nfs/simple.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nfs/simple.nix), +which tests NFS client and server functionality in the +Linux kernel (including whether locks are maintained across server +crashes), requires three machines: a server and two clients. + +There are a few special NixOS configuration options for test VMs: + +`virtualisation.memorySize` + +: The memory of the VM in megabytes. + +`virtualisation.vlans` + +: The virtual networks to which the VM is connected. See + [`nat.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nat.nix) + for an example. + +`virtualisation.writableStore` + +: By default, the Nix store in the VM is not writable. If you enable + this option, a writable union file system is mounted on top of the + Nix store to make it appear writable. This is necessary for tests + that run Nix operations that modify the store. + +For more options, see the module +[`qemu-vm.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix). + +The test script is a sequence of Python statements that perform various +actions, such as starting VMs, executing commands in the VMs, and so on. +Each virtual machine is represented as an object stored in the variable +`name` if this is also the identifier of the machine in the declarative +config. If you didn\'t specify multiple machines using the `nodes` +attribute, it is just `machine`. The following example starts the +machine, waits until it has finished booting, then executes a command +and checks that the output is more-or-less correct: + +```py +machine.start() +machine.wait_for_unit("default.target") +if not "Linux" in machine.succeed("uname"): + raise Exception("Wrong OS") +``` + +The first line is actually unnecessary; machines are implicitly started +when you first execute an action on them (such as `wait_for_unit` or +`succeed`). If you have multiple machines, you can speed up the test by +starting them in parallel: + +```py +start_all() +``` + +## Machine objects {#ssec-machine-objects} + +The following methods are available on machine objects: + +`start` + +: Start the virtual machine. This method is asynchronous --- it does + not wait for the machine to finish booting. + +`shutdown` + +: Shut down the machine, waiting for the VM to exit. + +`crash` + +: Simulate a sudden power failure, by telling the VM to exit + immediately. + +`block` + +: Simulate unplugging the Ethernet cable that connects the machine to + the other machines. + +`unblock` + +: Undo the effect of `block`. + +`screenshot` + +: Take a picture of the display of the virtual machine, in PNG format. + The screenshot is linked from the HTML log. + +`get_screen_text_variants` + +: Return a list of different interpretations of what is currently + visible on the machine\'s screen using optical character + recognition. The number and order of the interpretations is not + specified and is subject to change, but if no exception is raised at + least one will be returned. + + ::: {.note} + This requires passing `enableOCR` to the test attribute set. + ::: + +`get_screen_text` + +: Return a textual representation of what is currently visible on the + machine\'s screen using optical character recognition. + + ::: {.note} + This requires passing `enableOCR` to the test attribute set. + ::: + +`send_monitor_command` + +: Send a command to the QEMU monitor. This is rarely used, but allows + doing stuff such as attaching virtual USB disks to a running + machine. + +`send_key` + +: Simulate pressing keys on the virtual keyboard, e.g., + `send_key("ctrl-alt-delete")`. + +`send_chars` + +: Simulate typing a sequence of characters on the virtual keyboard, + e.g., `send_chars("foobar\n")` will type the string `foobar` + followed by the Enter key. + +`execute` + +: Execute a shell command, returning a list `(status, stdout)`. + If the command detaches, it must close stdout, as `execute` will wait + for this to consume all output reliably. This can be achieved by + redirecting stdout to stderr `>&2`, to `/dev/console`, `/dev/null` or + a file. Examples of detaching commands are `sleep 365d &`, where the + shell forks a new process that can write to stdout and `xclip -i`, where + the `xclip` command itself forks without closing stdout. + Takes an optional parameter `check_return` that defaults to `True`. + Setting this parameter to `False` will not check for the return code + and return -1 instead. This can be used for commands that shut down + the VM and would therefore break the pipe that would be used for + retrieving the return code. + +`succeed` + +: Execute a shell command, raising an exception if the exit status is + not zero, otherwise returning the standard output. Commands are run + with `set -euo pipefail` set: + + - If several commands are separated by `;` and one fails, the + command as a whole will fail. + + - For pipelines, the last non-zero exit status will be returned + (if there is one, zero will be returned otherwise). + + - Dereferencing unset variables fail the command. + + - It will wait for stdout to be closed. See `execute` for the + implications. + +`fail` + +: Like `succeed`, but raising an exception if the command returns a zero + status. + +`wait_until_succeeds` + +: Repeat a shell command with 1-second intervals until it succeeds. + +`wait_until_fails` + +: Repeat a shell command with 1-second intervals until it fails. + +`wait_for_unit` + +: Wait until the specified systemd unit has reached the "active" + state. + +`wait_for_file` + +: Wait until the specified file exists. + +`wait_for_open_port` + +: Wait until a process is listening on the given TCP port (on + `localhost`, at least). + +`wait_for_closed_port` + +: Wait until nobody is listening on the given TCP port. + +`wait_for_x` + +: Wait until the X11 server is accepting connections. + +`wait_for_text` + +: Wait until the supplied regular expressions matches the textual + contents of the screen by using optical character recognition (see + `get_screen_text` and `get_screen_text_variants`). + + ::: {.note} + This requires passing `enableOCR` to the test attribute set. + ::: + +`wait_for_console_text` + +: Wait until the supplied regular expressions match a line of the + serial console output. This method is useful when OCR is not + possibile or accurate enough. + +`wait_for_window` + +: Wait until an X11 window has appeared whose name matches the given + regular expression, e.g., `wait_for_window("Terminal")`. + +`copy_from_host` + +: Copies a file from host to machine, e.g., + `copy_from_host("myfile", "/etc/my/important/file")`. + + The first argument is the file on the host. The file needs to be + accessible while building the nix derivation. The second argument is + the location of the file on the machine. + +`systemctl` + +: Runs `systemctl` commands with optional support for + `systemctl --user` + + ```py + machine.systemctl("list-jobs --no-pager") # runs `systemctl list-jobs --no-pager` + machine.systemctl("list-jobs --no-pager", "any-user") # spawns a shell for `any-user` and runs `systemctl --user list-jobs --no-pager` + ``` + +`shell_interact` + +: Allows you to directly interact with the guest shell. This should + only be used during test development, not in production tests. + Killing the interactive session with `Ctrl-d` or `Ctrl-c` also ends + the guest session. + +To test user units declared by `systemd.user.services` the optional +`user` argument can be used: + +```py +machine.start() +machine.wait_for_x() +machine.wait_for_unit("xautolock.service", "x-session-user") +``` + +This applies to `systemctl`, `get_unit_info`, `wait_for_unit`, +`start_job` and `stop_job`. + +For faster dev cycles it\'s also possible to disable the code-linters +(this shouldn\'t be commited though): + +```nix +import ./make-test-python.nix { + skipLint = true; + machine = + { config, pkgs, ... }: + { configuration… + }; + + testScript = + '' + Python code… + ''; +} +``` + +This will produce a Nix warning at evaluation time. To fully disable the +linter, wrap the test script in comment directives to disable the Black +linter directly (again, don\'t commit this within the Nixpkgs +repository): + +```nix + testScript = + '' + # fmt: off + Python code… + # fmt: on + ''; +``` + +## Failing tests early {#ssec-failing-tests-early} + +To fail tests early when certain invariables are no longer met (instead of waiting for the build to time out), the decorator `polling_condition` is provided. For example, if we are testing a program `foo` that should not quit after being started, we might write the following: + +```py +@polling_condition +def foo_running(): + machine.succeed("pgrep -x foo") + + +machine.succeed("foo --start") +machine.wait_until_succeeds("pgrep -x foo") + +with foo_running: + ... # Put `foo` through its paces +``` + + +`polling_condition` takes the following (optional) arguments: + +`seconds_interval` + +: + specifies how often the condition should be polled: + + ```py + @polling_condition(seconds_interval=10) + def foo_running(): + machine.succeed("pgrep -x foo") + ``` + +`description` + +: + is used in the log when the condition is checked. If this is not provided, the description is pulled from the docstring of the function. These two are therefore equivalent: + + ```py + @polling_condition + def foo_running(): + "check that foo is running" + machine.succeed("pgrep -x foo") + ``` + + ```py + @polling_condition(description="check that foo is running") + def foo_running(): + machine.succeed("pgrep -x foo") + ``` |