diff options
Diffstat (limited to 'nixos/doc/manual/from_md/development')
23 files changed, 4028 insertions, 0 deletions
diff --git a/nixos/doc/manual/from_md/development/activation-script.section.xml b/nixos/doc/manual/from_md/development/activation-script.section.xml new file mode 100644 index 00000000000..0d9e911216e --- /dev/null +++ b/nixos/doc/manual/from_md/development/activation-script.section.xml @@ -0,0 +1,150 @@ +<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-activation-script"> + <title>Activation script</title> + <para> + The activation script is a bash script called to activate the new + configuration which resides in a NixOS system in + <literal>$out/activate</literal>. Since its contents depend on your + system configuration, the contents may differ. This chapter explains + how the script works in general and some common NixOS snippets. + Please be aware that the script is executed on every boot and system + switch, so tasks that can be performed in other places should be + performed there (for example letting a directory of a service be + created by systemd using mechanisms like + <literal>StateDirectory</literal>, + <literal>CacheDirectory</literal>, … or if that’s not possible using + <literal>preStart</literal> of the service). + </para> + <para> + Activation scripts are defined as snippets using + <xref linkend="opt-system.activationScripts" />. They can either be + a simple multiline string or an attribute set that can depend on + other snippets. The builder for the activation script will take + these dependencies into account and order the snippets accordingly. + As a simple example: + </para> + <programlisting language="bash"> +system.activationScripts.my-activation-script = { + deps = [ "etc" ]; + # supportsDryActivation = true; + text = '' + echo "Hallo i bims" + ''; +}; +</programlisting> + <para> + This example creates an activation script snippet that is run after + the <literal>etc</literal> snippet. The special variable + <literal>supportsDryActivation</literal> can be set so the snippet + is also run when <literal>nixos-rebuild dry-activate</literal> is + run. To differentiate between real and dry activation, the + <literal>$NIXOS_ACTION</literal> environment variable can be read + which is set to <literal>dry-activate</literal> when a dry + activation is done. + </para> + <para> + An activation script can write to special files instructing + <literal>switch-to-configuration</literal> to restart/reload units. + The script will take these requests into account and will + incorperate the unit configuration as described above. This means + that the activation script will <quote>fake</quote> a modified unit + file and <literal>switch-to-configuration</literal> will act + accordingly. By doing so, configuration like + <link linkend="opt-systemd.services">systemd.services.<name>.restartIfChanged</link> + is respected. Since the activation script is run + <emphasis role="strong">after</emphasis> services are already + stopped, + <link linkend="opt-systemd.services">systemd.services.<name>.stopIfChanged</link> + cannot be taken into account anymore and the unit is always + restarted instead of being stopped and started afterwards. + </para> + <para> + The files that can be written to are + <literal>/run/nixos/activation-restart-list</literal> and + <literal>/run/nixos/activation-reload-list</literal> with their + respective counterparts for dry activation being + <literal>/run/nixos/dry-activation-restart-list</literal> and + <literal>/run/nixos/dry-activation-reload-list</literal>. Those + files can contain newline-separated lists of unit names where + duplicates are being ignored. These files are not create + automatically and activation scripts must take the possiblility into + account that they have to create them first. + </para> + <section xml:id="sec-activation-script-nixos-snippets"> + <title>NixOS snippets</title> + <para> + There are some snippets NixOS enables by default because disabling + them would most likely break you system. This section lists a few + of them and what they do: + </para> + <itemizedlist spacing="compact"> + <listitem> + <para> + <literal>binsh</literal> creates <literal>/bin/sh</literal> + which points to the runtime shell + </para> + </listitem> + <listitem> + <para> + <literal>etc</literal> sets up the contents of + <literal>/etc</literal>, this includes systemd units and + excludes <literal>/etc/passwd</literal>, + <literal>/etc/group</literal>, and + <literal>/etc/shadow</literal> (which are managed by the + <literal>users</literal> snippet) + </para> + </listitem> + <listitem> + <para> + <literal>hostname</literal> sets the system’s hostname in the + kernel (not in <literal>/etc</literal>) + </para> + </listitem> + <listitem> + <para> + <literal>modprobe</literal> sets the path to the + <literal>modprobe</literal> binary for module auto-loading + </para> + </listitem> + <listitem> + <para> + <literal>nix</literal> prepares the nix store and adds a + default initial channel + </para> + </listitem> + <listitem> + <para> + <literal>specialfs</literal> is responsible for mounting + filesystems like <literal>/proc</literal> and + <literal>sys</literal> + </para> + </listitem> + <listitem> + <para> + <literal>users</literal> creates and removes users and groups + by managing <literal>/etc/passwd</literal>, + <literal>/etc/group</literal> and + <literal>/etc/shadow</literal>. This also creates home + directories + </para> + </listitem> + <listitem> + <para> + <literal>usrbinenv</literal> creates + <literal>/usr/bin/env</literal> + </para> + </listitem> + <listitem> + <para> + <literal>var</literal> creates some directories in + <literal>/var</literal> that are not service-specific + </para> + </listitem> + <listitem> + <para> + <literal>wrappers</literal> creates setuid wrappers like + <literal>ping</literal> and <literal>sudo</literal> + </para> + </listitem> + </itemizedlist> + </section> +</section> diff --git a/nixos/doc/manual/from_md/development/assertions.section.xml b/nixos/doc/manual/from_md/development/assertions.section.xml new file mode 100644 index 00000000000..0844d484d60 --- /dev/null +++ b/nixos/doc/manual/from_md/development/assertions.section.xml @@ -0,0 +1,58 @@ +<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-assertions"> + <title>Warnings and Assertions</title> + <para> + When configuration problems are detectable in a module, it is a good + idea to write an assertion or warning. Doing so provides clear + feedback to the user and prevents errors after the build. + </para> + <para> + Although Nix has the <literal>abort</literal> and + <literal>builtins.trace</literal> + <link xlink:href="https://nixos.org/nix/manual/#ssec-builtins">functions</link> + to perform such tasks, they are not ideally suited for NixOS + modules. Instead of these functions, you can declare your warnings + and assertions using the NixOS module system. + </para> + <section xml:id="sec-assertions-warnings"> + <title>Warnings</title> + <para> + This is an example of using <literal>warnings</literal>. + </para> + <programlisting language="bash"> +{ config, lib, ... }: +{ + config = lib.mkIf config.services.foo.enable { + warnings = + if config.services.foo.bar + then [ ''You have enabled the bar feature of the foo service. + This is known to cause some specific problems in certain situations. + '' ] + else []; + } +} +</programlisting> + </section> + <section xml:id="sec-assertions-assetions"> + <title>Assertions</title> + <para> + This example, extracted from the + <link xlink:href="https://github.com/NixOS/nixpkgs/blob/release-17.09/nixos/modules/services/logging/syslogd.nix"><literal>syslogd</literal> + module</link> shows how to use <literal>assertions</literal>. + Since there can only be one active syslog daemon at a time, an + assertion is useful to prevent such a broken system from being + built. + </para> + <programlisting language="bash"> +{ config, lib, ... }: +{ + config = lib.mkIf config.services.syslogd.enable { + assertions = + [ { assertion = !config.services.rsyslogd.enable; + message = "rsyslogd conflicts with syslogd"; + } + ]; + } +} +</programlisting> + </section> +</section> diff --git a/nixos/doc/manual/from_md/development/building-nixos.chapter.xml b/nixos/doc/manual/from_md/development/building-nixos.chapter.xml new file mode 100644 index 00000000000..ad9349da068 --- /dev/null +++ b/nixos/doc/manual/from_md/development/building-nixos.chapter.xml @@ -0,0 +1,72 @@ +<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-building-image"> + <title>Building a NixOS (Live) ISO</title> + <para> + Default live installer configurations are available inside + <literal>nixos/modules/installer/cd-dvd</literal>. For building + other system images, + <link xlink:href="https://github.com/nix-community/nixos-generators">nixos-generators</link> + is a good place to start looking at. + </para> + <para> + You have two options: + </para> + <itemizedlist spacing="compact"> + <listitem> + <para> + Use any of those default configurations as is + </para> + </listitem> + <listitem> + <para> + Combine them with (any of) your host config(s) + </para> + </listitem> + </itemizedlist> + <para> + System images, such as the live installer ones, know how to enforce + configuration settings on wich they immediately depend in order to + work correctly. + </para> + <para> + However, if you are confident, you can opt to override those + enforced values with <literal>mkForce</literal>. + </para> + <section xml:id="sec-building-image-instructions"> + <title>Practical Instructions</title> + <programlisting> +$ git clone https://github.com/NixOS/nixpkgs.git +$ cd nixpkgs/nixos +$ nix-build -A config.system.build.isoImage -I nixos-config=modules/installer/cd-dvd/installation-cd-minimal.nix default.nix +</programlisting> + <para> + To check the content of an ISO image, mount it like so: + </para> + <programlisting> +# mount -o loop -t iso9660 ./result/iso/cd.iso /mnt/iso +</programlisting> + </section> + <section xml:id="sec-building-image-tech-notes"> + <title>Technical Notes</title> + <para> + The config value enforcement is implemented via + <literal>mkImageMediaOverride = mkOverride 60;</literal> and + therefore primes over simple value assignments, but also yields to + <literal>mkForce</literal>. + </para> + <para> + This property allows image designers to implement in semantically + correct ways those configuration values upon which the correct + functioning of the image depends. + </para> + <para> + For example, the iso base image overrides those file systems which + it needs at a minimum for correct functioning, while the installer + base image overrides the entire file system layout because there + can’t be any other guarantees on a live medium than those given by + the live medium itself. The latter is especially true befor + formatting the target block device(s). On the other hand, the + netboot iso only overrides its minimum dependencies since netboot + images are always made-to-target. + </para> + </section> +</chapter> diff --git a/nixos/doc/manual/from_md/development/building-parts.chapter.xml b/nixos/doc/manual/from_md/development/building-parts.chapter.xml new file mode 100644 index 00000000000..4df24cc9565 --- /dev/null +++ b/nixos/doc/manual/from_md/development/building-parts.chapter.xml @@ -0,0 +1,124 @@ +<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-building-parts"> + <title>Building Specific Parts of NixOS</title> + <para> + With the command <literal>nix-build</literal>, you can build + specific parts of your NixOS configuration. This is done as follows: + </para> + <programlisting> +$ cd /path/to/nixpkgs/nixos +$ nix-build -A config.option +</programlisting> + <para> + where <literal>option</literal> is a NixOS option with type + <quote>derivation</quote> (i.e. something that can be built). + Attributes of interest include: + </para> + <variablelist> + <varlistentry> + <term> + <literal>system.build.toplevel</literal> + </term> + <listitem> + <para> + The top-level option that builds the entire NixOS system. + Everything else in your configuration is indirectly pulled in + by this option. This is what <literal>nixos-rebuild</literal> + builds and what <literal>/run/current-system</literal> points + to afterwards. + </para> + <para> + A shortcut to build this is: + </para> + <programlisting> +$ nix-build -A system +</programlisting> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>system.build.manual.manualHTML</literal> + </term> + <listitem> + <para> + The NixOS manual. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>system.build.etc</literal> + </term> + <listitem> + <para> + A tree of symlinks that form the static parts of + <literal>/etc</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>system.build.initialRamdisk</literal> , + <literal>system.build.kernel</literal> + </term> + <listitem> + <para> + The initial ramdisk and kernel of the system. This allows a + quick way to test whether the kernel and the initial ramdisk + boot correctly, by using QEMU’s <literal>-kernel</literal> and + <literal>-initrd</literal> options: + </para> + <programlisting> +$ nix-build -A config.system.build.initialRamdisk -o initrd +$ nix-build -A config.system.build.kernel -o kernel +$ qemu-system-x86_64 -kernel ./kernel/bzImage -initrd ./initrd/initrd -hda /dev/null +</programlisting> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>system.build.nixos-rebuild</literal> , + <literal>system.build.nixos-install</literal> , + <literal>system.build.nixos-generate-config</literal> + </term> + <listitem> + <para> + These build the corresponding NixOS commands. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>systemd.units.unit-name.unit</literal> + </term> + <listitem> + <para> + This builds the unit with the specified name. Note that since + unit names contain dots (e.g. + <literal>httpd.service</literal>), you need to put them + between quotes, like this: + </para> + <programlisting> +$ nix-build -A 'config.systemd.units."httpd.service".unit' +</programlisting> + <para> + You can also test individual units, without rebuilding the + whole system, by putting them in + <literal>/run/systemd/system</literal>: + </para> + <programlisting> +$ cp $(nix-build -A 'config.systemd.units."httpd.service".unit')/httpd.service \ + /run/systemd/system/tmp-httpd.service +# systemctl daemon-reload +# systemctl start tmp-httpd.service +</programlisting> + <para> + Note that the unit must not have the same name as any unit in + <literal>/etc/systemd/system</literal> since those take + precedence over <literal>/run/systemd/system</literal>. That’s + why the unit is installed as + <literal>tmp-httpd.service</literal> here. + </para> + </listitem> + </varlistentry> + </variablelist> +</chapter> diff --git a/nixos/doc/manual/from_md/development/freeform-modules.section.xml b/nixos/doc/manual/from_md/development/freeform-modules.section.xml new file mode 100644 index 00000000000..86a9cf3140d --- /dev/null +++ b/nixos/doc/manual/from_md/development/freeform-modules.section.xml @@ -0,0 +1,87 @@ +<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-freeform-modules"> + <title>Freeform modules</title> + <para> + Freeform modules allow you to define values for option paths that + have not been declared explicitly. This can be used to add + attribute-specific types to what would otherwise have to be + <literal>attrsOf</literal> options in order to accept all attribute + names. + </para> + <para> + This feature can be enabled by using the attribute + <literal>freeformType</literal> to define a freeform type. By doing + this, all assignments without an associated option will be merged + using the freeform type and combined into the resulting + <literal>config</literal> set. Since this feature nullifies name + checking for entire option trees, it is only recommended for use in + submodules. + </para> + <anchor xml:id="ex-freeform-module" /> + <para> + <emphasis role="strong">Example: Freeform submodule</emphasis> + </para> + <para> + The following shows a submodule assigning a freeform type that + allows arbitrary attributes with <literal>str</literal> values below + <literal>settings</literal>, but also declares an option for the + <literal>settings.port</literal> attribute to have it type-checked + and assign a default value. See + <link linkend="ex-settings-typed-attrs">Example: Declaring a + type-checked <literal>settings</literal> attribute</link> for a more + complete example. + </para> + <programlisting language="bash"> +{ lib, config, ... }: { + + options.settings = lib.mkOption { + type = lib.types.submodule { + + freeformType = with lib.types; attrsOf str; + + # We want this attribute to be checked for the correct type + options.port = lib.mkOption { + type = lib.types.port; + # Declaring the option also allows defining a default value + default = 8080; + }; + + }; + }; +} +</programlisting> + <para> + And the following shows what such a module then allows + </para> + <programlisting language="bash"> +{ + # Not a declared option, but the freeform type allows this + settings.logLevel = "debug"; + + # Not allowed because the the freeform type only allows strings + # settings.enable = true; + + # Allowed because there is a port option declared + settings.port = 80; + + # Not allowed because the port option doesn't allow strings + # settings.port = "443"; +} +</programlisting> + <note> + <para> + Freeform attributes cannot depend on other attributes of the same + set without infinite recursion: + </para> + <programlisting language="bash"> +{ + # This throws infinite recursion encountered + settings.logLevel = lib.mkIf (config.settings.port == 80) "debug"; +} +</programlisting> + <para> + To prevent this, declare options for all attributes that need to + depend on others. For above example this means to declare + <literal>logLevel</literal> to be an option. + </para> + </note> +</section> diff --git a/nixos/doc/manual/from_md/development/importing-modules.section.xml b/nixos/doc/manual/from_md/development/importing-modules.section.xml new file mode 100644 index 00000000000..cb04dde67c8 --- /dev/null +++ b/nixos/doc/manual/from_md/development/importing-modules.section.xml @@ -0,0 +1,47 @@ +<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-importing-modules"> + <title>Importing Modules</title> + <para> + Sometimes NixOS modules need to be used in configuration but exist + outside of Nixpkgs. These modules can be imported: + </para> + <programlisting language="bash"> +{ config, lib, pkgs, ... }: + +{ + imports = + [ # Use a locally-available module definition in + # ./example-module/default.nix + ./example-module + ]; + + services.exampleModule.enable = true; +} +</programlisting> + <para> + The environment variable <literal>NIXOS_EXTRA_MODULE_PATH</literal> + is an absolute path to a NixOS module that is included alongside the + Nixpkgs NixOS modules. Like any NixOS module, this module can import + additional modules: + </para> + <programlisting language="bash"> +# ./module-list/default.nix +[ + ./example-module1 + ./example-module2 +] +</programlisting> + <programlisting language="bash"> +# ./extra-module/default.nix +{ imports = import ./module-list.nix; } +</programlisting> + <programlisting language="bash"> +# NIXOS_EXTRA_MODULE_PATH=/absolute/path/to/extra-module +{ config, lib, pkgs, ... }: + +{ + # No `imports` needed + + services.exampleModule1.enable = true; +} +</programlisting> +</section> diff --git a/nixos/doc/manual/from_md/development/linking-nixos-tests-to-packages.section.xml b/nixos/doc/manual/from_md/development/linking-nixos-tests-to-packages.section.xml new file mode 100644 index 00000000000..666bbec6162 --- /dev/null +++ b/nixos/doc/manual/from_md/development/linking-nixos-tests-to-packages.section.xml @@ -0,0 +1,10 @@ +<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-linking-nixos-tests-to-packages"> + <title>Linking NixOS tests to packages</title> + <para> + You can link NixOS module tests to the packages that they exercised, + so that the tests can be run automatically during code review when + the package gets changed. This is + <link xlink:href="https://nixos.org/manual/nixpkgs/stable/#ssec-nixos-tests-linking">described + in the nixpkgs manual</link>. + </para> +</section> diff --git a/nixos/doc/manual/from_md/development/meta-attributes.section.xml b/nixos/doc/manual/from_md/development/meta-attributes.section.xml new file mode 100644 index 00000000000..1eb6e0f3036 --- /dev/null +++ b/nixos/doc/manual/from_md/development/meta-attributes.section.xml @@ -0,0 +1,95 @@ +<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-meta-attributes"> + <title>Meta Attributes</title> + <para> + Like Nix packages, NixOS modules can declare meta-attributes to + provide extra information. Module meta attributes are defined in the + <literal>meta.nix</literal> special module. + </para> + <para> + <literal>meta</literal> is a top level attribute like + <literal>options</literal> and <literal>config</literal>. Available + meta-attributes are <literal>maintainers</literal>, + <literal>doc</literal>, and <literal>buildDocsInSandbox</literal>. + </para> + <para> + Each of the meta-attributes must be defined at most once per module + file. + </para> + <programlisting language="bash"> +{ config, lib, pkgs, ... }: +{ + options = { + ... + }; + + config = { + ... + }; + + meta = { + maintainers = with lib.maintainers; [ ericsagnes ]; + doc = ./default.xml; + buildDocsInSandbox = true; + }; +} +</programlisting> + <itemizedlist> + <listitem> + <para> + <literal>maintainers</literal> contains a list of the module + maintainers. + </para> + </listitem> + <listitem> + <para> + <literal>doc</literal> points to a valid DocBook file containing + the module documentation. Its contents is automatically added to + <xref linkend="ch-configuration" />. Changes to a module + documentation have to be checked to not break building the NixOS + manual: + </para> + <programlisting> +$ nix-build nixos/release.nix -A manual.x86_64-linux +</programlisting> + </listitem> + <listitem> + <para> + <literal>buildDocsInSandbox</literal> indicates whether the + option documentation for the module can be built in a derivation + sandbox. This option is currently only honored for modules + shipped by nixpkgs. User modules and modules taken from + <literal>NIXOS_EXTRA_MODULE_PATH</literal> are always built + outside of the sandbox, as has been the case in previous + releases. + </para> + <para> + Building NixOS option documentation in a sandbox allows caching + of the built documentation, which greatly decreases the amount + of time needed to evaluate a system configuration that has NixOS + documentation enabled. The sandbox also restricts which + attributes may be referenced by documentation attributes (such + as option descriptions) to the <literal>options</literal> and + <literal>lib</literal> module arguments and the + <literal>pkgs.formats</literal> attribute of the + <literal>pkgs</literal> argument, <literal>config</literal> and + the rest of <literal>pkgs</literal> are disallowed and will + cause doc build failures when used. This restriction is + necessary because we cannot reproduce the full nixpkgs + instantiation with configuration and overlays from a system + configuration inside the sandbox. The <literal>options</literal> + argument only includes options of modules that are also built + inside the sandbox, referencing an option of a module that isn’t + built in the sandbox is also forbidden. + </para> + <para> + The default is <literal>true</literal> and should usually not be + changed; set it to <literal>false</literal> only if the module + requires access to <literal>pkgs</literal> in its documentation + (e.g. because it loads information from a linked package to + build an option type) or if its documentation depends on other + modules that also aren’t sandboxed (e.g. by using types defined + in the other module). + </para> + </listitem> + </itemizedlist> +</section> diff --git a/nixos/doc/manual/from_md/development/nixos-tests.chapter.xml b/nixos/doc/manual/from_md/development/nixos-tests.chapter.xml new file mode 100644 index 00000000000..b9ff2269676 --- /dev/null +++ b/nixos/doc/manual/from_md/development/nixos-tests.chapter.xml @@ -0,0 +1,14 @@ +<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xi="http://www.w3.org/2001/XInclude" xml:id="sec-nixos-tests"> + <title>NixOS Tests</title> + <para> + When you add some feature to NixOS, you should write a test for it. + NixOS tests are kept in the directory + <literal>nixos/tests</literal>, and are executed (using Nix) by a + testing framework that automatically starts one or more virtual + machines containing the NixOS system(s) required for the test. + </para> + <xi:include href="writing-nixos-tests.section.xml" /> + <xi:include href="running-nixos-tests.section.xml" /> + <xi:include href="running-nixos-tests-interactively.section.xml" /> + <xi:include href="linking-nixos-tests-to-packages.section.xml" /> +</chapter> diff --git a/nixos/doc/manual/from_md/development/option-declarations.section.xml b/nixos/doc/manual/from_md/development/option-declarations.section.xml new file mode 100644 index 00000000000..0ac5e0eeca2 --- /dev/null +++ b/nixos/doc/manual/from_md/development/option-declarations.section.xml @@ -0,0 +1,327 @@ +<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-option-declarations"> + <title>Option Declarations</title> + <para> + An option declaration specifies the name, type and description of a + NixOS configuration option. It is invalid to define an option that + hasn’t been declared in any module. An option declaration generally + looks like this: + </para> + <programlisting language="bash"> +options = { + name = mkOption { + type = type specification; + default = default value; + example = example value; + description = "Description for use in the NixOS manual."; + }; +}; +</programlisting> + <para> + The attribute names within the <literal>name</literal> attribute + path must be camel cased in general but should, as an exception, + match the + <link xlink:href="https://nixos.org/nixpkgs/manual/#sec-package-naming"> + package attribute name</link> when referencing a Nixpkgs package. + For example, the option + <literal>services.nix-serve.bindAddress</literal> references the + <literal>nix-serve</literal> Nixpkgs package. + </para> + <para> + The function <literal>mkOption</literal> accepts the following + arguments. + </para> + <variablelist> + <varlistentry> + <term> + <literal>type</literal> + </term> + <listitem> + <para> + The type of the option (see + <xref linkend="sec-option-types" />). This argument is + mandatory for nixpkgs modules. Setting this is highly + recommended for the sake of documentation and type checking. + In case it is not set, a fallback type with unspecified + behavior is used. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>default</literal> + </term> + <listitem> + <para> + The default value used if no value is defined by any module. A + default is not required; but if a default is not given, then + users of the module will have to define the value of the + option, otherwise an error will be thrown. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>defaultText</literal> + </term> + <listitem> + <para> + A textual representation of the default value to be rendered + verbatim in the manual. Useful if the default value is a + complex expression or depends on other values or packages. Use + <literal>lib.literalExpression</literal> for a Nix expression, + <literal>lib.literalDocBook</literal> for a plain English + description in DocBook format. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>example</literal> + </term> + <listitem> + <para> + An example value that will be shown in the NixOS manual. You + can use <literal>lib.literalExpression</literal> and + <literal>lib.literalDocBook</literal> in the same way as in + <literal>defaultText</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>description</literal> + </term> + <listitem> + <para> + A textual description of the option, in DocBook format, that + will be included in the NixOS manual. + </para> + </listitem> + </varlistentry> + </variablelist> + <section xml:id="sec-option-declarations-util"> + <title>Utility functions for common option patterns</title> + <section xml:id="sec-option-declarations-util-mkEnableOption"> + <title><literal>mkEnableOption</literal></title> + <para> + Creates an Option attribute set for a boolean value option i.e + an option to be toggled on or off. + </para> + <para> + This function takes a single string argument, the name of the + thing to be toggled. + </para> + <para> + The option’s description is <quote>Whether to enable + <name>.</quote>. + </para> + <para> + For example: + </para> + <anchor xml:id="ex-options-declarations-util-mkEnableOption-magic" /> + <programlisting language="bash"> +lib.mkEnableOption "magic" +# is like +lib.mkOption { + type = lib.types.bool; + default = false; + example = true; + description = "Whether to enable magic."; +} +</programlisting> + <section xml:id="sec-option-declarations-util-mkPackageOption"> + <title><literal>mkPackageOption</literal></title> + <para> + Usage: + </para> + <programlisting language="bash"> +mkPackageOption pkgs "name" { default = [ "path" "in" "pkgs" ]; example = "literal example"; } +</programlisting> + <para> + Creates an Option attribute set for an option that specifies + the package a module should use for some purpose. + </para> + <para> + <emphasis role="strong">Note</emphasis>: You shouldn’t + necessarily make package options for all of your modules. You + can always overwrite a specific package throughout nixpkgs by + using + <link xlink:href="https://nixos.org/manual/nixpkgs/stable/#chap-overlays">nixpkgs + overlays</link>. + </para> + <para> + The default package is specified as a list of strings + representing its attribute path in nixpkgs. Because of this, + you need to pass nixpkgs itself as the first argument. + </para> + <para> + The second argument is the name of the option, used in the + description <quote>The <name> package to use.</quote>. + You can also pass an example value, either a literal string or + a package’s attribute path. + </para> + <para> + You can omit the default path if the name of the option is + also attribute path in nixpkgs. + </para> + <anchor xml:id="ex-options-declarations-util-mkPackageOption" /> + <para> + Examples: + </para> + <anchor xml:id="ex-options-declarations-util-mkPackageOption-hello" /> + <programlisting language="bash"> +lib.mkPackageOption pkgs "hello" { } +# is like +lib.mkOption { + type = lib.types.package; + default = pkgs.hello; + defaultText = lib.literalExpression "pkgs.hello"; + description = "The hello package to use."; +} +</programlisting> + <anchor xml:id="ex-options-declarations-util-mkPackageOption-ghc" /> + <programlisting language="bash"> +lib.mkPackageOption pkgs "GHC" { + default = [ "ghc" ]; + example = "pkgs.haskell.package.ghc921.ghc.withPackages (hkgs: [ hkgs.primes ])"; +} +# is like +lib.mkOption { + type = lib.types.package; + default = pkgs.ghc; + defaultText = lib.literalExpression "pkgs.ghc"; + example = lib.literalExpression "pkgs.haskell.package.ghc921.ghc.withPackages (hkgs: [ hkgs.primes ])"; + description = "The GHC package to use."; +} +</programlisting> + <section xml:id="sec-option-declarations-eot"> + <title>Extensible Option Types</title> + <para> + Extensible option types is a feature that allow to extend + certain types declaration through multiple module files. + This feature only work with a restricted set of types, + namely <literal>enum</literal> and + <literal>submodules</literal> and any composed forms of + them. + </para> + <para> + Extensible option types can be used for + <literal>enum</literal> options that affects multiple + modules, or as an alternative to related + <literal>enable</literal> options. + </para> + <para> + As an example, we will take the case of display managers. + There is a central display manager module for generic + display manager options and a module file per display + manager backend (sddm, gdm ...). + </para> + <para> + There are two approaches we could take with this module + structure: + </para> + <itemizedlist> + <listitem> + <para> + Configuring the display managers independently by adding + an enable option to every display manager module + backend. (NixOS) + </para> + </listitem> + <listitem> + <para> + Configuring the display managers in the central module + by adding an option to select which display manager + backend to use. + </para> + </listitem> + </itemizedlist> + <para> + Both approaches have problems. + </para> + <para> + Making backends independent can quickly become hard to + manage. For display managers, there can only be one enabled + at a time, but the type system cannot enforce this + restriction as there is no relation between each backend’s + <literal>enable</literal> option. As a result, this + restriction has to be done explicitly by adding assertions + in each display manager backend module. + </para> + <para> + On the other hand, managing the display manager backends in + the central module will require changing the central module + option every time a new backend is added or removed. + </para> + <para> + By using extensible option types, it is possible to create a + placeholder option in the central module + (<link linkend="ex-option-declaration-eot-service">Example: + Extensible type placeholder in the service module</link>), + and to extend it in each backend module + (<link linkend="ex-option-declaration-eot-backend-gdm">Example: + Extending + <literal>services.xserver.displayManager.enable</literal> in + the <literal>gdm</literal> module</link>, + <link linkend="ex-option-declaration-eot-backend-sddm">Example: + Extending + <literal>services.xserver.displayManager.enable</literal> in + the <literal>sddm</literal> module</link>). + </para> + <para> + As a result, <literal>displayManager.enable</literal> option + values can be added without changing the main service module + file and the type system automatically enforces that there + can only be a single display manager enabled. + </para> + <anchor xml:id="ex-option-declaration-eot-service" /> + <para> + <emphasis role="strong">Example: Extensible type placeholder + in the service module</emphasis> + </para> + <programlisting language="bash"> +services.xserver.displayManager.enable = mkOption { + description = "Display manager to use"; + type = with types; nullOr (enum [ ]); +}; +</programlisting> + <anchor xml:id="ex-option-declaration-eot-backend-gdm" /> + <para> + <emphasis role="strong">Example: Extending + <literal>services.xserver.displayManager.enable</literal> in + the <literal>gdm</literal> module</emphasis> + </para> + <programlisting language="bash"> +services.xserver.displayManager.enable = mkOption { + type = with types; nullOr (enum [ "gdm" ]); +}; +</programlisting> + <anchor xml:id="ex-option-declaration-eot-backend-sddm" /> + <para> + <emphasis role="strong">Example: Extending + <literal>services.xserver.displayManager.enable</literal> in + the <literal>sddm</literal> module</emphasis> + </para> + <programlisting language="bash"> +services.xserver.displayManager.enable = mkOption { + type = with types; nullOr (enum [ "sddm" ]); +}; +</programlisting> + <para> + The placeholder declaration is a standard + <literal>mkOption</literal> declaration, but it is important + that extensible option declarations only use the + <literal>type</literal> argument. + </para> + <para> + Extensible option types work with any of the composed + variants of <literal>enum</literal> such as + <literal>with types; nullOr (enum [ "foo" "bar" ])</literal> + or + <literal>with types; listOf (enum [ "foo" "bar" ])</literal>. + </para> + </section> + </section> + </section> + </section> +</section> diff --git a/nixos/doc/manual/from_md/development/option-def.section.xml b/nixos/doc/manual/from_md/development/option-def.section.xml new file mode 100644 index 00000000000..8c9ef181aff --- /dev/null +++ b/nixos/doc/manual/from_md/development/option-def.section.xml @@ -0,0 +1,104 @@ +<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-option-definitions"> + <title>Option Definitions</title> + <para> + Option definitions are generally straight-forward bindings of values + to option names, like + </para> + <programlisting language="bash"> +config = { + services.httpd.enable = true; +}; +</programlisting> + <para> + However, sometimes you need to wrap an option definition or set of + option definitions in a <emphasis>property</emphasis> to achieve + certain effects: + </para> + <section xml:id="sec-option-definitions-delaying-conditionals"> + <title>Delaying Conditionals</title> + <para> + If a set of option definitions is conditional on the value of + another option, you may need to use <literal>mkIf</literal>. + Consider, for instance: + </para> + <programlisting language="bash"> +config = if config.services.httpd.enable then { + environment.systemPackages = [ ... ]; + ... +} else {}; +</programlisting> + <para> + This definition will cause Nix to fail with an <quote>infinite + recursion</quote> error. Why? Because the value of + <literal>config.services.httpd.enable</literal> depends on the + value being constructed here. After all, you could also write the + clearly circular and contradictory: + </para> + <programlisting language="bash"> +config = if config.services.httpd.enable then { + services.httpd.enable = false; +} else { + services.httpd.enable = true; +}; +</programlisting> + <para> + The solution is to write: + </para> + <programlisting language="bash"> +config = mkIf config.services.httpd.enable { + environment.systemPackages = [ ... ]; + ... +}; +</programlisting> + <para> + The special function <literal>mkIf</literal> causes the evaluation + of the conditional to be <quote>pushed down</quote> into the + individual definitions, as if you had written: + </para> + <programlisting language="bash"> +config = { + environment.systemPackages = if config.services.httpd.enable then [ ... ] else []; + ... +}; +</programlisting> + </section> + <section xml:id="sec-option-definitions-setting-priorities"> + <title>Setting Priorities</title> + <para> + A module can override the definitions of an option in other + modules by setting a <emphasis>priority</emphasis>. All option + definitions that do not have the lowest priority value are + discarded. By default, option definitions have priority 1000. You + can specify an explicit priority by using + <literal>mkOverride</literal>, e.g. + </para> + <programlisting language="bash"> +services.openssh.enable = mkOverride 10 false; +</programlisting> + <para> + This definition causes all other definitions with priorities above + 10 to be discarded. The function <literal>mkForce</literal> is + equal to <literal>mkOverride 50</literal>. + </para> + </section> + <section xml:id="sec-option-definitions-merging"> + <title>Merging Configurations</title> + <para> + In conjunction with <literal>mkIf</literal>, it is sometimes + useful for a module to return multiple sets of option definitions, + to be merged together as if they were declared in separate + modules. This can be done using <literal>mkMerge</literal>: + </para> + <programlisting language="bash"> +config = mkMerge + [ # Unconditional stuff. + { environment.systemPackages = [ ... ]; + } + # Conditional stuff. + (mkIf config.services.bla.enable { + environment.systemPackages = [ ... ]; + }) + ]; +</programlisting> + </section> +</section> diff --git a/nixos/doc/manual/from_md/development/option-types.section.xml b/nixos/doc/manual/from_md/development/option-types.section.xml new file mode 100644 index 00000000000..44472929270 --- /dev/null +++ b/nixos/doc/manual/from_md/development/option-types.section.xml @@ -0,0 +1,1038 @@ +<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-option-types"> + <title>Options Types</title> + <para> + Option types are a way to put constraints on the values a module + option can take. Types are also responsible of how values are merged + in case of multiple value definitions. + </para> + <section xml:id="sec-option-types-basic"> + <title>Basic Types</title> + <para> + Basic types are the simplest available types in the module system. + Basic types include multiple string types that mainly differ in + how definition merging is handled. + </para> + <variablelist> + <varlistentry> + <term> + <literal>types.bool</literal> + </term> + <listitem> + <para> + A boolean, its values can be <literal>true</literal> or + <literal>false</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.path</literal> + </term> + <listitem> + <para> + A filesystem path is anything that starts with a slash when + coerced to a string. Even if derivations can be considered + as paths, the more specific <literal>types.package</literal> + should be preferred. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.package</literal> + </term> + <listitem> + <para> + A top-level store path. This can be an attribute set + pointing to a store path, like a derivation or a flake + input. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.anything</literal> + </term> + <listitem> + <para> + A type that accepts any value and recursively merges + attribute sets together. This type is recommended when the + option type is unknown. + </para> + <anchor xml:id="ex-types-anything" /> + <para> + <emphasis role="strong">Example: + <literal>types.anything</literal> Example</emphasis> + </para> + <para> + Two definitions of this type like + </para> + <programlisting language="bash"> +{ + str = lib.mkDefault "foo"; + pkg.hello = pkgs.hello; + fun.fun = x: x + 1; +} +</programlisting> + <programlisting language="bash"> +{ + str = lib.mkIf true "bar"; + pkg.gcc = pkgs.gcc; + fun.fun = lib.mkForce (x: x + 2); +} +</programlisting> + <para> + will get merged to + </para> + <programlisting language="bash"> +{ + str = "bar"; + pkg.gcc = pkgs.gcc; + pkg.hello = pkgs.hello; + fun.fun = x: x + 2; +} +</programlisting> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.raw</literal> + </term> + <listitem> + <para> + A type which doesn’t do any checking, merging or nested + evaluation. It accepts a single arbitrary value that is not + recursed into, making it useful for values coming from + outside the module system, such as package sets or arbitrary + data. Options of this type are still evaluated according to + priorities and conditionals, so <literal>mkForce</literal>, + <literal>mkIf</literal> and co. still work on the option + value itself, but not for any value nested within it. This + type should only be used when checking, merging and nested + evaluation are not desirable. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.optionType</literal> + </term> + <listitem> + <para> + The type of an option’s type. Its merging operation ensures + that nested options have the correct file location + annotated, and that if possible, multiple option definitions + are correctly merged together. The main use case is as the + type of the <literal>_module.freeformType</literal> option. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.attrs</literal> + </term> + <listitem> + <para> + A free-form attribute set. + </para> + <warning> + <para> + This type will be deprecated in the future because it + doesn't recurse into attribute sets, silently drops + earlier attribute definitions, and doesn't discharge + <literal>lib.mkDefault</literal>, + <literal>lib.mkIf</literal> and co. For allowing arbitrary + attribute sets, prefer + <literal>types.attrsOf types.anything</literal> instead + which doesn't have these problems. + </para> + </warning> + </listitem> + </varlistentry> + </variablelist> + <para> + Integer-related types: + </para> + <variablelist> + <varlistentry> + <term> + <literal>types.int</literal> + </term> + <listitem> + <para> + A signed integer. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.ints.{s8, s16, s32}</literal> + </term> + <listitem> + <para> + Signed integers with a fixed length (8, 16 or 32 bits). They + go from −2^n/2 to 2^n/2−1 respectively (e.g. + <literal>−128</literal> to <literal>127</literal> for 8 + bits). + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.ints.unsigned</literal> + </term> + <listitem> + <para> + An unsigned integer (that is >= 0). + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.ints.{u8, u16, u32}</literal> + </term> + <listitem> + <para> + Unsigned integers with a fixed length (8, 16 or 32 bits). + They go from 0 to 2^n−1 respectively (e.g. + <literal>0</literal> to <literal>255</literal> for 8 bits). + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.ints.positive</literal> + </term> + <listitem> + <para> + A positive integer (that is > 0). + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.port</literal> + </term> + <listitem> + <para> + A port number. This type is an alias to + <literal>types.ints.u16</literal>. + </para> + </listitem> + </varlistentry> + </variablelist> + <para> + String-related types: + </para> + <variablelist> + <varlistentry> + <term> + <literal>types.str</literal> + </term> + <listitem> + <para> + A string. Multiple definitions cannot be merged. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.lines</literal> + </term> + <listitem> + <para> + A string. Multiple definitions are concatenated with a new + line <literal>"\n"</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.commas</literal> + </term> + <listitem> + <para> + A string. Multiple definitions are concatenated with a comma + <literal>","</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.envVar</literal> + </term> + <listitem> + <para> + A string. Multiple definitions are concatenated with a + collon <literal>":"</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.strMatching</literal> + </term> + <listitem> + <para> + A string matching a specific regular expression. Multiple + definitions cannot be merged. The regular expression is + processed using <literal>builtins.match</literal>. + </para> + </listitem> + </varlistentry> + </variablelist> + </section> + <section xml:id="sec-option-types-value"> + <title>Value Types</title> + <para> + Value types are types that take a value parameter. + </para> + <variablelist> + <varlistentry> + <term> + <literal>types.enum</literal> + <emphasis><literal>l</literal></emphasis> + </term> + <listitem> + <para> + One element of the list + <emphasis><literal>l</literal></emphasis>, e.g. + <literal>types.enum [ "left" "right" ]</literal>. + Multiple definitions cannot be merged. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.separatedString</literal> + <emphasis><literal>sep</literal></emphasis> + </term> + <listitem> + <para> + A string with a custom separator + <emphasis><literal>sep</literal></emphasis>, e.g. + <literal>types.separatedString "|"</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.ints.between</literal> + <emphasis><literal>lowest highest</literal></emphasis> + </term> + <listitem> + <para> + An integer between + <emphasis><literal>lowest</literal></emphasis> and + <emphasis><literal>highest</literal></emphasis> (both + inclusive). Useful for creating types like + <literal>types.port</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.submodule</literal> + <emphasis><literal>o</literal></emphasis> + </term> + <listitem> + <para> + A set of sub options + <emphasis><literal>o</literal></emphasis>. + <emphasis><literal>o</literal></emphasis> can be an + attribute set, a function returning an attribute set, or a + path to a file containing such a value. Submodules are used + in composed types to create modular options. This is + equivalent to + <literal>types.submoduleWith { modules = toList o; shorthandOnlyDefinesConfig = true; }</literal>. + Submodules are detailed in + <link linkend="section-option-types-submodule">Submodule</link>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.submoduleWith</literal> { + <emphasis><literal>modules</literal></emphasis>, + <emphasis><literal>specialArgs</literal></emphasis> ? {}, + <emphasis><literal>shorthandOnlyDefinesConfig</literal></emphasis> + ? false } + </term> + <listitem> + <para> + Like <literal>types.submodule</literal>, but more flexible + and with better defaults. It has parameters + </para> + <itemizedlist> + <listitem> + <para> + <emphasis><literal>modules</literal></emphasis> A list + of modules to use by default for this submodule type. + This gets combined with all option definitions to build + the final list of modules that will be included. + </para> + <note> + <para> + Only options defined with this argument are included + in rendered documentation. + </para> + </note> + </listitem> + <listitem> + <para> + <emphasis><literal>specialArgs</literal></emphasis> An + attribute set of extra arguments to be passed to the + module functions. The option + <literal>_module.args</literal> should be used instead + for most arguments since it allows overriding. + <emphasis><literal>specialArgs</literal></emphasis> + should only be used for arguments that can't go through + the module fixed-point, because of infinite recursion or + other problems. An example is overriding the + <literal>lib</literal> argument, because + <literal>lib</literal> itself is used to define + <literal>_module.args</literal>, which makes using + <literal>_module.args</literal> to define it impossible. + </para> + </listitem> + <listitem> + <para> + <emphasis><literal>shorthandOnlyDefinesConfig</literal></emphasis> + Whether definitions of this type should default to the + <literal>config</literal> section of a module (see + <link linkend="ex-module-syntax">Example: Structure of + NixOS Modules</link>) if it is an attribute set. + Enabling this only has a benefit when the submodule + defines an option named <literal>config</literal> or + <literal>options</literal>. In such a case it would + allow the option to be set with + <literal>the-submodule.config = "value"</literal> + instead of requiring + <literal>the-submodule.config.config = "value"</literal>. + This is because only when modules + <emphasis>don't</emphasis> set the + <literal>config</literal> or <literal>options</literal> + keys, all keys are interpreted as option definitions in + the <literal>config</literal> section. Enabling this + option implicitly puts all attributes in the + <literal>config</literal> section. + </para> + <para> + With this option enabled, defining a + non-<literal>config</literal> section requires using a + function: + <literal>the-submodule = { ... }: { options = { ... }; }</literal>. + </para> + </listitem> + </itemizedlist> + </listitem> + </varlistentry> + </variablelist> + </section> + <section xml:id="sec-option-types-composed"> + <title>Composed Types</title> + <para> + Composed types are types that take a type as parameter. + <literal>listOf int</literal> and + <literal>either int str</literal> are examples of composed types. + </para> + <variablelist> + <varlistentry> + <term> + <literal>types.listOf</literal> + <emphasis><literal>t</literal></emphasis> + </term> + <listitem> + <para> + A list of <emphasis><literal>t</literal></emphasis> type, + e.g. <literal>types.listOf int</literal>. Multiple + definitions are merged with list concatenation. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.attrsOf</literal> + <emphasis><literal>t</literal></emphasis> + </term> + <listitem> + <para> + An attribute set of where all the values are of + <emphasis><literal>t</literal></emphasis> type. Multiple + definitions result in the joined attribute set. + </para> + <note> + <para> + This type is <emphasis>strict</emphasis> in its values, + which in turn means attributes cannot depend on other + attributes. See <literal> types.lazyAttrsOf</literal> for + a lazy version. + </para> + </note> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.lazyAttrsOf</literal> + <emphasis><literal>t</literal></emphasis> + </term> + <listitem> + <para> + An attribute set of where all the values are of + <emphasis><literal>t</literal></emphasis> type. Multiple + definitions result in the joined attribute set. This is the + lazy version of <literal>types.attrsOf </literal>, allowing + attributes to depend on each other. + </para> + <warning> + <para> + This version does not fully support conditional + definitions! With an option <literal>foo</literal> of this + type and a definition + <literal>foo.attr = lib.mkIf false 10</literal>, + evaluating <literal>foo ? attr</literal> will return + <literal>true</literal> even though it should be false. + Accessing the value will then throw an error. For types + <emphasis><literal>t</literal></emphasis> that have an + <literal>emptyValue</literal> defined, that value will be + returned instead of throwing an error. So if the type of + <literal>foo.attr</literal> was + <literal>lazyAttrsOf (nullOr int)</literal>, + <literal>null</literal> would be returned instead for the + same <literal>mkIf false</literal> definition. + </para> + </warning> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.nullOr</literal> + <emphasis><literal>t</literal></emphasis> + </term> + <listitem> + <para> + <literal>null</literal> or type + <emphasis><literal>t</literal></emphasis>. Multiple + definitions are merged according to type + <emphasis><literal>t</literal></emphasis>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.uniq</literal> + <emphasis><literal>t</literal></emphasis> + </term> + <listitem> + <para> + Ensures that type <emphasis><literal>t</literal></emphasis> + cannot be merged. It is used to ensure option definitions + are declared only once. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.unique</literal> + <literal>{ message = m }</literal> + <emphasis><literal>t</literal></emphasis> + </term> + <listitem> + <para> + Ensures that type <emphasis><literal>t</literal></emphasis> + cannot be merged. Prints the message + <emphasis><literal>m</literal></emphasis>, after the line + <literal>The option <option path> is defined multiple times.</literal> + and before a list of definition locations. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.either</literal> + <emphasis><literal>t1 t2</literal></emphasis> + </term> + <listitem> + <para> + Type <emphasis><literal>t1</literal></emphasis> or type + <emphasis><literal>t2</literal></emphasis>, e.g. + <literal>with types; either int str</literal>. Multiple + definitions cannot be merged. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.oneOf</literal> [ + <emphasis><literal>t1 t2</literal></emphasis> ... ] + </term> + <listitem> + <para> + Type <emphasis><literal>t1</literal></emphasis> or type + <emphasis><literal>t2</literal></emphasis> and so forth, + e.g. <literal>with types; oneOf [ int str bool ]</literal>. + Multiple definitions cannot be merged. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>types.coercedTo</literal> + <emphasis><literal>from f to</literal></emphasis> + </term> + <listitem> + <para> + Type <emphasis><literal>to</literal></emphasis> or type + <emphasis><literal>from</literal></emphasis> which will be + coerced to type <emphasis><literal>to</literal></emphasis> + using function <emphasis><literal>f</literal></emphasis> + which takes an argument of type + <emphasis><literal>from</literal></emphasis> and return a + value of type <emphasis><literal>to</literal></emphasis>. + Can be used to preserve backwards compatibility of an option + if its type was changed. + </para> + </listitem> + </varlistentry> + </variablelist> + </section> + <section xml:id="section-option-types-submodule"> + <title>Submodule</title> + <para> + <literal>submodule</literal> is a very powerful type that defines + a set of sub-options that are handled like a separate module. + </para> + <para> + It takes a parameter <emphasis><literal>o</literal></emphasis>, + that should be a set, or a function returning a set with an + <literal>options</literal> key defining the sub-options. Submodule + option definitions are type-checked accordingly to the + <literal>options</literal> declarations. Of course, you can nest + submodule option definitons for even higher modularity. + </para> + <para> + The option set can be defined directly + (<link linkend="ex-submodule-direct">Example: Directly defined + submodule</link>) or as reference + (<link linkend="ex-submodule-reference">Example: Submodule defined + as a reference</link>). + </para> + <anchor xml:id="ex-submodule-direct" /> + <para> + <emphasis role="strong">Example: Directly defined + submodule</emphasis> + </para> + <programlisting language="bash"> +options.mod = mkOption { + description = "submodule example"; + type = with types; submodule { + options = { + foo = mkOption { + type = int; + }; + bar = mkOption { + type = str; + }; + }; + }; +}; +</programlisting> + <anchor xml:id="ex-submodule-reference" /> + <para> + <emphasis role="strong">Example: Submodule defined as a + reference</emphasis> + </para> + <programlisting language="bash"> +let + modOptions = { + options = { + foo = mkOption { + type = int; + }; + bar = mkOption { + type = int; + }; + }; + }; +in +options.mod = mkOption { + description = "submodule example"; + type = with types; submodule modOptions; +}; +</programlisting> + <para> + The <literal>submodule</literal> type is especially interesting + when used with composed types like <literal>attrsOf</literal> or + <literal>listOf</literal>. When composed with + <literal>listOf</literal> + (<link linkend="ex-submodule-listof-declaration">Example: + Declaration of a list of submodules</link>), + <literal>submodule</literal> allows multiple definitions of the + submodule option set + (<link linkend="ex-submodule-listof-definition">Example: + Definition of a list of submodules</link>). + </para> + <anchor xml:id="ex-submodule-listof-declaration" /> + <para> + <emphasis role="strong">Example: Declaration of a list of + submodules</emphasis> + </para> + <programlisting language="bash"> +options.mod = mkOption { + description = "submodule example"; + type = with types; listOf (submodule { + options = { + foo = mkOption { + type = int; + }; + bar = mkOption { + type = str; + }; + }; + }); +}; +</programlisting> + <anchor xml:id="ex-submodule-listof-definition" /> + <para> + <emphasis role="strong">Example: Definition of a list of + submodules</emphasis> + </para> + <programlisting language="bash"> +config.mod = [ + { foo = 1; bar = "one"; } + { foo = 2; bar = "two"; } +]; +</programlisting> + <para> + When composed with <literal>attrsOf</literal> + (<link linkend="ex-submodule-attrsof-declaration">Example: + Declaration of attribute sets of submodules</link>), + <literal>submodule</literal> allows multiple named definitions of + the submodule option set + (<link linkend="ex-submodule-attrsof-definition">Example: + Definition of attribute sets of submodules</link>). + </para> + <anchor xml:id="ex-submodule-attrsof-declaration" /> + <para> + <emphasis role="strong">Example: Declaration of attribute sets of + submodules</emphasis> + </para> + <programlisting language="bash"> +options.mod = mkOption { + description = "submodule example"; + type = with types; attrsOf (submodule { + options = { + foo = mkOption { + type = int; + }; + bar = mkOption { + type = str; + }; + }; + }); +}; +</programlisting> + <anchor xml:id="ex-submodule-attrsof-definition" /> + <para> + <emphasis role="strong">Example: Definition of attribute sets of + submodules</emphasis> + </para> + <programlisting language="bash"> +config.mod.one = { foo = 1; bar = "one"; }; +config.mod.two = { foo = 2; bar = "two"; }; +</programlisting> + </section> + <section xml:id="sec-option-types-extending"> + <title>Extending types</title> + <para> + Types are mainly characterized by their <literal>check</literal> + and <literal>merge</literal> functions. + </para> + <variablelist> + <varlistentry> + <term> + <literal>check</literal> + </term> + <listitem> + <para> + The function to type check the value. Takes a value as + parameter and return a boolean. It is possible to extend a + type check with the <literal>addCheck</literal> function + (<link linkend="ex-extending-type-check-1">Example: Adding a + type check</link>), or to fully override the check function + (<link linkend="ex-extending-type-check-2">Example: + Overriding a type check</link>). + </para> + <anchor xml:id="ex-extending-type-check-1" /> + <para> + <emphasis role="strong">Example: Adding a type + check</emphasis> + </para> + <programlisting language="bash"> +byte = mkOption { + description = "An integer between 0 and 255."; + type = types.addCheck types.int (x: x >= 0 && x <= 255); +}; +</programlisting> + <anchor xml:id="ex-extending-type-check-2" /> + <para> + <emphasis role="strong">Example: Overriding a type + check</emphasis> + </para> + <programlisting language="bash"> +nixThings = mkOption { + description = "words that start with 'nix'"; + type = types.str // { + check = (x: lib.hasPrefix "nix" x) + }; +}; +</programlisting> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>merge</literal> + </term> + <listitem> + <para> + Function to merge the options values when multiple values + are set. The function takes two parameters, + <literal>loc</literal> the option path as a list of strings, + and <literal>defs</literal> the list of defined values as a + list. It is possible to override a type merge function for + custom needs. + </para> + </listitem> + </varlistentry> + </variablelist> + </section> + <section xml:id="sec-option-types-custom"> + <title>Custom Types</title> + <para> + Custom types can be created with the + <literal>mkOptionType</literal> function. As type creation + includes some more complex topics such as submodule handling, it + is recommended to get familiar with <literal>types.nix</literal> + code before creating a new type. + </para> + <para> + The only required parameter is <literal>name</literal>. + </para> + <variablelist> + <varlistentry> + <term> + <literal>name</literal> + </term> + <listitem> + <para> + A string representation of the type function name. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>definition</literal> + </term> + <listitem> + <para> + Description of the type used in documentation. Give + information of the type and any of its arguments. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>check</literal> + </term> + <listitem> + <para> + A function to type check the definition value. Takes the + definition value as a parameter and returns a boolean + indicating the type check result, <literal>true</literal> + for success and <literal>false</literal> for failure. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>merge</literal> + </term> + <listitem> + <para> + A function to merge multiple definitions values. Takes two + parameters: + </para> + <variablelist> + <varlistentry> + <term> + <emphasis><literal>loc</literal></emphasis> + </term> + <listitem> + <para> + The option path as a list of strings, e.g. + <literal>["boot" "loader "grub" "enable"]</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <emphasis><literal>defs</literal></emphasis> + </term> + <listitem> + <para> + The list of sets of defined <literal>value</literal> + and <literal>file</literal> where the value was + defined, e.g. + <literal>[ { file = "/foo.nix"; value = 1; } { file = "/bar.nix"; value = 2 } ]</literal>. + The <literal>merge</literal> function should return + the merged value or throw an error in case the values + are impossible or not meant to be merged. + </para> + </listitem> + </varlistentry> + </variablelist> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>getSubOptions</literal> + </term> + <listitem> + <para> + For composed types that can take a submodule as type + parameter, this function generate sub-options documentation. + It takes the current option prefix as a list and return the + set of sub-options. Usually defined in a recursive manner by + adding a term to the prefix, e.g. + <literal>prefix: elemType.getSubOptions (prefix ++ ["prefix"])</literal> + where + <emphasis><literal>"prefix"</literal></emphasis> + is the newly added prefix. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>getSubModules</literal> + </term> + <listitem> + <para> + For composed types that can take a submodule as type + parameter, this function should return the type parameters + submodules. If the type parameter is called + <literal>elemType</literal>, the function should just + recursively look into submodules by returning + <literal>elemType.getSubModules;</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>substSubModules</literal> + </term> + <listitem> + <para> + For composed types that can take a submodule as type + parameter, this function can be used to substitute the + parameter of a submodule type. It takes a module as + parameter and return the type with the submodule options + substituted. It is usually defined as a type function call + with a recursive call to <literal>substSubModules</literal>, + e.g for a type <literal>composedType</literal> that take an + <literal>elemtype</literal> type parameter, this function + should be defined as + <literal>m: composedType (elemType.substSubModules m)</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>typeMerge</literal> + </term> + <listitem> + <para> + A function to merge multiple type declarations. Takes the + type to merge <literal>functor</literal> as parameter. A + <literal>null</literal> return value means that type cannot + be merged. + </para> + <variablelist> + <varlistentry> + <term> + <emphasis><literal>f</literal></emphasis> + </term> + <listitem> + <para> + The type to merge <literal>functor</literal>. + </para> + </listitem> + </varlistentry> + </variablelist> + <para> + Note: There is a generic <literal>defaultTypeMerge</literal> + that work with most of value and composed types. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>functor</literal> + </term> + <listitem> + <para> + An attribute set representing the type. It is used for type + operations and has the following keys: + </para> + <variablelist> + <varlistentry> + <term> + <literal>type</literal> + </term> + <listitem> + <para> + The type function. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>wrapped</literal> + </term> + <listitem> + <para> + Holds the type parameter for composed types. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>payload</literal> + </term> + <listitem> + <para> + Holds the value parameter for value types. The types + that have a <literal>payload</literal> are the + <literal>enum</literal>, + <literal>separatedString</literal> and + <literal>submodule</literal> types. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>binOp</literal> + </term> + <listitem> + <para> + A binary operation that can merge the payloads of two + same types. Defined as a function that take two + payloads as parameters and return the payloads merged. + </para> + </listitem> + </varlistentry> + </variablelist> + </listitem> + </varlistentry> + </variablelist> + </section> +</section> diff --git a/nixos/doc/manual/from_md/development/replace-modules.section.xml b/nixos/doc/manual/from_md/development/replace-modules.section.xml new file mode 100644 index 00000000000..cf8a39ba844 --- /dev/null +++ b/nixos/doc/manual/from_md/development/replace-modules.section.xml @@ -0,0 +1,70 @@ +<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-replace-modules"> + <title>Replace Modules</title> + <para> + Modules that are imported can also be disabled. The option + declarations, config implementation and the imports of a disabled + module will be ignored, allowing another to take it's place. This + can be used to import a set of modules from another channel while + keeping the rest of the system on a stable release. + </para> + <para> + <literal>disabledModules</literal> is a top level attribute like + <literal>imports</literal>, <literal>options</literal> and + <literal>config</literal>. It contains a list of modules that will + be disabled. This can either be the full path to the module or a + string with the filename relative to the modules path (eg. + <nixpkgs/nixos/modules> for nixos). + </para> + <para> + This example will replace the existing postgresql module with the + version defined in the nixos-unstable channel while keeping the rest + of the modules and packages from the original nixos channel. This + only overrides the module definition, this won't use postgresql from + nixos-unstable unless explicitly configured to do so. + </para> + <programlisting language="bash"> +{ config, lib, pkgs, ... }: + +{ + disabledModules = [ "services/databases/postgresql.nix" ]; + + imports = + [ # Use postgresql service from nixos-unstable channel. + # sudo nix-channel --add https://nixos.org/channels/nixos-unstable nixos-unstable + <nixos-unstable/nixos/modules/services/databases/postgresql.nix> + ]; + + services.postgresql.enable = true; +} +</programlisting> + <para> + This example shows how to define a custom module as a replacement + for an existing module. Importing this module will disable the + original module without having to know it's implementation details. + </para> + <programlisting language="bash"> +{ config, lib, pkgs, ... }: + +with lib; + +let + cfg = config.programs.man; +in + +{ + disabledModules = [ "services/programs/man.nix" ]; + + options = { + programs.man.enable = mkOption { + type = types.bool; + default = true; + description = "Whether to enable manual pages."; + }; + }; + + config = mkIf cfg.enabled { + warnings = [ "disabled manpages for production deployments." ]; + }; +} +</programlisting> +</section> diff --git a/nixos/doc/manual/from_md/development/running-nixos-tests-interactively.section.xml b/nixos/doc/manual/from_md/development/running-nixos-tests-interactively.section.xml new file mode 100644 index 00000000000..0e47350a0d2 --- /dev/null +++ b/nixos/doc/manual/from_md/development/running-nixos-tests-interactively.section.xml @@ -0,0 +1,39 @@ +<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-running-nixos-tests-interactively"> + <title>Running Tests interactively</title> + <para> + The test itself can be run interactively. This is particularly + useful when developing or debugging a test: + </para> + <programlisting> +$ nix-build . -A nixosTests.login.driverInteractive +$ ./result/bin/nixos-test-driver +[...] +>>> +</programlisting> + <para> + You can then take any Python statement, e.g. + </para> + <programlisting language="python"> +>>> start_all() +>>> test_script() +>>> machine.succeed("touch /tmp/foo") +>>> print(machine.succeed("pwd")) # Show stdout of command +</programlisting> + <para> + The function <literal>test_script</literal> executes the entire test + script and drops you back into the test driver command line upon its + completion. This allows you to inspect the state of the VMs after + the test (e.g. to debug the test script). + </para> + <para> + You can re-use the VM states coming from a previous run by setting + the <literal>--keep-vm-state</literal> flag. + </para> + <programlisting> +$ ./result/bin/nixos-test-driver --keep-vm-state +</programlisting> + <para> + The machine state is stored in the + <literal>$TMPDIR/vm-state-machinename</literal> directory. + </para> +</section> diff --git a/nixos/doc/manual/from_md/development/running-nixos-tests.section.xml b/nixos/doc/manual/from_md/development/running-nixos-tests.section.xml new file mode 100644 index 00000000000..da2e5076c95 --- /dev/null +++ b/nixos/doc/manual/from_md/development/running-nixos-tests.section.xml @@ -0,0 +1,34 @@ +<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-running-nixos-tests"> + <title>Running Tests</title> + <para> + You can run tests using <literal>nix-build</literal>. For example, + to run the test + <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix"><literal>login.nix</literal></link>, + you just do: + </para> + <programlisting> +$ nix-build '<nixpkgs/nixos/tests/login.nix>' +</programlisting> + <para> + or, if you don’t want to rely on <literal>NIX_PATH</literal>: + </para> + <programlisting> +$ cd /my/nixpkgs/nixos/tests +$ nix-build login.nix +… +running the VM test script +machine: QEMU running (pid 8841) +… +6 out of 6 tests succeeded +</programlisting> + <para> + After building/downloading all required dependencies, this will + perform a build that starts a QEMU/KVM virtual machine containing a + NixOS system. The virtual machine mounts the Nix store of the host; + this makes VM creation very fast, as no disk image needs to be + created. Afterwards, you can view a log of the test: + </para> + <programlisting> +$ nix-store --read-log result +</programlisting> +</section> diff --git a/nixos/doc/manual/from_md/development/settings-options.section.xml b/nixos/doc/manual/from_md/development/settings-options.section.xml new file mode 100644 index 00000000000..746011a2d07 --- /dev/null +++ b/nixos/doc/manual/from_md/development/settings-options.section.xml @@ -0,0 +1,389 @@ +<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-settings-options"> + <title>Options for Program Settings</title> + <para> + Many programs have configuration files where program-specific + settings can be declared. File formats can be separated into two + categories: + </para> + <itemizedlist> + <listitem> + <para> + Nix-representable ones: These can trivially be mapped to a + subset of Nix syntax. E.g. JSON is an example, since its values + like <literal>{"foo":{"bar":10}}</literal> + can be mapped directly to Nix: + <literal>{ foo = { bar = 10; }; }</literal>. Other examples are + INI, YAML and TOML. The following section explains the + convention for these settings. + </para> + </listitem> + <listitem> + <para> + Non-nix-representable ones: These can't be trivially mapped to a + subset of Nix syntax. Most generic programming languages are in + this group, e.g. bash, since the statement + <literal>if true; then echo hi; fi</literal> doesn't have a + trivial representation in Nix. + </para> + <para> + Currently there are no fixed conventions for these, but it is + common to have a <literal>configFile</literal> option for + setting the configuration file path directly. The default value + of <literal>configFile</literal> can be an auto-generated file, + with convenient options for controlling the contents. For + example an option of type <literal>attrsOf str</literal> can be + used for representing environment variables which generates a + section like <literal>export FOO="foo"</literal>. + Often it can also be useful to also include an + <literal>extraConfig</literal> option of type + <literal>lines</literal> to allow arbitrary text after the + autogenerated part of the file. + </para> + </listitem> + </itemizedlist> + <section xml:id="sec-settings-nix-representable"> + <title>Nix-representable Formats (JSON, YAML, TOML, INI, + ...)</title> + <para> + By convention, formats like this are handled with a generic + <literal>settings</literal> option, representing the full program + configuration as a Nix value. The type of this option should + represent the format. The most common formats have a predefined + type and string generator already declared under + <literal>pkgs.formats</literal>: + </para> + <variablelist> + <varlistentry> + <term> + <literal>pkgs.formats.json</literal> { } + </term> + <listitem> + <para> + A function taking an empty attribute set (for future + extensibility) and returning a set with JSON-specific + attributes <literal>type</literal> and + <literal>generate</literal> as specified + <link linkend="pkgs-formats-result">below</link>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>pkgs.formats.yaml</literal> { } + </term> + <listitem> + <para> + A function taking an empty attribute set (for future + extensibility) and returning a set with YAML-specific + attributes <literal>type</literal> and + <literal>generate</literal> as specified + <link linkend="pkgs-formats-result">below</link>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>pkgs.formats.ini</literal> { + <emphasis><literal>listsAsDuplicateKeys</literal></emphasis> ? + false, <emphasis><literal>listToValue</literal></emphasis> ? + null, ... } + </term> + <listitem> + <para> + A function taking an attribute set with values + </para> + <variablelist> + <varlistentry> + <term> + <literal>listsAsDuplicateKeys</literal> + </term> + <listitem> + <para> + A boolean for controlling whether list values can be + used to represent duplicate INI keys + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>listToValue</literal> + </term> + <listitem> + <para> + A function for turning a list of values into a single + value. + </para> + </listitem> + </varlistentry> + </variablelist> + <para> + It returns a set with INI-specific attributes + <literal>type</literal> and <literal>generate</literal> as + specified <link linkend="pkgs-formats-result">below</link>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>pkgs.formats.toml</literal> { } + </term> + <listitem> + <para> + A function taking an empty attribute set (for future + extensibility) and returning a set with TOML-specific + attributes <literal>type</literal> and + <literal>generate</literal> as specified + <link linkend="pkgs-formats-result">below</link>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>pkgs.formats.elixirConf { elixir ? pkgs.elixir }</literal> + </term> + <listitem> + <para> + A function taking an attribute set with values + </para> + <variablelist> + <varlistentry> + <term> + <literal>elixir</literal> + </term> + <listitem> + <para> + The Elixir package which will be used to format the + generated output + </para> + </listitem> + </varlistentry> + </variablelist> + <para> + It returns a set with Elixir-Config-specific attributes + <literal>type</literal>, <literal>lib</literal>, and + <literal>generate</literal> as specified + <link linkend="pkgs-formats-result">below</link>. + </para> + <para> + The <literal>lib</literal> attribute contains functions to + be used in settings, for generating special Elixir values: + </para> + <variablelist> + <varlistentry> + <term> + <literal>mkRaw elixirCode</literal> + </term> + <listitem> + <para> + Outputs the given string as raw Elixir code + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>mkGetEnv { envVariable, fallback ? null }</literal> + </term> + <listitem> + <para> + Makes the configuration fetch an environment variable + at runtime + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>mkAtom atom</literal> + </term> + <listitem> + <para> + Outputs the given string as an Elixir atom, instead of + the default Elixir binary string. Note: lowercase + atoms still needs to be prefixed with + <literal>:</literal> + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>mkTuple array</literal> + </term> + <listitem> + <para> + Outputs the given array as an Elixir tuple, instead of + the default Elixir list + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>mkMap attrset</literal> + </term> + <listitem> + <para> + Outputs the given attribute set as an Elixir map, + instead of the default Elixir keyword list + </para> + </listitem> + </varlistentry> + </variablelist> + </listitem> + </varlistentry> + </variablelist> + <para xml:id="pkgs-formats-result"> + These functions all return an attribute set with these values: + </para> + <variablelist> + <varlistentry> + <term> + <literal>type</literal> + </term> + <listitem> + <para> + A module system type representing a value of the format + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>lib</literal> + </term> + <listitem> + <para> + Utility functions for convenience, or special interactions + with the format. This attribute is optional. It may contain + inside a <literal>types</literal> attribute containing types + specific to this format. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>generate</literal> + <emphasis><literal>filename jsonValue</literal></emphasis> + </term> + <listitem> + <para> + A function that can render a value of the format to a file. + Returns a file path. + </para> + <note> + <para> + This function puts the value contents in the Nix store. So + this should be avoided for secrets. + </para> + </note> + </listitem> + </varlistentry> + </variablelist> + <anchor xml:id="ex-settings-nix-representable" /> + <para> + <emphasis role="strong">Example: Module with conventional + <literal>settings</literal> option</emphasis> + </para> + <para> + The following shows a module for an example program that uses a + JSON configuration file. It demonstrates how above values can be + used, along with some other related best practices. See the + comments for explanations. + </para> + <programlisting language="bash"> +{ options, config, lib, pkgs, ... }: +let + cfg = config.services.foo; + # Define the settings format used for this program + settingsFormat = pkgs.formats.json {}; +in { + + options.services.foo = { + enable = lib.mkEnableOption "foo service"; + + settings = lib.mkOption { + # Setting this type allows for correct merging behavior + type = settingsFormat.type; + default = {}; + description = '' + Configuration for foo, see + <link xlink:href="https://example.com/docs/foo"/> + for supported settings. + ''; + }; + }; + + config = lib.mkIf cfg.enable { + # We can assign some default settings here to make the service work by just + # enabling it. We use `mkDefault` for values that can be changed without + # problems + services.foo.settings = { + # Fails at runtime without any value set + log_level = lib.mkDefault "WARN"; + + # We assume systemd's `StateDirectory` is used, so we require this value, + # therefore no mkDefault + data_path = "/var/lib/foo"; + + # Since we use this to create a user we need to know the default value at + # eval time + user = lib.mkDefault "foo"; + }; + + environment.etc."foo.json".source = + # The formats generator function takes a filename and the Nix value + # representing the format value and produces a filepath with that value + # rendered in the format + settingsFormat.generate "foo-config.json" cfg.settings; + + # We know that the `user` attribute exists because we set a default value + # for it above, allowing us to use it without worries here + users.users.${cfg.settings.user} = { isSystemUser = true; }; + + # ... + }; +} +</programlisting> + <section xml:id="sec-settings-attrs-options"> + <title>Option declarations for attributes</title> + <para> + Some <literal>settings</literal> attributes may deserve some + extra care. They may need a different type, default or merging + behavior, or they are essential options that should show their + documentation in the manual. This can be done using + <xref linkend="sec-freeform-modules" />. + </para> + <para> + We extend above example using freeform modules to declare an + option for the port, which will enforce it to be a valid integer + and make it show up in the manual. + </para> + <anchor xml:id="ex-settings-typed-attrs" /> + <para> + <emphasis role="strong">Example: Declaring a type-checked + <literal>settings</literal> attribute</emphasis> + </para> + <programlisting language="bash"> +settings = lib.mkOption { + type = lib.types.submodule { + + freeformType = settingsFormat.type; + + # Declare an option for the port such that the type is checked and this option + # is shown in the manual. + options.port = lib.mkOption { + type = lib.types.port; + default = 8080; + description = '' + Which port this service should listen on. + ''; + }; + + }; + default = {}; + description = '' + Configuration for Foo, see + <link xlink:href="https://example.com/docs/foo"/> + for supported values. + ''; +}; +</programlisting> + </section> + </section> +</section> diff --git a/nixos/doc/manual/from_md/development/sources.chapter.xml b/nixos/doc/manual/from_md/development/sources.chapter.xml new file mode 100644 index 00000000000..aac18c9d06c --- /dev/null +++ b/nixos/doc/manual/from_md/development/sources.chapter.xml @@ -0,0 +1,90 @@ +<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-getting-sources"> + <title>Getting the Sources</title> + <para> + By default, NixOS’s <literal>nixos-rebuild</literal> command uses + the NixOS and Nixpkgs sources provided by the + <literal>nixos</literal> channel (kept in + <literal>/nix/var/nix/profiles/per-user/root/channels/nixos</literal>). + To modify NixOS, however, you should check out the latest sources + from Git. This is as follows: + </para> + <programlisting> +$ git clone https://github.com/NixOS/nixpkgs +$ cd nixpkgs +$ git remote update origin +</programlisting> + <para> + This will check out the latest Nixpkgs sources to + <literal>./nixpkgs</literal> the NixOS sources to + <literal>./nixpkgs/nixos</literal>. (The NixOS source tree lives in + a subdirectory of the Nixpkgs repository.) The + <literal>nixpkgs</literal> repository has branches that correspond + to each Nixpkgs/NixOS channel (see <xref linkend="sec-upgrading" /> + for more information about channels). Thus, the Git branch + <literal>origin/nixos-17.03</literal> will contain the latest built + and tested version available in the <literal>nixos-17.03</literal> + channel. + </para> + <para> + It’s often inconvenient to develop directly on the master branch, + since if somebody has just committed (say) a change to GCC, then the + binary cache may not have caught up yet and you’ll have to rebuild + everything from source. So you may want to create a local branch + based on your current NixOS version: + </para> + <programlisting> +$ nixos-version +17.09pre104379.6e0b727 (Hummingbird) + +$ git checkout -b local 6e0b727 +</programlisting> + <para> + Or, to base your local branch on the latest version available in a + NixOS channel: + </para> + <programlisting> +$ git remote update origin +$ git checkout -b local origin/nixos-17.03 +</programlisting> + <para> + (Replace <literal>nixos-17.03</literal> with the name of the channel + you want to use.) You can use <literal>git merge</literal> or + <literal>git rebase</literal> to keep your local branch in sync with + the channel, e.g. + </para> + <programlisting> +$ git remote update origin +$ git merge origin/nixos-17.03 +</programlisting> + <para> + You can use <literal>git cherry-pick</literal> to copy commits from + your local branch to the upstream branch. + </para> + <para> + If you want to rebuild your system using your (modified) sources, + you need to tell <literal>nixos-rebuild</literal> about them using + the <literal>-I</literal> flag: + </para> + <programlisting> +# nixos-rebuild switch -I nixpkgs=/my/sources/nixpkgs +</programlisting> + <para> + If you want <literal>nix-env</literal> to use the expressions in + <literal>/my/sources</literal>, use + <literal>nix-env -f /my/sources/nixpkgs</literal>, or change the + default by adding a symlink in <literal>~/.nix-defexpr</literal>: + </para> + <programlisting> +$ ln -s /my/sources/nixpkgs ~/.nix-defexpr/nixpkgs +</programlisting> + <para> + You may want to delete the symlink + <literal>~/.nix-defexpr/channels_root</literal> to prevent root’s + NixOS channel from clashing with your own tree (this may break the + command-not-found utility though). If you want to go back to the + default state, you may just remove the + <literal>~/.nix-defexpr</literal> directory completely, log out and + log in again and it should have been recreated with a link to the + root channels. + </para> +</chapter> diff --git a/nixos/doc/manual/from_md/development/testing-installer.chapter.xml b/nixos/doc/manual/from_md/development/testing-installer.chapter.xml new file mode 100644 index 00000000000..286d49f3c29 --- /dev/null +++ b/nixos/doc/manual/from_md/development/testing-installer.chapter.xml @@ -0,0 +1,22 @@ +<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="ch-testing-installer"> + <title>Testing the Installer</title> + <para> + Building, burning, and booting from an installation CD is rather + tedious, so here is a quick way to see if the installer works + properly: + </para> + <programlisting> +# mount -t tmpfs none /mnt +# nixos-generate-config --root /mnt +$ nix-build '<nixpkgs/nixos>' -A config.system.build.nixos-install +# ./result/bin/nixos-install +</programlisting> + <para> + To start a login shell in the new NixOS installation in + <literal>/mnt</literal>: + </para> + <programlisting> +$ nix-build '<nixpkgs/nixos>' -A config.system.build.nixos-enter +# ./result/bin/nixos-enter +</programlisting> +</chapter> diff --git a/nixos/doc/manual/from_md/development/unit-handling.section.xml b/nixos/doc/manual/from_md/development/unit-handling.section.xml new file mode 100644 index 00000000000..4c980e1213a --- /dev/null +++ b/nixos/doc/manual/from_md/development/unit-handling.section.xml @@ -0,0 +1,131 @@ +<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-unit-handling"> + <title>Unit handling</title> + <para> + To figure out what units need to be + started/stopped/restarted/reloaded, the script first checks the + current state of the system, similar to what + <literal>systemctl list-units</literal> shows. For each of the + units, the script goes through the following checks: + </para> + <itemizedlist> + <listitem> + <para> + Is the unit file still in the new system? If not, + <emphasis role="strong">stop</emphasis> the service unless it + sets <literal>X-StopOnRemoval</literal> in the + <literal>[Unit]</literal> section to <literal>false</literal>. + </para> + </listitem> + <listitem> + <para> + Is it a <literal>.target</literal> unit? If so, + <emphasis role="strong">start</emphasis> it unless it sets + <literal>RefuseManualStart</literal> in the + <literal>[Unit]</literal> section to <literal>true</literal> or + <literal>X-OnlyManualStart</literal> in the + <literal>[Unit]</literal> section to <literal>true</literal>. + Also <emphasis role="strong">stop</emphasis> the unit again + unless it sets <literal>X-StopOnReconfiguration</literal> to + <literal>false</literal>. + </para> + </listitem> + <listitem> + <para> + Are the contents of the unit files different? They are compared + by parsing them and comparing their contents. If they are + different but only <literal>X-Reload-Triggers</literal> in the + <literal>[Unit]</literal> section is changed, + <emphasis role="strong">reload</emphasis> the unit. The NixOS + module system allows setting these triggers with the option + <link linkend="opt-systemd.services">systemd.services.<name>.reloadTriggers</link>. + There are some additional keys in the <literal>[Unit]</literal> + section that are ignored as well. If the unit files differ in + any way, the following actions are performed: + </para> + <itemizedlist> + <listitem> + <para> + <literal>.path</literal> and <literal>.slice</literal> units + are ignored. There is no need to restart them since changes + in their values are applied by systemd when systemd is + reloaded. + </para> + </listitem> + <listitem> + <para> + <literal>.mount</literal> units are + <emphasis role="strong">reload</emphasis>ed. These mostly + come from the <literal>/etc/fstab</literal> parser. + </para> + </listitem> + <listitem> + <para> + <literal>.socket</literal> units are currently ignored. This + is to be fixed at a later point. + </para> + </listitem> + <listitem> + <para> + The rest of the units (mostly <literal>.service</literal> + units) are then <emphasis role="strong">reload</emphasis>ed + if <literal>X-ReloadIfChanged</literal> in the + <literal>[Service]</literal> section is set to + <literal>true</literal> (exposed via + <link linkend="opt-systemd.services">systemd.services.<name>.reloadIfChanged</link>). + A little exception is done for units that were deactivated + in the meantime, for example because they require a unit + that got stopped before. These are + <emphasis role="strong">start</emphasis>ed instead of + reloaded. + </para> + </listitem> + <listitem> + <para> + If the reload flag is not set, some more flags decide if the + unit is skipped. These flags are + <literal>X-RestartIfChanged</literal> in the + <literal>[Service]</literal> section (exposed via + <link linkend="opt-systemd.services">systemd.services.<name>.restartIfChanged</link>), + <literal>RefuseManualStop</literal> in the + <literal>[Unit]</literal> section, and + <literal>X-OnlyManualStart</literal> in the + <literal>[Unit]</literal> section. + </para> + </listitem> + <listitem> + <para> + Further behavior depends on the unit having + <literal>X-StopIfChanged</literal> in the + <literal>[Service]</literal> section set to + <literal>true</literal> (exposed via + <link linkend="opt-systemd.services">systemd.services.<name>.stopIfChanged</link>). + This is set to <literal>true</literal> by default and must + be explicitly turned off if not wanted. If the flag is + enabled, the unit is + <emphasis role="strong">stop</emphasis>ped and then + <emphasis role="strong">start</emphasis>ed. If not, the unit + is <emphasis role="strong">restart</emphasis>ed. The goal of + the flag is to make sure that the new unit never runs in the + old environment which is still in place before the + activation script is run. This behavior is different when + the service is socket-activated, as outlined in the + following steps. + </para> + </listitem> + <listitem> + <para> + The last thing that is taken into account is whether the + unit is a service and socket-activated. If + <literal>X-StopIfChanged</literal> is + <emphasis role="strong">not</emphasis> set, the service is + <emphasis role="strong">restart</emphasis>ed with the + others. If it is set, both the service and the socket are + <emphasis role="strong">stop</emphasis>ped and the socket is + <emphasis role="strong">start</emphasis>ed, leaving socket + activation to start the service when it’s needed. + </para> + </listitem> + </itemizedlist> + </listitem> + </itemizedlist> +</section> diff --git a/nixos/doc/manual/from_md/development/what-happens-during-a-system-switch.chapter.xml b/nixos/doc/manual/from_md/development/what-happens-during-a-system-switch.chapter.xml new file mode 100644 index 00000000000..66ba792ddac --- /dev/null +++ b/nixos/doc/manual/from_md/development/what-happens-during-a-system-switch.chapter.xml @@ -0,0 +1,122 @@ +<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xi="http://www.w3.org/2001/XInclude" xml:id="sec-switching-systems"> + <title>What happens during a system switch?</title> + <para> + Running <literal>nixos-rebuild switch</literal> is one of the more + common tasks under NixOS. This chapter explains some of the + internals of this command to make it simpler for new module + developers to configure their units correctly and to make it easier + to understand what is happening and why for curious administrators. + </para> + <para> + <literal>nixos-rebuild</literal>, like many deployment solutions, + calls <literal>switch-to-configuration</literal> which resides in a + NixOS system at <literal>$out/bin/switch-to-configuration</literal>. + The script is called with the action that is to be performed like + <literal>switch</literal>, <literal>test</literal>, + <literal>boot</literal>. There is also the + <literal>dry-activate</literal> action which does not really perform + the actions but rather prints what it would do if you called it with + <literal>test</literal>. This feature can be used to check what + service states would be changed if the configuration was switched + to. + </para> + <para> + If the action is <literal>switch</literal> or + <literal>boot</literal>, the bootloader is updated first so the + configuration will be the next one to boot. Unless + <literal>NIXOS_NO_SYNC</literal> is set to <literal>1</literal>, + <literal>/nix/store</literal> is synced to disk. + </para> + <para> + If the action is <literal>switch</literal> or + <literal>test</literal>, the currently running system is inspected + and the actions to switch to the new system are calculated. This + process takes two data sources into account: + <literal>/etc/fstab</literal> and the current systemd status. Mounts + and swaps are read from <literal>/etc/fstab</literal> and the + corresponding actions are generated. If a new mount is added, for + example, the proper <literal>.mount</literal> unit is marked to be + started. The current systemd state is inspected, the difference + between the current system and the desired configuration is + calculated and actions are generated to get to this state. There are + a lot of nuances that can be controlled by the units which are + explained here. + </para> + <para> + After calculating what should be done, the actions are carried out. + The order of actions is always the same: + </para> + <itemizedlist spacing="compact"> + <listitem> + <para> + Stop units (<literal>systemctl stop</literal>) + </para> + </listitem> + <listitem> + <para> + Run activation script (<literal>$out/activate</literal>) + </para> + </listitem> + <listitem> + <para> + See if the activation script requested more units to restart + </para> + </listitem> + <listitem> + <para> + Restart systemd if needed + (<literal>systemd daemon-reexec</literal>) + </para> + </listitem> + <listitem> + <para> + Forget about the failed state of units + (<literal>systemctl reset-failed</literal>) + </para> + </listitem> + <listitem> + <para> + Reload systemd (<literal>systemctl daemon-reload</literal>) + </para> + </listitem> + <listitem> + <para> + Reload systemd user instances + (<literal>systemctl --user daemon-reload</literal>) + </para> + </listitem> + <listitem> + <para> + Set up tmpfiles (<literal>systemd-tmpfiles --create</literal>) + </para> + </listitem> + <listitem> + <para> + Reload units (<literal>systemctl reload</literal>) + </para> + </listitem> + <listitem> + <para> + Restart units (<literal>systemctl restart</literal>) + </para> + </listitem> + <listitem> + <para> + Start units (<literal>systemctl start</literal>) + </para> + </listitem> + <listitem> + <para> + Inspect what changed during these actions and print units that + failed and that were newly started + </para> + </listitem> + </itemizedlist> + <para> + Most of these actions are either self-explaining but some of them + have to do with our units or the activation script. For this reason, + these topics are explained in the next sections. + </para> + <xi:include href="unit-handling.section.xml" /> + <xi:include href="activation-script.section.xml" /> +</chapter> diff --git a/nixos/doc/manual/from_md/development/writing-documentation.chapter.xml b/nixos/doc/manual/from_md/development/writing-documentation.chapter.xml new file mode 100644 index 00000000000..079c8006057 --- /dev/null +++ b/nixos/doc/manual/from_md/development/writing-documentation.chapter.xml @@ -0,0 +1,144 @@ +<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xi="http://www.w3.org/2001/XInclude" xml:id="sec-writing-documentation"> + <title>Writing NixOS Documentation</title> + <para> + As NixOS grows, so too does the need for a catalogue and explanation + of its extensive functionality. Collecting pertinent information + from disparate sources and presenting it in an accessible style + would be a worthy contribution to the project. + </para> + <section xml:id="sec-writing-docs-building-the-manual"> + <title>Building the Manual</title> + <para> + The DocBook sources of the <xref linkend="book-nixos-manual" /> + are in the + <link xlink:href="https://github.com/NixOS/nixpkgs/tree/master/nixos/doc/manual"><literal>nixos/doc/manual</literal></link> + subdirectory of the Nixpkgs repository. + </para> + <para> + You can quickly validate your edits with <literal>make</literal>: + </para> + <programlisting> +$ cd /path/to/nixpkgs/nixos/doc/manual +$ nix-shell +nix-shell$ make +</programlisting> + <para> + Once you are done making modifications to the manual, it's + important to build it before committing. You can do that as + follows: + </para> + <programlisting> +nix-build nixos/release.nix -A manual.x86_64-linux +</programlisting> + <para> + When this command successfully finishes, it will tell you where + the manual got generated. The HTML will be accessible through the + <literal>result</literal> symlink at + <literal>./result/share/doc/nixos/index.html</literal>. + </para> + </section> + <section xml:id="sec-writing-docs-editing-docbook-xml"> + <title>Editing DocBook XML</title> + <para> + For general information on how to write in DocBook, see + <link xlink:href="http://www.docbook.org/tdg5/en/html/docbook.html">DocBook + 5: The Definitive Guide</link>. + </para> + <para> + Emacs nXML Mode is very helpful for editing DocBook XML because it + validates the document as you write, and precisely locates errors. + To use it, see <xref linkend="sec-emacs-docbook-xml" />. + </para> + <para> + <link xlink:href="http://pandoc.org">Pandoc</link> can generate + DocBook XML from a multitude of formats, which makes a good + starting point. Here is an example of Pandoc invocation to convert + GitHub-Flavoured MarkDown to DocBook 5 XML: + </para> + <programlisting> +pandoc -f markdown_github -t docbook5 docs.md -o my-section.md +</programlisting> + <para> + Pandoc can also quickly convert a single + <literal>section.xml</literal> to HTML, which is helpful when + drafting. + </para> + <para> + Sometimes writing valid DocBook is simply too difficult. In this + case, submit your documentation updates in a + <link xlink:href="https://github.com/NixOS/nixpkgs/issues/new">GitHub + Issue</link> and someone will handle the conversion to XML for + you. + </para> + </section> + <section xml:id="sec-writing-docs-creating-a-topic"> + <title>Creating a Topic</title> + <para> + You can use an existing topic as a basis for the new topic or + create a topic from scratch. + </para> + <para> + Keep the following guidelines in mind when you create and add a + topic: + </para> + <itemizedlist> + <listitem> + <para> + The NixOS + <link xlink:href="http://www.docbook.org/tdg5/en/html/book.html"><literal>book</literal></link> + element is in <literal>nixos/doc/manual/manual.xml</literal>. + It includes several + <link xlink:href="http://www.docbook.org/tdg5/en/html/book.html"><literal>parts</literal></link> + which are in subdirectories. + </para> + </listitem> + <listitem> + <para> + Store the topic file in the same directory as the + <literal>part</literal> to which it belongs. If your topic is + about configuring a NixOS module, then the XML file can be + stored alongside the module definition <literal>nix</literal> + file. + </para> + </listitem> + <listitem> + <para> + If you include multiple words in the file name, separate the + words with a dash. For example: + <literal>ipv6-config.xml</literal>. + </para> + </listitem> + <listitem> + <para> + Make sure that the <literal>xml:id</literal> value is unique. + You can use abbreviations if the ID is too long. For example: + <literal>nixos-config</literal>. + </para> + </listitem> + <listitem> + <para> + Determine whether your topic is a chapter or a section. If you + are unsure, open an existing topic file and check whether the + main element is chapter or section. + </para> + </listitem> + </itemizedlist> + </section> + <section xml:id="sec-writing-docs-adding-a-topic"> + <title>Adding a Topic to the Book</title> + <para> + Open the parent XML file and add an <literal>xi:include</literal> + element to the list of chapters with the file name of the topic + that you created. If you created a <literal>section</literal>, you + add the file to the <literal>chapter</literal> file. If you + created a <literal>chapter</literal>, you add the file to the + <literal>part</literal> file. + </para> + <para> + If the topic is about configuring a NixOS module, it can be + automatically included in the manual by using the + <literal>meta.doc</literal> attribute. See + <xref linkend="sec-meta-attributes" /> for an explanation. + </para> + </section> +</chapter> diff --git a/nixos/doc/manual/from_md/development/writing-modules.chapter.xml b/nixos/doc/manual/from_md/development/writing-modules.chapter.xml new file mode 100644 index 00000000000..367731eda09 --- /dev/null +++ b/nixos/doc/manual/from_md/development/writing-modules.chapter.xml @@ -0,0 +1,245 @@ +<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xi="http://www.w3.org/2001/XInclude" xml:id="sec-writing-modules"> + <title>Writing NixOS Modules</title> + <para> + NixOS has a modular system for declarative configuration. This + system combines multiple <emphasis>modules</emphasis> to produce the + full system configuration. One of the modules that constitute the + configuration is <literal>/etc/nixos/configuration.nix</literal>. + Most of the others live in the + <link xlink:href="https://github.com/NixOS/nixpkgs/tree/master/nixos/modules"><literal>nixos/modules</literal></link> + subdirectory of the Nixpkgs tree. + </para> + <para> + Each NixOS module is a file that handles one logical aspect of the + configuration, such as a specific kind of hardware, a service, or + network settings. A module configuration does not have to handle + everything from scratch; it can use the functionality provided by + other modules for its implementation. Thus a module can + <emphasis>declare</emphasis> options that can be used by other + modules, and conversely can <emphasis>define</emphasis> options + provided by other modules in its own implementation. For example, + the module + <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/security/pam.nix"><literal>pam.nix</literal></link> + declares the option <literal>security.pam.services</literal> that + allows other modules (e.g. + <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/networking/ssh/sshd.nix"><literal>sshd.nix</literal></link>) + to define PAM services; and it defines the option + <literal>environment.etc</literal> (declared by + <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/system/etc/etc.nix"><literal>etc.nix</literal></link>) + to cause files to be created in <literal>/etc/pam.d</literal>. + </para> + <para> + In <xref linkend="sec-configuration-syntax" />, we saw the following + structure of NixOS modules: + </para> + <programlisting language="bash"> +{ config, pkgs, ... }: + +{ option definitions +} +</programlisting> + <para> + This is actually an <emphasis>abbreviated</emphasis> form of module + that only defines options, but does not declare any. The structure + of full NixOS modules is shown in + <link linkend="ex-module-syntax">Example: Structure of NixOS + Modules</link>. + </para> + <anchor xml:id="ex-module-syntax" /> + <para> + <emphasis role="strong">Example: Structure of NixOS + Modules</emphasis> + </para> + <programlisting language="bash"> +{ config, pkgs, ... }: + +{ + imports = + [ paths of other modules + ]; + + options = { + option declarations + }; + + config = { + option definitions + }; +} +</programlisting> + <para> + The meaning of each part is as follows. + </para> + <itemizedlist> + <listitem> + <para> + The first line makes the current Nix expression a function. The + variable <literal>pkgs</literal> contains Nixpkgs (by default, + it takes the <literal>nixpkgs</literal> entry of + <literal>NIX_PATH</literal>, see the + <link xlink:href="https://nixos.org/manual/nix/stable/#sec-common-env">Nix + manual</link> for further details), while + <literal>config</literal> contains the full system + configuration. This line can be omitted if there is no reference + to <literal>pkgs</literal> and <literal>config</literal> inside + the module. + </para> + </listitem> + <listitem> + <para> + This <literal>imports</literal> list enumerates the paths to + other NixOS modules that should be included in the evaluation of + the system configuration. A default set of modules is defined in + the file <literal>modules/module-list.nix</literal>. These don't + need to be added in the import list. + </para> + </listitem> + <listitem> + <para> + The attribute <literal>options</literal> is a nested set of + <emphasis>option declarations</emphasis> (described below). + </para> + </listitem> + <listitem> + <para> + The attribute <literal>config</literal> is a nested set of + <emphasis>option definitions</emphasis> (also described below). + </para> + </listitem> + </itemizedlist> + <para> + <link linkend="locate-example">Example: NixOS Module for the + <quote>locate</quote> Service</link> shows a module that handles the + regular update of the <quote>locate</quote> database, an index of + all files in the file system. This module declares two options that + can be defined by other modules (typically the user’s + <literal>configuration.nix</literal>): + <literal>services.locate.enable</literal> (whether the database + should be updated) and <literal>services.locate.interval</literal> + (when the update should be done). It implements its functionality by + defining two options declared by other modules: + <literal>systemd.services</literal> (the set of all systemd + services) and <literal>systemd.timers</literal> (the list of + commands to be executed periodically by <literal>systemd</literal>). + </para> + <para> + Care must be taken when writing systemd services using + <literal>Exec*</literal> directives. By default systemd performs + substitution on <literal>%<char></literal> specifiers in these + directives, expands environment variables from + <literal>$FOO</literal> and <literal>${FOO}</literal>, splits + arguments on whitespace, and splits commands on + <literal>;</literal>. All of these must be escaped to avoid + unexpected substitution or splitting when interpolating into an + <literal>Exec*</literal> directive, e.g. when using an + <literal>extraArgs</literal> option to pass additional arguments to + the service. The functions + <literal>utils.escapeSystemdExecArg</literal> and + <literal>utils.escapeSystemdExecArgs</literal> are provided for + this, see <link linkend="exec-escaping-example">Example: Escaping in + Exec directives</link> for an example. When using these functions + system environment substitution should <emphasis>not</emphasis> be + disabled explicitly. + </para> + <anchor xml:id="locate-example" /> + <para> + <emphasis role="strong">Example: NixOS Module for the + <quote>locate</quote> Service</emphasis> + </para> + <programlisting language="bash"> +{ config, lib, pkgs, ... }: + +with lib; + +let + cfg = config.services.locate; +in { + options.services.locate = { + enable = mkOption { + type = types.bool; + default = false; + description = '' + If enabled, NixOS will periodically update the database of + files used by the locate command. + ''; + }; + + interval = mkOption { + type = types.str; + default = "02:15"; + example = "hourly"; + description = '' + Update the locate database at this interval. Updates by + default at 2:15 AM every day. + + The format is described in + systemd.time(7). + ''; + }; + + # Other options omitted for documentation + }; + + config = { + systemd.services.update-locatedb = + { description = "Update Locate Database"; + path = [ pkgs.su ]; + script = + '' + mkdir -m 0755 -p $(dirname ${toString cfg.output}) + exec updatedb \ + --localuser=${cfg.localuser} \ + ${optionalString (!cfg.includeStore) "--prunepaths='/nix/store'"} \ + --output=${toString cfg.output} ${concatStringsSep " " cfg.extraFlags} + ''; + }; + + systemd.timers.update-locatedb = mkIf cfg.enable + { description = "Update timer for locate database"; + partOf = [ "update-locatedb.service" ]; + wantedBy = [ "timers.target" ]; + timerConfig.OnCalendar = cfg.interval; + }; + }; +} +</programlisting> + <anchor xml:id="exec-escaping-example" /> + <para> + <emphasis role="strong">Example: Escaping in Exec + directives</emphasis> + </para> + <programlisting language="bash"> +{ config, lib, pkgs, utils, ... }: + +with lib; + +let + cfg = config.services.echo; + echoAll = pkgs.writeScript "echo-all" '' + #! ${pkgs.runtimeShell} + for s in "$@"; do + printf '%s\n' "$s" + done + ''; + args = [ "a%Nything" "lang=\${LANG}" ";" "/bin/sh -c date" ]; +in { + systemd.services.echo = + { description = "Echo to the journal"; + wantedBy = [ "multi-user.target" ]; + serviceConfig.Type = "oneshot"; + serviceConfig.ExecStart = '' + ${echoAll} ${utils.escapeSystemdExecArgs args} + ''; + }; +} +</programlisting> + <xi:include href="option-declarations.section.xml" /> + <xi:include href="option-types.section.xml" /> + <xi:include href="option-def.section.xml" /> + <xi:include href="assertions.section.xml" /> + <xi:include href="meta-attributes.section.xml" /> + <xi:include href="importing-modules.section.xml" /> + <xi:include href="replace-modules.section.xml" /> + <xi:include href="freeform-modules.section.xml" /> + <xi:include href="settings-options.section.xml" /> +</chapter> diff --git a/nixos/doc/manual/from_md/development/writing-nixos-tests.section.xml b/nixos/doc/manual/from_md/development/writing-nixos-tests.section.xml new file mode 100644 index 00000000000..45c9c40c609 --- /dev/null +++ b/nixos/doc/manual/from_md/development/writing-nixos-tests.section.xml @@ -0,0 +1,616 @@ +<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-writing-nixos-tests"> + <title>Writing Tests</title> + <para> + A NixOS test is a Nix expression that has the following structure: + </para> + <programlisting language="bash"> +import ./make-test-python.nix { + + # Either the configuration of a single machine: + machine = + { config, pkgs, ... }: + { configuration… + }; + + # Or a set of machines: + nodes = + { machine1 = + { config, pkgs, ... }: { … }; + machine2 = + { config, pkgs, ... }: { … }; + … + }; + + testScript = + '' + Python code… + ''; +} +</programlisting> + <para> + The attribute <literal>testScript</literal> is a bit of Python code + that executes the test (described below). During the test, it will + start one or more virtual machines, the configuration of which is + described by the attribute <literal>machine</literal> (if you need + only one machine in your test) or by the attribute + <literal>nodes</literal> (if you need multiple machines). For + instance, + <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix"><literal>login.nix</literal></link> + only needs a single machine to test whether users can log in on the + virtual console, whether device ownership is correctly maintained + when switching between consoles, and so on. On the other hand, + <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nfs/simple.nix"><literal>nfs/simple.nix</literal></link>, + which tests NFS client and server functionality in the Linux kernel + (including whether locks are maintained across server crashes), + requires three machines: a server and two clients. + </para> + <para> + There are a few special NixOS configuration options for test VMs: + </para> + <variablelist> + <varlistentry> + <term> + <literal>virtualisation.memorySize</literal> + </term> + <listitem> + <para> + The memory of the VM in megabytes. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>virtualisation.vlans</literal> + </term> + <listitem> + <para> + The virtual networks to which the VM is connected. See + <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nat.nix"><literal>nat.nix</literal></link> + for an example. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>virtualisation.writableStore</literal> + </term> + <listitem> + <para> + By default, the Nix store in the VM is not writable. If you + enable this option, a writable union file system is mounted on + top of the Nix store to make it appear writable. This is + necessary for tests that run Nix operations that modify the + store. + </para> + </listitem> + </varlistentry> + </variablelist> + <para> + For more options, see the module + <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix"><literal>qemu-vm.nix</literal></link>. + </para> + <para> + The test script is a sequence of Python statements that perform + various actions, such as starting VMs, executing commands in the + VMs, and so on. Each virtual machine is represented as an object + stored in the variable <literal>name</literal> if this is also the + identifier of the machine in the declarative config. If you didn't + specify multiple machines using the <literal>nodes</literal> + attribute, it is just <literal>machine</literal>. The following + example starts the machine, waits until it has finished booting, + then executes a command and checks that the output is more-or-less + correct: + </para> + <programlisting language="python"> +machine.start() +machine.wait_for_unit("default.target") +if not "Linux" in machine.succeed("uname"): + raise Exception("Wrong OS") +</programlisting> + <para> + The first line is actually unnecessary; machines are implicitly + started when you first execute an action on them (such as + <literal>wait_for_unit</literal> or <literal>succeed</literal>). If + you have multiple machines, you can speed up the test by starting + them in parallel: + </para> + <programlisting language="python"> +start_all() +</programlisting> + <section xml:id="ssec-machine-objects"> + <title>Machine objects</title> + <para> + The following methods are available on machine objects: + </para> + <variablelist> + <varlistentry> + <term> + <literal>start</literal> + </term> + <listitem> + <para> + Start the virtual machine. This method is asynchronous — it + does not wait for the machine to finish booting. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>shutdown</literal> + </term> + <listitem> + <para> + Shut down the machine, waiting for the VM to exit. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>crash</literal> + </term> + <listitem> + <para> + Simulate a sudden power failure, by telling the VM to exit + immediately. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>block</literal> + </term> + <listitem> + <para> + Simulate unplugging the Ethernet cable that connects the + machine to the other machines. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>unblock</literal> + </term> + <listitem> + <para> + Undo the effect of <literal>block</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>screenshot</literal> + </term> + <listitem> + <para> + Take a picture of the display of the virtual machine, in PNG + format. The screenshot is linked from the HTML log. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>get_screen_text_variants</literal> + </term> + <listitem> + <para> + Return a list of different interpretations of what is + currently visible on the machine's screen using optical + character recognition. The number and order of the + interpretations is not specified and is subject to change, + but if no exception is raised at least one will be returned. + </para> + <note> + <para> + This requires passing <literal>enableOCR</literal> to the + test attribute set. + </para> + </note> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>get_screen_text</literal> + </term> + <listitem> + <para> + Return a textual representation of what is currently visible + on the machine's screen using optical character recognition. + </para> + <note> + <para> + This requires passing <literal>enableOCR</literal> to the + test attribute set. + </para> + </note> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>send_monitor_command</literal> + </term> + <listitem> + <para> + Send a command to the QEMU monitor. This is rarely used, but + allows doing stuff such as attaching virtual USB disks to a + running machine. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>send_key</literal> + </term> + <listitem> + <para> + Simulate pressing keys on the virtual keyboard, e.g., + <literal>send_key("ctrl-alt-delete")</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>send_chars</literal> + </term> + <listitem> + <para> + Simulate typing a sequence of characters on the virtual + keyboard, e.g., + <literal>send_chars("foobar\n")</literal> will + type the string <literal>foobar</literal> followed by the + Enter key. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>execute</literal> + </term> + <listitem> + <para> + Execute a shell command, returning a list + <literal>(status, stdout)</literal>. If the command + detaches, it must close stdout, as + <literal>execute</literal> will wait for this to consume all + output reliably. This can be achieved by redirecting stdout + to stderr <literal>>&2</literal>, to + <literal>/dev/console</literal>, + <literal>/dev/null</literal> or a file. Examples of + detaching commands are <literal>sleep 365d &</literal>, + where the shell forks a new process that can write to stdout + and <literal>xclip -i</literal>, where the + <literal>xclip</literal> command itself forks without + closing stdout. Takes an optional parameter + <literal>check_return</literal> that defaults to + <literal>True</literal>. Setting this parameter to + <literal>False</literal> will not check for the return code + and return -1 instead. This can be used for commands that + shut down the VM and would therefore break the pipe that + would be used for retrieving the return code. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>succeed</literal> + </term> + <listitem> + <para> + Execute a shell command, raising an exception if the exit + status is not zero, otherwise returning the standard output. + Commands are run with <literal>set -euo pipefail</literal> + set: + </para> + <itemizedlist> + <listitem> + <para> + If several commands are separated by + <literal>;</literal> and one fails, the command as a + whole will fail. + </para> + </listitem> + <listitem> + <para> + For pipelines, the last non-zero exit status will be + returned (if there is one, zero will be returned + otherwise). + </para> + </listitem> + <listitem> + <para> + Dereferencing unset variables fail the command. + </para> + </listitem> + <listitem> + <para> + It will wait for stdout to be closed. See + <literal>execute</literal> for the implications. + </para> + </listitem> + </itemizedlist> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>fail</literal> + </term> + <listitem> + <para> + Like <literal>succeed</literal>, but raising an exception if + the command returns a zero status. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>wait_until_succeeds</literal> + </term> + <listitem> + <para> + Repeat a shell command with 1-second intervals until it + succeeds. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>wait_until_fails</literal> + </term> + <listitem> + <para> + Repeat a shell command with 1-second intervals until it + fails. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>wait_for_unit</literal> + </term> + <listitem> + <para> + Wait until the specified systemd unit has reached the + <quote>active</quote> state. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>wait_for_file</literal> + </term> + <listitem> + <para> + Wait until the specified file exists. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>wait_for_open_port</literal> + </term> + <listitem> + <para> + Wait until a process is listening on the given TCP port (on + <literal>localhost</literal>, at least). + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>wait_for_closed_port</literal> + </term> + <listitem> + <para> + Wait until nobody is listening on the given TCP port. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>wait_for_x</literal> + </term> + <listitem> + <para> + Wait until the X11 server is accepting connections. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>wait_for_text</literal> + </term> + <listitem> + <para> + Wait until the supplied regular expressions matches the + textual contents of the screen by using optical character + recognition (see <literal>get_screen_text</literal> and + <literal>get_screen_text_variants</literal>). + </para> + <note> + <para> + This requires passing <literal>enableOCR</literal> to the + test attribute set. + </para> + </note> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>wait_for_console_text</literal> + </term> + <listitem> + <para> + Wait until the supplied regular expressions match a line of + the serial console output. This method is useful when OCR is + not possibile or accurate enough. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>wait_for_window</literal> + </term> + <listitem> + <para> + Wait until an X11 window has appeared whose name matches the + given regular expression, e.g., + <literal>wait_for_window("Terminal")</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>copy_from_host</literal> + </term> + <listitem> + <para> + Copies a file from host to machine, e.g., + <literal>copy_from_host("myfile", "/etc/my/important/file")</literal>. + </para> + <para> + The first argument is the file on the host. The file needs + to be accessible while building the nix derivation. The + second argument is the location of the file on the machine. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>systemctl</literal> + </term> + <listitem> + <para> + Runs <literal>systemctl</literal> commands with optional + support for <literal>systemctl --user</literal> + </para> + <programlisting language="python"> +machine.systemctl("list-jobs --no-pager") # runs `systemctl list-jobs --no-pager` +machine.systemctl("list-jobs --no-pager", "any-user") # spawns a shell for `any-user` and runs `systemctl --user list-jobs --no-pager` +</programlisting> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>shell_interact</literal> + </term> + <listitem> + <para> + Allows you to directly interact with the guest shell. This + should only be used during test development, not in + production tests. Killing the interactive session with + <literal>Ctrl-d</literal> or <literal>Ctrl-c</literal> also + ends the guest session. + </para> + </listitem> + </varlistentry> + </variablelist> + <para> + To test user units declared by + <literal>systemd.user.services</literal> the optional + <literal>user</literal> argument can be used: + </para> + <programlisting language="python"> +machine.start() +machine.wait_for_x() +machine.wait_for_unit("xautolock.service", "x-session-user") +</programlisting> + <para> + This applies to <literal>systemctl</literal>, + <literal>get_unit_info</literal>, + <literal>wait_for_unit</literal>, <literal>start_job</literal> and + <literal>stop_job</literal>. + </para> + <para> + For faster dev cycles it's also possible to disable the + code-linters (this shouldn't be commited though): + </para> + <programlisting language="bash"> +import ./make-test-python.nix { + skipLint = true; + machine = + { config, pkgs, ... }: + { configuration… + }; + + testScript = + '' + Python code… + ''; +} +</programlisting> + <para> + This will produce a Nix warning at evaluation time. To fully + disable the linter, wrap the test script in comment directives to + disable the Black linter directly (again, don't commit this within + the Nixpkgs repository): + </para> + <programlisting language="bash"> + testScript = + '' + # fmt: off + Python code… + # fmt: on + ''; +</programlisting> + </section> + <section xml:id="ssec-failing-tests-early"> + <title>Failing tests early</title> + <para> + To fail tests early when certain invariables are no longer met + (instead of waiting for the build to time out), the decorator + <literal>polling_condition</literal> is provided. For example, if + we are testing a program <literal>foo</literal> that should not + quit after being started, we might write the following: + </para> + <programlisting language="python"> +@polling_condition +def foo_running(): + machine.succeed("pgrep -x foo") + + +machine.succeed("foo --start") +machine.wait_until_succeeds("pgrep -x foo") + +with foo_running: + ... # Put `foo` through its paces +</programlisting> + <para> + <literal>polling_condition</literal> takes the following + (optional) arguments: + </para> + <para> + <literal>seconds_interval</literal> + </para> + <para> + : specifies how often the condition should be polled: + </para> + <programlisting> +```py +@polling_condition(seconds_interval=10) +def foo_running(): + machine.succeed("pgrep -x foo") +``` +</programlisting> + <para> + <literal>description</literal> + </para> + <para> + : is used in the log when the condition is checked. If this is not + provided, the description is pulled from the docstring of the + function. These two are therefore equivalent: + </para> + <programlisting> +```py +@polling_condition +def foo_running(): + "check that foo is running" + machine.succeed("pgrep -x foo") +``` + +```py +@polling_condition(description="check that foo is running") +def foo_running(): + machine.succeed("pgrep -x foo") +``` +</programlisting> + </section> +</section> |