diff options
Diffstat (limited to 'nixos/doc/manual/development')
18 files changed, 517 insertions, 973 deletions
diff --git a/nixos/doc/manual/development/assertions.section.md b/nixos/doc/manual/development/assertions.section.md new file mode 100644 index 00000000000..cc6d81e5699 --- /dev/null +++ b/nixos/doc/manual/development/assertions.section.md @@ -0,0 +1,40 @@ +# Warnings and Assertions {#sec-assertions} + +When configuration problems are detectable in a module, it is a good idea to write an assertion or warning. Doing so provides clear feedback to the user and prevents errors after the build. + +Although Nix has the `abort` and `builtins.trace` [functions](https://nixos.org/nix/manual/#ssec-builtins) to perform such tasks, they are not ideally suited for NixOS modules. Instead of these functions, you can declare your warnings and assertions using the NixOS module system. + +## Warnings {#sec-assertions-warnings} + +This is an example of using `warnings`. + +```nix +{ config, lib, ... }: +{ + config = lib.mkIf config.services.foo.enable { + warnings = + if config.services.foo.bar + then [ ''You have enabled the bar feature of the foo service. + This is known to cause some specific problems in certain situations. + '' ] + else []; + } +} +``` + +## Assertions {#sec-assertions-assetions} + +This example, extracted from the [`syslogd` module](https://github.com/NixOS/nixpkgs/blob/release-17.09/nixos/modules/services/logging/syslogd.nix) shows how to use `assertions`. Since there can only be one active syslog daemon at a time, an assertion is useful to prevent such a broken system from being built. + +```nix +{ config, lib, ... }: +{ + config = lib.mkIf config.services.syslogd.enable { + assertions = + [ { assertion = !config.services.rsyslogd.enable; + message = "rsyslogd conflicts with syslogd"; + } + ]; + } +} +``` diff --git a/nixos/doc/manual/development/assertions.xml b/nixos/doc/manual/development/assertions.xml deleted file mode 100644 index 32f90cf2e7c..00000000000 --- a/nixos/doc/manual/development/assertions.xml +++ /dev/null @@ -1,74 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xmlns:xi="http://www.w3.org/2001/XInclude" - version="5.0" - xml:id="sec-assertions"> - <title>Warnings and Assertions</title> - - <para> - When configuration problems are detectable in a module, it is a good idea to - write an assertion or warning. Doing so provides clear feedback to the user - and prevents errors after the build. - </para> - - <para> - Although Nix has the <literal>abort</literal> and - <literal>builtins.trace</literal> - <link xlink:href="https://nixos.org/nix/manual/#ssec-builtins">functions</link> - to perform such tasks, they are not ideally suited for NixOS modules. Instead - of these functions, you can declare your warnings and assertions using the - NixOS module system. - </para> - - <section xml:id="sec-assertions-warnings"> - <title>Warnings</title> - - <para> - This is an example of using <literal>warnings</literal>. - </para> - -<programlisting> -<![CDATA[ -{ config, lib, ... }: -{ - config = lib.mkIf config.services.foo.enable { - warnings = - if config.services.foo.bar - then [ ''You have enabled the bar feature of the foo service. - This is known to cause some specific problems in certain situations. - '' ] - else []; - } -} -]]> -</programlisting> - </section> - - <section xml:id="sec-assertions-assertions"> - <title>Assertions</title> - - <para> - This example, extracted from the - <link xlink:href="https://github.com/NixOS/nixpkgs/blob/release-17.09/nixos/modules/services/logging/syslogd.nix"> - <literal>syslogd</literal> module </link> shows how to use - <literal>assertions</literal>. Since there can only be one active syslog - daemon at a time, an assertion is useful to prevent such a broken system - from being built. - </para> - -<programlisting> -<![CDATA[ -{ config, lib, ... }: -{ - config = lib.mkIf config.services.syslogd.enable { - assertions = - [ { assertion = !config.services.rsyslogd.enable; - message = "rsyslogd conflicts with syslogd"; - } - ]; - } -} -]]> -</programlisting> - </section> -</section> diff --git a/nixos/doc/manual/development/building-nixos.chapter.md b/nixos/doc/manual/development/building-nixos.chapter.md new file mode 100644 index 00000000000..699a75f4115 --- /dev/null +++ b/nixos/doc/manual/development/building-nixos.chapter.md @@ -0,0 +1,18 @@ +# Building Your Own NixOS CD {#sec-building-cd} +Building a NixOS CD is as easy as configuring your own computer. The idea is to use another module which will replace your `configuration.nix` to configure the system that would be installed on the CD. + +Default CD/DVD configurations are available inside `nixos/modules/installer/cd-dvd` + +```ShellSession +$ git clone https://github.com/NixOS/nixpkgs.git +$ cd nixpkgs/nixos +$ nix-build -A config.system.build.isoImage -I nixos-config=modules/installer/cd-dvd/installation-cd-minimal.nix default.nix +``` + +Before burning your CD/DVD, you can check the content of the image by mounting anywhere like suggested by the following command: + +```ShellSession +# mount -o loop -t iso9660 ./result/iso/cd.iso /mnt/iso</screen> +``` + +If you want to customize your NixOS CD in more detail, or generate other kinds of images, you might want to check out [nixos-generators](https://github.com/nix-community/nixos-generators). This can also be a good starting point when you want to use Nix to build a 'minimal' image that doesn't include a NixOS installation. diff --git a/nixos/doc/manual/development/building-nixos.xml b/nixos/doc/manual/development/building-nixos.xml deleted file mode 100644 index 56a596baed0..00000000000 --- a/nixos/doc/manual/development/building-nixos.xml +++ /dev/null @@ -1,27 +0,0 @@ -<chapter xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xmlns:xi="http://www.w3.org/2001/XInclude" - version="5.0" - xml:id="sec-building-cd"> - <title>Building Your Own NixOS CD</title> - <para> - Building a NixOS CD is as easy as configuring your own computer. The idea is - to use another module which will replace your - <filename>configuration.nix</filename> to configure the system that would be - installed on the CD. - </para> - <para> - Default CD/DVD configurations are available inside - <filename>nixos/modules/installer/cd-dvd</filename>. -<screen> -<prompt>$ </prompt>git clone https://github.com/NixOS/nixpkgs.git -<prompt>$ </prompt>cd nixpkgs/nixos -<prompt>$ </prompt>nix-build -A config.system.build.isoImage -I nixos-config=modules/installer/cd-dvd/installation-cd-minimal.nix default.nix</screen> - </para> - <para> - Before burning your CD/DVD, you can check the content of the image by - mounting anywhere like suggested by the following command: -<screen> -<prompt># </prompt>mount -o loop -t iso9660 ./result/iso/cd.iso /mnt/iso</screen> - </para> -</chapter> diff --git a/nixos/doc/manual/development/development.xml b/nixos/doc/manual/development/development.xml index 43f511b3e96..eb505567962 100644 --- a/nixos/doc/manual/development/development.xml +++ b/nixos/doc/manual/development/development.xml @@ -13,8 +13,7 @@ <xi:include href="writing-modules.xml" /> <xi:include href="building-parts.xml" /> <xi:include href="writing-documentation.xml" /> - <xi:include href="building-nixos.xml" /> + <xi:include href="../from_md/development/building-nixos.chapter.xml" /> <xi:include href="nixos-tests.xml" /> <xi:include href="testing-installer.xml" /> - <xi:include href="releases.xml" /> </part> diff --git a/nixos/doc/manual/development/meta-attributes.xml b/nixos/doc/manual/development/meta-attributes.xml index 3d019a4987e..c40be0a50c3 100644 --- a/nixos/doc/manual/development/meta-attributes.xml +++ b/nixos/doc/manual/development/meta-attributes.xml @@ -57,7 +57,7 @@ linkend="ch-configuration"/>. Changes to a module documentation have to be checked to not break building the NixOS manual: </para> -<programlisting>$ nix-build nixos/release.nix -A manual</programlisting> +<screen><prompt>$ </prompt>nix-build nixos/release.nix -A manual.x86_64-linux</screen> </callout> </calloutlist> </section> diff --git a/nixos/doc/manual/development/nixos-tests.xml b/nixos/doc/manual/development/nixos-tests.xml index 2695082e386..702fc03f668 100644 --- a/nixos/doc/manual/development/nixos-tests.xml +++ b/nixos/doc/manual/development/nixos-tests.xml @@ -13,7 +13,7 @@ xlink:href="https://github.com/NixOS/nixpkgs/tree/master/nixos/tests">nixos/test one or more virtual machines containing the NixOS system(s) required for the test. </para> - <xi:include href="writing-nixos-tests.xml" /> - <xi:include href="running-nixos-tests.xml" /> - <xi:include href="running-nixos-tests-interactively.xml" /> + <xi:include href="../from_md/development/writing-nixos-tests.section.xml" /> + <xi:include href="../from_md/development/running-nixos-tests.section.xml" /> + <xi:include href="../from_md/development/running-nixos-tests-interactively.section.xml" /> </chapter> diff --git a/nixos/doc/manual/development/option-types.xml b/nixos/doc/manual/development/option-types.xml index 957349ad181..3d2191e2f3f 100644 --- a/nixos/doc/manual/development/option-types.xml +++ b/nixos/doc/manual/development/option-types.xml @@ -23,16 +23,6 @@ <variablelist> <varlistentry> <term> - <varname>types.attrs</varname> - </term> - <listitem> - <para> - A free-form attribute set. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> <varname>types.bool</varname> </term> <listitem> @@ -64,6 +54,64 @@ </para> </listitem> </varlistentry> + <varlistentry> + <term> + <varname>types.anything</varname> + </term> + <listitem> + <para> + A type that accepts any value and recursively merges attribute sets together. + This type is recommended when the option type is unknown. + <example xml:id="ex-types-anything"> + <title><literal>types.anything</literal> Example</title> + <para> + Two definitions of this type like +<programlisting> +{ + str = lib.mkDefault "foo"; + pkg.hello = pkgs.hello; + fun.fun = x: x + 1; +} +</programlisting> +<programlisting> +{ + str = lib.mkIf true "bar"; + pkg.gcc = pkgs.gcc; + fun.fun = lib.mkForce (x: x + 2); +} +</programlisting> + will get merged to +<programlisting> +{ + str = "bar"; + pkg.gcc = pkgs.gcc; + pkg.hello = pkgs.hello; + fun.fun = x: x + 2; +} +</programlisting> + </para> + </example> + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <varname>types.attrs</varname> + </term> + <listitem> + <para> + A free-form attribute set. + <warning><para> + This type will be deprecated in the future because it doesn't recurse + into attribute sets, silently drops earlier attribute definitions, and + doesn't discharge <literal>lib.mkDefault</literal>, <literal>lib.mkIf + </literal> and co. For allowing arbitrary attribute sets, prefer + <literal>types.attrsOf types.anything</literal> instead which doesn't + have these problems. + </para></warning> + </para> + </listitem> + </varlistentry> </variablelist> <para> @@ -387,17 +435,6 @@ </varlistentry> <varlistentry> <term> - <varname>types.loaOf</varname> <replaceable>t</replaceable> - </term> - <listitem> - <para> - An attribute set or a list of <replaceable>t</replaceable> type. Multiple - definitions are merged according to the value. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> <varname>types.nullOr</varname> <replaceable>t</replaceable> </term> <listitem> diff --git a/nixos/doc/manual/development/releases.xml b/nixos/doc/manual/development/releases.xml deleted file mode 100755 index 8abc66dfec1..00000000000 --- a/nixos/doc/manual/development/releases.xml +++ /dev/null @@ -1,301 +0,0 @@ -<chapter xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xmlns:xi="http://www.w3.org/2001/XInclude" - version="5.0" - xml:id="ch-releases"> - <title>Releases</title> - <section xml:id="release-process"> - <title>Release process</title> - - <para> - Going through an example of releasing NixOS 17.09: - </para> - - <section xml:id="one-month-before-the-beta"> - <title>One month before the beta</title> - - <itemizedlist spacing="compact"> - <listitem> - <para> - Send an email to the nix-devel mailinglist as a warning about upcoming - beta "feature freeze" in a month. - </para> - </listitem> - <listitem> - <para> - Discuss with Eelco Dolstra and the community (via IRC, ML) about what - will reach the deadline. Any issue or Pull Request targeting the release - should be included in the release milestone. - </para> - </listitem> - </itemizedlist> - </section> - - <section xml:id="at-beta-release-time"> - <title>At beta release time</title> - - <itemizedlist spacing="compact"> - <listitem> - <para> - <link xlink:href="https://github.com/NixOS/nixpkgs/issues/13559">Create - an issue for tracking Zero Hydra Failures progress. ZHF is an effort to - get build failures down to zero.</link> - </para> - </listitem> - <listitem> - <para> - <literal>git tag -a -s -m "Release 17.09-beta" 17.09-beta - && git push origin 17.09-beta</literal> - </para> - </listitem> - <listitem> - <para> - From the master branch run <literal>git checkout -b - release-17.09</literal>. - </para> - </listitem> - <listitem> - <para> - <link xlink:href="https://github.com/NixOS/nixos-org-configurations/pull/18"> - Make sure a channel is created at https://nixos.org/channels/. </link> - </para> - </listitem> - <listitem> - <para> - <link xlink:href="https://github.com/NixOS/nixpkgs/compare/bdf161ed8d21...6b63c4616790"> - Bump the <literal>system.defaultChannel</literal> attribute in - <literal>nixos/modules/misc/version.nix</literal> </link> - </para> - </listitem> - <listitem> - <para> - <link xlink:href="https://github.com/NixOS/nixpkgs/commit/d6b08acd1ccac0d9d502c4b635e00b04d3387f06"> - Update <literal>versionSuffix</literal> in - <literal>nixos/release.nix</literal></link>, use - <literal>git rev-list --count 17.09-beta</literal> - to get the commit count. - </para> - </listitem> - <listitem> - <para> - <literal>echo -n "18.03" > .version</literal> on master. - </para> - </listitem> - <listitem> - <para> - <link xlink:href="https://github.com/NixOS/nixpkgs/commit/b8a4095003e27659092892a4708bb3698231a842"> - Pick a new name for the unstable branch. </link> - </para> - </listitem> - <listitem> - <para> - Create a new release notes file for the upcoming release + 1, in this - case <literal>rl-1803.xml</literal>. - </para> - </listitem> - <listitem> - <para> - Create two Hydra jobsets: release-17.09 and release-17.09-small with - <literal>stableBranch</literal> set to false. - </para> - </listitem> - <listitem> - <para> - Remove attributes that we know we will not be able to support, - especially if there is a stable alternative. E.g. Check that our - Linux kernels' - <link xlink:href="https://www.kernel.org/category/releases.html"> - projected end-of-life</link> are after our release projected - end-of-life - </para> - </listitem> - <listitem> - <para> - Edit changelog at - <literal>nixos/doc/manual/release-notes/rl-1709.xml</literal> (double - check desktop versions are noted) - </para> - <itemizedlist spacing="compact"> - <listitem> - <para> - Get all new NixOS modules <literal>git diff - release-17.03..release-17.09 nixos/modules/module-list.nix|grep - ^+</literal> - </para> - </listitem> - <listitem> - <para> - Note systemd, kernel, glibc and Nix upgrades. - </para> - </listitem> - </itemizedlist> - </listitem> - </itemizedlist> - </section> - - <section xml:id="during-beta"> - <title>During Beta</title> - - <itemizedlist spacing="compact"> - <listitem> - <para> - Monitor the master branch for bugfixes and minor updates and cherry-pick - them to the release branch. - </para> - </listitem> - </itemizedlist> - </section> - - <section xml:id="before-the-final-release"> - <title>Before the final release</title> - - <itemizedlist spacing="compact"> - <listitem> - <para> - Re-check that the release notes are complete. - </para> - </listitem> - <listitem> - <para> - Release Nix (currently only Eelco Dolstra can do that). - <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/installer/tools/nix-fallback-paths.nix"> - Make sure fallback is updated. </link> - </para> - </listitem> - <listitem> - <para> - <link xlink:href="https://github.com/NixOS/nixpkgs/commit/40fd9ae3ac8048758abdcfc7d28a78b5f22fe97e"> - Update README.md with new stable NixOS version information. </link> - </para> - </listitem> - <listitem> - <para> - Change <literal>stableBranch</literal> to <literal>true</literal> in Hydra and wait for - the channel to update. - </para> - </listitem> - </itemizedlist> - </section> - - <section xml:id="at-final-release-time"> - <title>At final release time</title> - - <itemizedlist spacing="compact"> - <listitem> - <para> - <literal>git tag -s -a -m "Release 15.09" 15.09</literal> - </para> - </listitem> - <listitem> - <para> - Update "Chapter 4. Upgrading NixOS" section of the manual to match - new stable release version. - </para> - </listitem> - <listitem> - <para> - Update the - <link xlink:href="https://github.com/NixOS/nixos-homepage/commit/2a37975d5a617ecdfca94696242b6f32ffcba9f1"><code>NIXOS_SERIES</code></link> - in the - <link xlink:href="https://github.com/NixOS/nixos-homepage">nixos-homepage</link> - repository. - </para> - </listitem> - <listitem> - <para> - Get number of commits for the release: <literal>git log - release-14.04..release-14.12 --format=%an|wc -l</literal> - </para> - </listitem> - <listitem> - <para> - Commits by contributor: <literal>git log release-14.04..release-14.12 - --format=%an|sort|uniq -c|sort -rn</literal> - </para> - </listitem> - <listitem> - <para> - Create a new topic on <link xlink:href="https://discourse.nixos.org/">the - Discourse instance</link> to announce the release with the above information. - Best to check how previous email was formulated to see what needs to be - included. - </para> - </listitem> - </itemizedlist> - </section> - </section> - <section xml:id="release-managers"> - <title>Release Management Team</title> - <para> - For each release there are two release managers. After each release the - release manager having managed two releases steps down and the release - management team of the last release appoints a new release manager. - </para> - <para> - This makes sure a release management team always consists of one release - manager who already has managed one release and one release manager being - introduced to their role, making it easier to pass on knowledge and - experience. - </para> - <para> - Release managers for the current NixOS release are tracked by GitHub team - <link xlink:href="https://github.com/orgs/NixOS/teams/nixos-release-managers/members"><literal>@NixOS/nixos-release-managers</literal></link>. - </para> - <para> - A release manager's role and responsibilities are: - </para> - <itemizedlist> - <listitem><para>manage the release process</para></listitem> - <listitem><para>start discussions about features and changes for a given release</para></listitem> - <listitem><para>create a roadmap</para></listitem> - <listitem><para>release in cooperation with Eelco Dolstra</para></listitem> - <listitem><para>decide which bug fixes, features, etc... get backported after a release</para></listitem> - </itemizedlist> - </section> - <section xml:id="release-schedule"> - <title>Release schedule</title> - - <informaltable> - <tgroup cols="2"> - <colspec align="left" /> - <colspec align="left" /> - <thead> - <row> - <entry> - Date - </entry> - <entry> - Event - </entry> - </row> - </thead> - <tbody> - <row> - <entry> - 2016-07-25 - </entry> - <entry> - Send email to nix-dev about upcoming branch-off - </entry> - </row> - <row> - <entry> - 2016-09-01 - </entry> - <entry><literal>release-16.09</literal> branch and corresponding jobsets are created, - change freeze - </entry> - </row> - <row> - <entry> - 2016-09-30 - </entry> - <entry> - NixOS 16.09 released - </entry> - </row> - </tbody> - </tgroup> - </informaltable> - </section> -</chapter> diff --git a/nixos/doc/manual/development/running-nixos-tests-interactively.section.md b/nixos/doc/manual/development/running-nixos-tests-interactively.section.md new file mode 100644 index 00000000000..3ba4e16e77f --- /dev/null +++ b/nixos/doc/manual/development/running-nixos-tests-interactively.section.md @@ -0,0 +1,44 @@ +# Running Tests interactively {#sec-running-nixos-tests-interactively} + +The test itself can be run interactively. This is particularly useful +when developing or debugging a test: + +```ShellSession +$ nix-build nixos/tests/login.nix -A driverInteractive +$ ./result/bin/nixos-test-driver +starting VDE switch for network 1 +> +``` + +You can then take any Python statement, e.g. + +```py +> start_all() +> test_script() +> machine.succeed("touch /tmp/foo") +> print(machine.succeed("pwd")) # Show stdout of command +``` + +The function `test_script` executes the entire test script and drops you +back into the test driver command line upon its completion. This allows +you to inspect the state of the VMs after the test (e.g. to debug the +test script). + +To just start and experiment with the VMs, run: + +```ShellSession +$ nix-build nixos/tests/login.nix -A driverInteractive +$ ./result/bin/nixos-run-vms +``` + +The script `nixos-run-vms` starts the virtual machines defined by test. + +You can re-use the VM states coming from a previous run by setting the +`--keep-vm-state` flag. + +```ShellSession +$ ./result/bin/nixos-run-vms --keep-vm-state +``` + +The machine state is stored in the `$TMPDIR/vm-state-machinename` +directory. diff --git a/nixos/doc/manual/development/running-nixos-tests-interactively.xml b/nixos/doc/manual/development/running-nixos-tests-interactively.xml deleted file mode 100644 index a11a9382764..00000000000 --- a/nixos/doc/manual/development/running-nixos-tests-interactively.xml +++ /dev/null @@ -1,49 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xmlns:xi="http://www.w3.org/2001/XInclude" - version="5.0" - xml:id="sec-running-nixos-tests-interactively"> - <title>Running Tests interactively</title> - - <para> - The test itself can be run interactively. This is particularly useful when - developing or debugging a test: -<screen> -<prompt>$ </prompt>nix-build nixos/tests/login.nix -A driver -<prompt>$ </prompt>./result/bin/nixos-test-driver -starting VDE switch for network 1 -<prompt>></prompt> -</screen> - You can then take any Python statement, e.g. -<screen> -<prompt>></prompt> start_all() -<prompt>></prompt> test_script() -<prompt>></prompt> machine.succeed("touch /tmp/foo") -<prompt>></prompt> print(machine.succeed("pwd")) # Show stdout of command -</screen> - The function <command>test_script</command> executes the entire test script - and drops you back into the test driver command line upon its completion. - This allows you to inspect the state of the VMs after the test (e.g. to debug - the test script). - </para> - - <para> - To just start and experiment with the VMs, run: -<screen> -<prompt>$ </prompt>nix-build nixos/tests/login.nix -A driver -<prompt>$ </prompt>./result/bin/nixos-run-vms -</screen> - The script <command>nixos-run-vms</command> starts the virtual machines - defined by test. - </para> - - <para> - You can re-use the VM states coming from a previous run - by setting the <command>--keep-vm-state</command> flag. -<screen> -<prompt>$ </prompt>./result/bin/nixos-run-vms --keep-vm-state -</screen> - The machine state is stored in the - <filename>$TMPDIR/vm-state-</filename><varname>machinename</varname> directory. - </para> -</section> diff --git a/nixos/doc/manual/development/running-nixos-tests.section.md b/nixos/doc/manual/development/running-nixos-tests.section.md new file mode 100644 index 00000000000..d6a456f0188 --- /dev/null +++ b/nixos/doc/manual/development/running-nixos-tests.section.md @@ -0,0 +1,31 @@ +# Running Tests {#sec-running-nixos-tests} + +You can run tests using `nix-build`. For example, to run the test +[`login.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix), +you just do: + +```ShellSession +$ nix-build '<nixpkgs/nixos/tests/login.nix>' +``` + +or, if you don't want to rely on `NIX_PATH`: + +```ShellSession +$ cd /my/nixpkgs/nixos/tests +$ nix-build login.nix +… +running the VM test script +machine: QEMU running (pid 8841) +… +6 out of 6 tests succeeded +``` + +After building/downloading all required dependencies, this will perform +a build that starts a QEMU/KVM virtual machine containing a NixOS +system. The virtual machine mounts the Nix store of the host; this makes +VM creation very fast, as no disk image needs to be created. Afterwards, +you can view a pretty-printed log of the test: + +```ShellSession +$ firefox result/log.html +``` diff --git a/nixos/doc/manual/development/running-nixos-tests.xml b/nixos/doc/manual/development/running-nixos-tests.xml deleted file mode 100644 index e9257c907da..00000000000 --- a/nixos/doc/manual/development/running-nixos-tests.xml +++ /dev/null @@ -1,36 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xmlns:xi="http://www.w3.org/2001/XInclude" - version="5.0" - xml:id="sec-running-nixos-tests"> - <title>Running Tests</title> - - <para> - You can run tests using <command>nix-build</command>. For example, to run the - test - <filename -xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix">login.nix</filename>, - you just do: -<screen> -<prompt>$ </prompt>nix-build '<nixpkgs/nixos/tests/login.nix>' -</screen> - or, if you don’t want to rely on <envar>NIX_PATH</envar>: -<screen> -<prompt>$ </prompt>cd /my/nixpkgs/nixos/tests -<prompt>$ </prompt>nix-build login.nix -… -running the VM test script -machine: QEMU running (pid 8841) -… -6 out of 6 tests succeeded -</screen> - After building/downloading all required dependencies, this will perform a - build that starts a QEMU/KVM virtual machine containing a NixOS system. The - virtual machine mounts the Nix store of the host; this makes VM creation very - fast, as no disk image needs to be created. Afterwards, you can view a - pretty-printed log of the test: -<screen> -<prompt>$ </prompt>firefox result/log.html -</screen> - </para> -</section> diff --git a/nixos/doc/manual/development/settings-options.xml b/nixos/doc/manual/development/settings-options.xml index c99c3af92f8..7292cac62b7 100644 --- a/nixos/doc/manual/development/settings-options.xml +++ b/nixos/doc/manual/development/settings-options.xml @@ -50,7 +50,7 @@ </varlistentry> <varlistentry> <term> - <varname>pkgs.formats.ini</varname> { <replaceable>listsAsDuplicateKeys</replaceable> ? false, ... } + <varname>pkgs.formats.ini</varname> { <replaceable>listsAsDuplicateKeys</replaceable> ? false, <replaceable>listToValue</replaceable> ? null, ... } </term> <listitem> <para> @@ -66,6 +66,16 @@ </para> </listitem> </varlistentry> + <varlistentry> + <term> + <varname>listToValue</varname> + </term> + <listitem> + <para> + A function for turning a list of values into a single value. + </para> + </listitem> + </varlistentry> </variablelist> It returns a set with INI-specific attributes <varname>type</varname> and <varname>generate</varname> as specified <link linkend='pkgs-formats-result'>below</link>. </para> @@ -167,7 +177,7 @@ in { # We know that the `user` attribute exists because we set a default value # for it above, allowing us to use it without worries here - users.users.${cfg.settings.user} = {}; + users.users.${cfg.settings.user} = { isSystemUser = true; }; # ... }; diff --git a/nixos/doc/manual/development/writing-documentation.xml b/nixos/doc/manual/development/writing-documentation.xml index 2183937ad0d..89fab666561 100644 --- a/nixos/doc/manual/development/writing-documentation.xml +++ b/nixos/doc/manual/development/writing-documentation.xml @@ -24,8 +24,9 @@ </para> <screen> - $ cd /path/to/nixpkgs/nixos/doc/manual - $ make +<prompt>$ </prompt>cd /path/to/nixpkgs/nixos/doc/manual +<prompt>$ </prompt>nix-shell +<prompt>nix-shell$ </prompt>make </screen> <para> diff --git a/nixos/doc/manual/development/writing-modules.xml b/nixos/doc/manual/development/writing-modules.xml index d244356dbed..04497db77b8 100644 --- a/nixos/doc/manual/development/writing-modules.xml +++ b/nixos/doc/manual/development/writing-modules.xml @@ -74,7 +74,10 @@ linkend="sec-configuration-syntax"/>, we saw the following structure <callout arearefs='module-syntax-1'> <para> This line makes the current Nix expression a function. The variable - <varname>pkgs</varname> contains Nixpkgs, while <varname>config</varname> + <varname>pkgs</varname> contains Nixpkgs (by default, it takes the + <varname>nixpkgs</varname> entry of <envar>NIX_PATH</envar>, see the <link + xlink:href="https://nixos.org/manual/nix/stable/#sec-common-env">Nix + manual</link> for further details), while <varname>config</varname> contains the full system configuration. This line can be omitted if there is no reference to <varname>pkgs</varname> and <varname>config</varname> inside the module. @@ -179,7 +182,7 @@ in { <xi:include href="option-declarations.xml" /> <xi:include href="option-types.xml" /> <xi:include href="option-def.xml" /> - <xi:include href="assertions.xml" /> + <xi:include href="../from_md/development/assertions.section.xml" /> <xi:include href="meta-attributes.xml" /> <xi:include href="importing-modules.xml" /> <xi:include href="replace-modules.xml" /> diff --git a/nixos/doc/manual/development/writing-nixos-tests.section.md b/nixos/doc/manual/development/writing-nixos-tests.section.md new file mode 100644 index 00000000000..8471e7608af --- /dev/null +++ b/nixos/doc/manual/development/writing-nixos-tests.section.md @@ -0,0 +1,301 @@ +# Writing Tests {#sec-writing-nixos-tests} + +A NixOS test is a Nix expression that has the following structure: + +```nix +import ./make-test-python.nix { + + # Either the configuration of a single machine: + machine = + { config, pkgs, ... }: + { configuration… + }; + + # Or a set of machines: + nodes = + { machine1 = + { config, pkgs, ... }: { … }; + machine2 = + { config, pkgs, ... }: { … }; + … + }; + + testScript = + '' + Python code… + ''; +} +``` + +The attribute `testScript` is a bit of Python code that executes the +test (described below). During the test, it will start one or more +virtual machines, the configuration of which is described by the +attribute `machine` (if you need only one machine in your test) or by +the attribute `nodes` (if you need multiple machines). For instance, +[`login.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix) +only needs a single machine to test whether users can log in +on the virtual console, whether device ownership is correctly maintained +when switching between consoles, and so on. On the other hand, +[`nfs/simple.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nfs/simple.nix), +which tests NFS client and server functionality in the +Linux kernel (including whether locks are maintained across server +crashes), requires three machines: a server and two clients. + +There are a few special NixOS configuration options for test VMs: + +`virtualisation.memorySize` + +: The memory of the VM in megabytes. + +`virtualisation.vlans` + +: The virtual networks to which the VM is connected. See + [`nat.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nat.nix) + for an example. + +`virtualisation.writableStore` + +: By default, the Nix store in the VM is not writable. If you enable + this option, a writable union file system is mounted on top of the + Nix store to make it appear writable. This is necessary for tests + that run Nix operations that modify the store. + +For more options, see the module +[`qemu-vm.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix). + +The test script is a sequence of Python statements that perform various +actions, such as starting VMs, executing commands in the VMs, and so on. +Each virtual machine is represented as an object stored in the variable +`name` if this is also the identifier of the machine in the declarative +config. If you didn\'t specify multiple machines using the `nodes` +attribute, it is just `machine`. The following example starts the +machine, waits until it has finished booting, then executes a command +and checks that the output is more-or-less correct: + +```py +machine.start() +machine.wait_for_unit("default.target") +if not "Linux" in machine.succeed("uname"): + raise Exception("Wrong OS") +``` + +The first line is actually unnecessary; machines are implicitly started +when you first execute an action on them (such as `wait_for_unit` or +`succeed`). If you have multiple machines, you can speed up the test by +starting them in parallel: + +```py +start_all() +``` + +The following methods are available on machine objects: + +`start` + +: Start the virtual machine. This method is asynchronous --- it does + not wait for the machine to finish booting. + +`shutdown` + +: Shut down the machine, waiting for the VM to exit. + +`crash` + +: Simulate a sudden power failure, by telling the VM to exit + immediately. + +`block` + +: Simulate unplugging the Ethernet cable that connects the machine to + the other machines. + +`unblock` + +: Undo the effect of `block`. + +`screenshot` + +: Take a picture of the display of the virtual machine, in PNG format. + The screenshot is linked from the HTML log. + +`get_screen_text_variants` + +: Return a list of different interpretations of what is currently + visible on the machine\'s screen using optical character + recognition. The number and order of the interpretations is not + specified and is subject to change, but if no exception is raised at + least one will be returned. + + ::: {.note} + This requires passing `enableOCR` to the test attribute set. + ::: + +`get_screen_text` + +: Return a textual representation of what is currently visible on the + machine\'s screen using optical character recognition. + + ::: {.note} + This requires passing `enableOCR` to the test attribute set. + ::: + +`send_monitor_command` + +: Send a command to the QEMU monitor. This is rarely used, but allows + doing stuff such as attaching virtual USB disks to a running + machine. + +`send_key` + +: Simulate pressing keys on the virtual keyboard, e.g., + `send_key("ctrl-alt-delete")`. + +`send_chars` + +: Simulate typing a sequence of characters on the virtual keyboard, + e.g., `send_chars("foobar\n")` will type the string `foobar` + followed by the Enter key. + +`execute` + +: Execute a shell command, returning a list `(status, stdout)`. + +`succeed` + +: Execute a shell command, raising an exception if the exit status is + not zero, otherwise returning the standard output. Commands are run + with `set -euo pipefail` set: + + - If several commands are separated by `;` and one fails, the + command as a whole will fail. + + - For pipelines, the last non-zero exit status will be returned + (if there is one, zero will be returned otherwise). + + - Dereferencing unset variables fail the command. + +`fail` + +: Like `succeed`, but raising an exception if the command returns a zero + status. + +`wait_until_succeeds` + +: Repeat a shell command with 1-second intervals until it succeeds. + +`wait_until_fails` + +: Repeat a shell command with 1-second intervals until it fails. + +`wait_for_unit` + +: Wait until the specified systemd unit has reached the "active" + state. + +`wait_for_file` + +: Wait until the specified file exists. + +`wait_for_open_port` + +: Wait until a process is listening on the given TCP port (on + `localhost`, at least). + +`wait_for_closed_port` + +: Wait until nobody is listening on the given TCP port. + +`wait_for_x` + +: Wait until the X11 server is accepting connections. + +`wait_for_text` + +: Wait until the supplied regular expressions matches the textual + contents of the screen by using optical character recognition (see + `get_screen_text` and `get_screen_text_variants`). + + ::: {.note} + This requires passing `enableOCR` to the test attribute set. + ::: + +`wait_for_console_text` + +: Wait until the supplied regular expressions match a line of the + serial console output. This method is useful when OCR is not + possibile or accurate enough. + +`wait_for_window` + +: Wait until an X11 window has appeared whose name matches the given + regular expression, e.g., `wait_for_window("Terminal")`. + +`copy_from_host` + +: Copies a file from host to machine, e.g., + `copy_from_host("myfile", "/etc/my/important/file")`. + + The first argument is the file on the host. The file needs to be + accessible while building the nix derivation. The second argument is + the location of the file on the machine. + +`systemctl` + +: Runs `systemctl` commands with optional support for + `systemctl --user` + + ```py + machine.systemctl("list-jobs --no-pager") # runs `systemctl list-jobs --no-pager` + machine.systemctl("list-jobs --no-pager", "any-user") # spawns a shell for `any-user` and runs `systemctl --user list-jobs --no-pager` + ``` + +`shell_interact` + +: Allows you to directly interact with the guest shell. This should + only be used during test development, not in production tests. + Killing the interactive session with `Ctrl-d` or `Ctrl-c` also ends + the guest session. + +To test user units declared by `systemd.user.services` the optional +`user` argument can be used: + +```py +machine.start() +machine.wait_for_x() +machine.wait_for_unit("xautolock.service", "x-session-user") +``` + +This applies to `systemctl`, `get_unit_info`, `wait_for_unit`, +`start_job` and `stop_job`. + +For faster dev cycles it\'s also possible to disable the code-linters +(this shouldn\'t be commited though): + +```nix +import ./make-test-python.nix { + skipLint = true; + machine = + { config, pkgs, ... }: + { configuration… + }; + + testScript = + '' + Python code… + ''; +} +``` + +This will produce a Nix warning at evaluation time. To fully disable the +linter, wrap the test script in comment directives to disable the Black +linter directly (again, don\'t commit this within the Nixpkgs +repository): + +```nix + testScript = + '' + # fmt: off + Python code… + # fmt: on + ''; +``` diff --git a/nixos/doc/manual/development/writing-nixos-tests.xml b/nixos/doc/manual/development/writing-nixos-tests.xml deleted file mode 100644 index 74ab23605b3..00000000000 --- a/nixos/doc/manual/development/writing-nixos-tests.xml +++ /dev/null @@ -1,453 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xmlns:xi="http://www.w3.org/2001/XInclude" - version="5.0" - xml:id="sec-writing-nixos-tests"> - <title>Writing Tests</title> - - <para> - A NixOS test is a Nix expression that has the following structure: -<programlisting> -import ./make-test-python.nix { - - # Either the configuration of a single machine: - machine = - { config, pkgs, ... }: - { <replaceable>configuration…</replaceable> - }; - - # Or a set of machines: - nodes = - { <replaceable>machine1</replaceable> = - { config, pkgs, ... }: { <replaceable>…</replaceable> }; - <replaceable>machine2</replaceable> = - { config, pkgs, ... }: { <replaceable>…</replaceable> }; - … - }; - - testScript = - '' - <replaceable>Python code…</replaceable> - ''; -} -</programlisting> - The attribute <literal>testScript</literal> is a bit of Python code that - executes the test (described below). During the test, it will start one or - more virtual machines, the configuration of which is described by the - attribute <literal>machine</literal> (if you need only one machine in your - test) or by the attribute <literal>nodes</literal> (if you need multiple - machines). For instance, - <filename -xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix">login.nix</filename> - only needs a single machine to test whether users can log in on the virtual - console, whether device ownership is correctly maintained when switching - between consoles, and so on. On the other hand, - <filename -xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nfs.nix">nfs.nix</filename>, - which tests NFS client and server functionality in the Linux kernel - (including whether locks are maintained across server crashes), requires - three machines: a server and two clients. - </para> - - <para> - There are a few special NixOS configuration options for test VMs: -<!-- FIXME: would be nice to generate this automatically. --> - <variablelist> - <varlistentry> - <term> - <option>virtualisation.memorySize</option> - </term> - <listitem> - <para> - The memory of the VM in megabytes. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <option>virtualisation.vlans</option> - </term> - <listitem> - <para> - The virtual networks to which the VM is connected. See - <filename - xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nat.nix">nat.nix</filename> - for an example. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <option>virtualisation.writableStore</option> - </term> - <listitem> - <para> - By default, the Nix store in the VM is not writable. If you enable this - option, a writable union file system is mounted on top of the Nix store - to make it appear writable. This is necessary for tests that run Nix - operations that modify the store. - </para> - </listitem> - </varlistentry> - </variablelist> - For more options, see the module - <filename -xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix">qemu-vm.nix</filename>. - </para> - - <para> - The test script is a sequence of Python statements that perform various - actions, such as starting VMs, executing commands in the VMs, and so on. Each - virtual machine is represented as an object stored in the variable - <literal><replaceable>name</replaceable></literal> if this is also the - identifier of the machine in the declarative config. - If you didn't specify multiple machines using the <literal>nodes</literal> - attribute, it is just <literal>machine</literal>. - The following example starts the machine, waits until it has finished booting, - then executes a command and checks that the output is more-or-less correct: -<programlisting> -machine.start() -machine.wait_for_unit("default.target") -if not "Linux" in machine.succeed("uname"): - raise Exception("Wrong OS") -</programlisting> - The first line is actually unnecessary; machines are implicitly started when - you first execute an action on them (such as <literal>wait_for_unit</literal> - or <literal>succeed</literal>). If you have multiple machines, you can speed - up the test by starting them in parallel: -<programlisting> -start_all() -</programlisting> - </para> - - <para> - The following methods are available on machine objects: - <variablelist> - <varlistentry> - <term> - <methodname>start</methodname> - </term> - <listitem> - <para> - Start the virtual machine. This method is asynchronous — it does not - wait for the machine to finish booting. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>shutdown</methodname> - </term> - <listitem> - <para> - Shut down the machine, waiting for the VM to exit. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>crash</methodname> - </term> - <listitem> - <para> - Simulate a sudden power failure, by telling the VM to exit immediately. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>block</methodname> - </term> - <listitem> - <para> - Simulate unplugging the Ethernet cable that connects the machine to the - other machines. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>unblock</methodname> - </term> - <listitem> - <para> - Undo the effect of <methodname>block</methodname>. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>screenshot</methodname> - </term> - <listitem> - <para> - Take a picture of the display of the virtual machine, in PNG format. The - screenshot is linked from the HTML log. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>get_screen_text</methodname> - </term> - <listitem> - <para> - Return a textual representation of what is currently visible on the - machine's screen using optical character recognition. - </para> - <note> - <para> - This requires passing <option>enableOCR</option> to the test attribute - set. - </para> - </note> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>send_monitor_command</methodname> - </term> - <listitem> - <para> - Send a command to the QEMU monitor. This is rarely used, but allows doing - stuff such as attaching virtual USB disks to a running machine. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>send_key</methodname> - </term> - <listitem> - <para> - Simulate pressing keys on the virtual keyboard, e.g., - <literal>send_key("ctrl-alt-delete")</literal>. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>send_chars</methodname> - </term> - <listitem> - <para> - Simulate typing a sequence of characters on the virtual keyboard, e.g., - <literal>send_chars("foobar\n")</literal> will type the string - <literal>foobar</literal> followed by the Enter key. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>execute</methodname> - </term> - <listitem> - <para> - Execute a shell command, returning a list - <literal>(<replaceable>status</replaceable>, - <replaceable>stdout</replaceable>)</literal>. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>succeed</methodname> - </term> - <listitem> - <para> - Execute a shell command, raising an exception if the exit status is not - zero, otherwise returning the standard output. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>fail</methodname> - </term> - <listitem> - <para> - Like <methodname>succeed</methodname>, but raising an exception if the - command returns a zero status. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_until_succeeds</methodname> - </term> - <listitem> - <para> - Repeat a shell command with 1-second intervals until it succeeds. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_until_fails</methodname> - </term> - <listitem> - <para> - Repeat a shell command with 1-second intervals until it fails. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_for_unit</methodname> - </term> - <listitem> - <para> - Wait until the specified systemd unit has reached the “active” state. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_for_file</methodname> - </term> - <listitem> - <para> - Wait until the specified file exists. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_for_open_port</methodname> - </term> - <listitem> - <para> - Wait until a process is listening on the given TCP port (on - <literal>localhost</literal>, at least). - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_for_closed_port</methodname> - </term> - <listitem> - <para> - Wait until nobody is listening on the given TCP port. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_for_x</methodname> - </term> - <listitem> - <para> - Wait until the X11 server is accepting connections. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_for_text</methodname> - </term> - <listitem> - <para> - Wait until the supplied regular expressions matches the textual contents - of the screen by using optical character recognition (see - <methodname>get_screen_text</methodname>). - </para> - <note> - <para> - This requires passing <option>enableOCR</option> to the test attribute - set. - </para> - </note> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_for_console_text</methodname> - </term> - <listitem> - <para> - Wait until the supplied regular expressions match a line of the serial - console output. This method is useful when OCR is not possibile or - accurate enough. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_for_window</methodname> - </term> - <listitem> - <para> - Wait until an X11 window has appeared whose name matches the given - regular expression, e.g., <literal>wait_for_window("Terminal")</literal>. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>copy_from_host</methodname> - </term> - <listitem> - <para> - Copies a file from host to machine, e.g., - <literal>copy_from_host("myfile", "/etc/my/important/file")</literal>. - </para> - <para> - The first argument is the file on the host. The file needs to be - accessible while building the nix derivation. The second argument is the - location of the file on the machine. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>systemctl</methodname> - </term> - <listitem> - <para> - Runs <literal>systemctl</literal> commands with optional support for - <literal>systemctl --user</literal> - </para> - <para> -<programlisting> -machine.systemctl("list-jobs --no-pager") # runs `systemctl list-jobs --no-pager` -machine.systemctl("list-jobs --no-pager", "any-user") # spawns a shell for `any-user` and runs `systemctl --user list-jobs --no-pager` -</programlisting> - </para> - </listitem> - </varlistentry> - </variablelist> - </para> - - <para> - To test user units declared by <literal>systemd.user.services</literal> the - optional <literal>user</literal> argument can be used: -<programlisting> -machine.start() -machine.wait_for_x() -machine.wait_for_unit("xautolock.service", "x-session-user") -</programlisting> - This applies to <literal>systemctl</literal>, <literal>get_unit_info</literal>, - <literal>wait_for_unit</literal>, <literal>start_job</literal> and - <literal>stop_job</literal>. - </para> - - <para> - For faster dev cycles it's also possible to disable the code-linters (this shouldn't - be commited though): -<programlisting> -import ./make-test-python.nix { - skipLint = true; - machine = - { config, pkgs, ... }: - { <replaceable>configuration…</replaceable> - }; - - testScript = - '' - <replaceable>Python code…</replaceable> - ''; -} -</programlisting> - </para> -</section> |