diff options
Diffstat (limited to 'nixos/doc/manual/development.xml')
| -rw-r--r-- | nixos/doc/manual/development.xml | 599 |
1 files changed, 599 insertions, 0 deletions
diff --git a/nixos/doc/manual/development.xml b/nixos/doc/manual/development.xml new file mode 100644 index 00000000000..d8b5f6f571c --- /dev/null +++ b/nixos/doc/manual/development.xml @@ -0,0 +1,599 @@ +<chapter xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink"> + +<title>Development</title> + +<para>This chapter has some random notes on hacking on +NixOS.</para> + + +<!--===============================================================--> + +<section> + +<title>Hacking on NixOS</title> + +<para>By default, NixOS’s <command>nixos-rebuild</command> command +uses the NixOS and Nixpkgs sources provided by the +<literal>nixos-unstable</literal> channel (kept in +<filename>/nix/var/nix/profiles/per-user/root/channels/nixos</filename>). +To modify NixOS, however, you should check out the latest sources from +Git. This is done using the following command: + +<screen> +$ nixos-checkout <replaceable>/my/sources</replaceable> +</screen> + +or + +<screen> +$ mkdir -p <replaceable>/my/sources</replaceable> +$ cd <replaceable>/my/sources</replaceable> +$ nix-env -i git +$ git clone git://github.com/NixOS/nixos.git +$ git clone git://github.com/NixOS/nixpkgs.git +</screen> + +This will check out the latest NixOS sources to +<filename><replaceable>/my/sources</replaceable>/nixos</filename> and +the Nixpkgs sources to +<filename><replaceable>/my/sources</replaceable>/nixpkgs</filename>. +If you want to rebuild your system using your (modified) sources, you +need to tell <command>nixos-rebuild</command> about them using the +<option>-I</option> flag: + +<screen> +$ nixos-rebuild switch -I <replaceable>/my/sources</replaceable> +</screen> + +</para> + +<para><command>nixos-rebuild</command> affects only the system profile. +To install packages to your user profile from expressions in +<replaceable>/my/sources</replaceable>, use +<command>nix-env -f <replaceable>/my/sources</replaceable>/nixpkgs</command>, +or change the default by replacing the symlink in +<filename>~/.nix-defexpr</filename>: + +<screen> +$ rm -f ~/.nix-defexpr/channels +$ ln -s <replaceable>/my/sources</replaceable>/nixpkgs ~/.nix-defexpr/nixpkgs +</screen> + +</para> + +<para>You should not pass the base directory +<filename><replaceable>/my/sources</replaceable></filename> +to <command>nix-env</command>, as it will break after interpreting expressions +in <filename>nixos/</filename> as packages.</para> + +</section> + + +<!--===============================================================--> + +<section> + +<title>Extending NixOS</title> + +<para>NixOS is based on a modular system for declarative configuration. + This system combines multiple <emphasis>modules</emphasis> to produce one + configuration. One of the module which compose your computer + configuration is <filename>/etc/nixos/configuration.nix</filename>. Other + modules are available under NixOS <filename>modules</filename> + directory</para> + +<para>A module is a file which handles one specific part of the + configuration. This part of the configuration could correspond to + hardware, a service, network settings, or preferences. A module + configuration does not have to handle everything from scratch, it can base + its configuration on other configurations provided by other modules. Thus + a module can <emphasis>define</emphasis> options to setup its + configuration, and it can also <emphasis>declare</emphasis> options to be + fed by other modules.</para> + +<!-- module syntax --> + +<para xml:id="para-module-syn">A module is a file which contains a Nix + expression. This expression should be either an expression which gets + evaluated into an attribute set or a function which returns an attribute + set.</para> + +<para>When the expression is a function, it should expect only one argument + which is an attribute set containing an attribute + named <varname>config</varname> and another attribute + named <varname>pkgs</varname>. The <varname>config</varname> attribute + contains the result of the merge of all modules. This attribute is + evaluated lazily, such as any Nix expression. For more details on how + options are merged, see the details in <xref linkend="para-opt-decl"/>. + The <varname>pkgs</varname> attribute + contains <emphasis>nixpkgs</emphasis> attribute set of packages. This + attribute is necessary for declaring options.</para> + +<example xml:id='module-syntax'><title>Usual module content</title> +<programlisting> +{ config, pkgs, ... }: <co xml:id='module-syntax-1' /> + +{ + imports = + [ <co xml:id='module-syntax-2' /> + ]; + + options = { + <co xml:id='module-syntax-3' /> + }; + + config = { + <co xml:id='module-syntax-4' /> + }; +}</programlisting> +</example> + +<para><xref linkend='module-syntax' /> Illustrates + a <emphasis>module</emphasis> skeleton. + +<calloutlist> + <callout arearefs='module-syntax-1'> + <para>This line makes the current Nix expression a function. This + line can be omitted if there is no reference to <varname>pkgs</varname> + and <varname>config</varname> inside the module.</para> + </callout> + + <callout arearefs='module-syntax-2'> + <para>This list is used to enumerate path to other modules which are + declaring options used by the current module. In NixOS, default modules + are listed in the file <filename>modules/module-list.nix</filename>. + The default modules don't need to be added in the import list.</para> + </callout> + + <callout arearefs='module-syntax-3'> + <para>This attribute set contains an attribute set of <emphasis>option + declaration</emphasis>.</para> + </callout> + + <callout arearefs='module-syntax-4'> + <para>This attribute set contains an attribute set of <emphasis>option + definitions</emphasis>. If the module does not have any imported + modules or any option declarations, then this attribute set can be used + in place of its parent attribute set. This is a common case for simple + modules such + as <filename>/etc/nixos/configuration.nix</filename>.</para> + </callout> +</calloutlist> + +</para> + +<!-- option definitions --> + +<para xml:id="para-opt-def">A module defines a configuration which would be + interpreted by other modules. To define a configuration, a module needs + to provide option definitions. An option definition is a simple + attribute assignment.</para> + +<para>Option definitions are made in a declarative manner. Without + properties, options will always be defined with the same value. To + introduce more flexibility in the system, option definitions are guarded + by <emphasis>properties</emphasis>.</para> + +<para>Properties are means to introduce conditional values inside option + definitions. This conditional values can be distinguished in two + categories. The condition which are local to the current configuration + and conditions which are dependent on others configurations. Local + properties are <varname>mkIf</varname> + and <varname>mkAssert</varname>. Global properties + are <varname>mkOverride</varname>, <varname>mkDefault</varname> + and <varname>mkOrder</varname>.</para> + +<para><varname>mkIf</varname> is used to remove the option definitions which + are below it if the condition is evaluated to + false. <varname>mkAssert</varname> expects the condition to be evaluated + to true otherwise it raises an error message.</para> + +<para><varname>mkOverride</varname> is used to mask previous definitions if + the current value has a lower mask number. The mask value is 100 (default) + for any option definition which does not use this property. + Thus, <varname>mkDefault</varname> is just a short-cut with a higher mask + (1000) than the default mask value. This means that a module can set an + option definition as a preference, and still let another module defining + it with a different value without using any property.</para> + +<para><varname>mkOrder</varname> is used to sort definitions based on the + rank number. The rank number will sort all options definitions before + giving the sorted list of option definition to the merge function defined + in the option declaration. A lower rank will move the definition to the + beginning and a higher rank will move the option toward the end. The + default rank is 100.</para> + +<!-- option declarations --> + +<para xml:id="para-opt-decl">A module may declare options which are used by + other module to change the configuration provided by the current module. + Changes to the option definitions are made with properties which are using + values extracted from the result of the merge of all modules + (the <varname>config</varname> argument).</para> + +<para>The <varname>config</varname> argument reproduce the same hierarchy of + all options declared in all modules. For each option, the result of the + option is available, it is either the default value or the merge of all + definitions of the option.</para> + +<para>Options are declared with the + function <varname>pkgs.lib.mkOption</varname>. This function expects an + attribute set which at least provides a description. A default value, an + example, a type, a merge function and a post-process function can be + added.</para> + +<para>Types are used to provide a merge strategy for options and to ensure + the type of each option definitions. They are defined + in <varname>pkgs.lib.types</varname>.</para> + +<para>The merge function expects a list of option definitions and merge + them to obtain one result of the same type.</para> + +<para>The post-process function (named <varname>apply</varname>) takes the + result of the merge or of the default value, and produce an output which + could have a different type than the type expected by the option.</para> + +<!-- end --> + +<example xml:id='locate-example'><title>Locate Module Example</title> +<programlisting> +{ config, pkgs, ... }: + +with pkgs.lib; + +let + cfg = config.services.locate; + locatedb = "/var/cache/locatedb"; + logfile = "/var/log/updatedb"; + cmd =''root updatedb --localuser=nobody --output=${locatedb} > ${logfile}''; +in + +{ + imports = [ /etc/nixos/nixos/modules/services/scheduling/cron.nix ]; + + options = { + services.locate = { + enable = mkOption { + default = false; + example = true; + type = with types; bool; + description = '' + If enabled, NixOS will periodically update the database of + files used by the <command>locate</command> command. + ''; + }; + + period = mkOption { + default = "15 02 * * *"; + type = with types; uniq string; + description = '' + This option defines (in the format used by cron) when the + locate database is updated. + The default is to update at 02:15 (at night) every day. + ''; + }; + }; + }; + + config = mkIf cfg.enable { + services.cron = { + enable = true; + systemCronJobs = "${cfg.period} root ${cmd}"; + }; + }; +}</programlisting> +</example> + +<para><xref linkend='locate-example' /> illustrates a module which handles + the regular update of the database which index all files on the file + system. This modules has option definitions to rely on the cron service + to run the command at predefined dates. In addition, this modules + provides option declarations to enable the indexing and to use different + period of time to run the indexing. Properties are used to prevent + ambiguous definitions of option (enable locate service and disable cron + services) and to ensure that no options would be defined if the locate + service is not enabled.</para> + +</section> + + +<!--===============================================================--> + +<section> + +<title>Building specific parts of NixOS</title> + +<para> + +<screen> +$ nix-build /etc/nixos/nixos -A <replaceable>attr</replaceable></screen> + +where <replaceable>attr</replaceable> is an attribute in +<filename>/etc/nixos/nixos/default.nix</filename>. Attributes of interest include: + +<variablelist> + + <varlistentry> + <term><varname>config</varname></term> + <listitem><para>The computer configuration generated from + the <envar>NIXOS_CONFIG</envar> environment variable (default + is <filename>/etc/nixos/configuration.nix</filename>) with the NixOS + default set of modules.</para></listitem> + </varlistentry> + + <varlistentry> + <term><varname>system</varname></term> + <listitem><para>The derivation which build your computer system. It is + built by the command <command>nixos-rebuild + build</command></para></listitem> + </varlistentry> + + <varlistentry> + <term><varname>vm</varname></term> + <listitem><para>The derivation which build your computer system inside a + virtual machine. It is built by the command <command>nixos-rebuild + build-vm</command></para></listitem> + </varlistentry> +</variablelist> + +</para> + +<para> +Most parts of NixOS can be built through the <varname>config</varname> +attribute set. This attribute set allows you to have a view of the merged +option definitions and all its derivations. Important derivations are store +inside the option <option>system.build</option> and can be listed with the +command <command>nix-instantiate --xml --eval-only /etc/nixos/nixos -A +config.system.build</command> +</para> + +</section> + + +<!--===============================================================--> + +<section> + +<title>Building your own NixOS CD</title> + +<para>Building a NixOS CD is as easy as configuring your own computer. The +idea is to use another module which will replace +your <filename>configuration.nix</filename> to configure the system that +would be installed on the CD.</para> + +<para>Default CD/DVD configurations are available +inside <filename>nixos/modules/installer/cd-dvd</filename>. To build them +you have to set <envar>NIXOS_CONFIG</envar> before +running <command>nix-build</command> to build the ISO. + +<screen> +$ export NIXOS_CONFIG=/etc/nixos/nixos/modules/installer/cd-dvd/installation-cd-minimal.nix +$ nix-build /etc/nixos/nixos -A config.system.build.isoImage</screen> + +</para> + +<para>Before burning your CD/DVD, you can check the content of the image by mounting anywhere like +suggested by the following command: + +<screen> +$ mount -o loop -t iso9660 ./result/iso/cd.iso /mnt/iso</screen> + +</para> + +</section> + + +<!--===============================================================--> + +<section> + +<title>Testing/building the NixOS Manual</title> + +<para>A quick way to see if your documentation improvements +or option descriptions look good: + +<screen> +$ nix-build -A config.system.build.manual</screen> + +</para> + +</section> + + +<!--===============================================================--> + +<section> + +<title>Testing the installer</title> + +<para>Building, burning, and +booting from an installation CD is rather +tedious, so here is a quick way to see if the installer works +properly: + +<screen> +$ export NIXOS_CONFIG=/etc/nixos/nixos/modules/installer/cd-dvd/installation-cd-minimal.nix +$ nix-build /etc/nixos/nixos -A config.system.build.nixosInstall +$ dd if=/dev/zero of=diskimage seek=2G count=0 bs=1 +$ yes | mke2fs -j diskimage +$ mount -o loop diskimage /mnt +$ ./result/bin/nixos-install</screen> + +</para> + +</section> + + + +<!--===============================================================--> + +<section> + +<title>Testing the <literal>initrd</literal></title> + +<para>A quick way to test whether the kernel and the initial ramdisk +boot correctly is to use QEMU’s <option>-kernel</option> and +<option>-initrd</option> options: + +<screen> +$ nix-build /etc/nixos/nixos -A config.system.build.initialRamdisk -o initrd +$ nix-build /etc/nixos/nixos -A config.system.build.kernel -o kernel +$ qemu-system-x86_64 -kernel ./kernel/bzImage -initrd ./initrd/initrd -hda /dev/null +</screen> + +</para> + +</section> + +<section> + + <title>Whole-system testing using virtual machines</title> + + <para> + Complete NixOS GNU/Linux systems can be tested in virtual machines + (VMs). This makes it possible to test a system upgrade or + configuration change before rebooting into it, using the + <command>nixos-rebuild build-vm</command> or + <command>nixos-rebuild build-vm-with-bootloader</command> command. + </para> + + <para> + <!-- The following is adapted from + http://wiki.nixos.org/wiki/NixOS_VM_tests, by Eelco Dolstra. --> + + The <filename>tests/</filename> directory in the NixOS source tree + contains several <emphasis>whole-system unit tests</emphasis>. + These tests can be run<footnote><para>NixOS tests can be run both from + NixOS and from a non-NixOS GNU/Linux distribution, provided the + Nix package manager is installed.</para></footnote> from the NixOS + source tree as follows: + +<screen> +$ nix-build tests/ -A nfs.test +</screen> + + This performs an automated test of the NFS client and server + functionality in the Linux kernel, including file locking + semantics (e.g., whether locks are maintained across server + crashes). It will first build or download all the dependencies of + the test (e.g., all packages needed to run a NixOS VM). The test + is defined in <link + xlink:href="https://nixos.org/repos/nix/nixos/trunk/tests/nfs.nix"> + <filename>tests/nfs.nix</filename></link>. If the test succeeds, + <command>nix-build</command> will place a symlink + <filename>./result</filename> in the current directory pointing at + the location in the Nix store of the test results (e.g., + screenshots, test reports, and so on). In particular, a + pretty-printed log of the test is written to + <filename>log.html</filename>, which can be viewed using a web + browser like this: + +<screen> +$ firefox result/log.html +</screen> + </para> + + <para> + It is also possible to run the test environment interactively, + allowing you to experiment with the VMs. For example: + +<screen> +$ nix-build tests/ -A nfs.driver +$ ./result/bin/nixos-run-vms +</screen> + + The script <command>nixos-run-vms</command> starts the three + virtual machines defined in the NFS test using QEMU/KVM. The root + file system of the VMs is created on the fly and kept across VM + restarts in + <filename>./</filename><varname>hostname</varname><filename>.qcow2</filename>. + </para> + + <para> + Finally, the test itself can be run interactively. This is + particularly useful when developing or debugging a test: + +<screen> +$ nix-build tests/ -A nfs.driver +$ ./result/bin/nixos-test-driver +starting VDE switch for network 1 +> +</screen> + + Perl statements can now be typed in to start or manipulate the + VMs: + +<screen> +> startAll; +(the VMs start booting) +> $server->waitForJob("nfs-kernel-nfsd"); +> $client1->succeed("flock -x /data/lock -c 'sleep 100000' &"); +> $client2->fail("flock -n -s /data/lock true"); +> $client1->shutdown; +(this releases client1's lock) +> $client2->succeed("flock -n -s /data/lock true"); +</screen> + + The function <command>testScript</command> executes the entire + test script and drops you back into the test driver command line + upon its completion. This allows you to inspect the state of the + VMs after the test (e.g. to debug the test script). + </para> + + <para> + This and other tests are continuously run on <link + xlink:href="http://hydra.nixos.org/jobset/nixos/trunk">the Hydra + instance at <literal>nixos.org</literal></link>, which allows + developers to be notified of any regressions introduced by a NixOS + or Nixpkgs change. + </para> + + <para> + The actual Nix programming interface to VM testing is in NixOS, + under <link + xlink:href="https://nixos.org/repos/nix/nixos/trunk/lib/testing.nix"> + <filename>lib/testing.nix</filename></link>. This file defines a + function which takes an attribute set containing a + <literal>nixpkgs</literal> attribute (the path to a Nixpkgs + checkout), and a <literal>system</literal> attribute (the system + type). It returns an attribute set containing several utility + functions, among which the main entry point is + <literal>makeTest</literal>. + </para> + + <para> + The <literal>makeTest</literal> function takes a function similar to + that found in <link + xlink:href="https://nixos.org/repos/nix/nixos/trunk/tests/nfs.nix"> + <filename>tests/nfs.nix</filename></link> (discussed above). It + returns an attribute set containing (among others): + + <variablelist> + + <varlistentry> + <term><varname>test</varname></term> + <listitem><para>A derivation containing the test log as an HTML file, + as seen above, suitable for presentation in the Hydra continuous + build system.</para></listitem> + </varlistentry> + + <varlistentry> + <term><varname>report</varname></term> + <listitem><para>A derivation containing a code coverage report, with + meta-data suitable for Hydra.</para></listitem> + </varlistentry> + + <varlistentry> + <term><varname>driver</varname></term> + <listitem><para>A derivation containing scripts to run the VM test or + interact with the VM network interactively, as seen above.</para> + </listitem> + </varlistentry> + + </variablelist> + </para> + +</section> + +</chapter> |