diff options
Diffstat (limited to 'nixos/doc/manual/development/writing-nixos-tests.xml')
| -rw-r--r-- | nixos/doc/manual/development/writing-nixos-tests.xml | 453 |
1 files changed, 0 insertions, 453 deletions
diff --git a/nixos/doc/manual/development/writing-nixos-tests.xml b/nixos/doc/manual/development/writing-nixos-tests.xml deleted file mode 100644 index 74ab23605b3..00000000000 --- a/nixos/doc/manual/development/writing-nixos-tests.xml +++ /dev/null @@ -1,453 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xmlns:xi="http://www.w3.org/2001/XInclude" - version="5.0" - xml:id="sec-writing-nixos-tests"> - <title>Writing Tests</title> - - <para> - A NixOS test is a Nix expression that has the following structure: -<programlisting> -import ./make-test-python.nix { - - # Either the configuration of a single machine: - machine = - { config, pkgs, ... }: - { <replaceable>configuration…</replaceable> - }; - - # Or a set of machines: - nodes = - { <replaceable>machine1</replaceable> = - { config, pkgs, ... }: { <replaceable>…</replaceable> }; - <replaceable>machine2</replaceable> = - { config, pkgs, ... }: { <replaceable>…</replaceable> }; - … - }; - - testScript = - '' - <replaceable>Python code…</replaceable> - ''; -} -</programlisting> - The attribute <literal>testScript</literal> is a bit of Python code that - executes the test (described below). During the test, it will start one or - more virtual machines, the configuration of which is described by the - attribute <literal>machine</literal> (if you need only one machine in your - test) or by the attribute <literal>nodes</literal> (if you need multiple - machines). For instance, - <filename -xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix">login.nix</filename> - only needs a single machine to test whether users can log in on the virtual - console, whether device ownership is correctly maintained when switching - between consoles, and so on. On the other hand, - <filename -xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nfs.nix">nfs.nix</filename>, - which tests NFS client and server functionality in the Linux kernel - (including whether locks are maintained across server crashes), requires - three machines: a server and two clients. - </para> - - <para> - There are a few special NixOS configuration options for test VMs: -<!-- FIXME: would be nice to generate this automatically. --> - <variablelist> - <varlistentry> - <term> - <option>virtualisation.memorySize</option> - </term> - <listitem> - <para> - The memory of the VM in megabytes. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <option>virtualisation.vlans</option> - </term> - <listitem> - <para> - The virtual networks to which the VM is connected. See - <filename - xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nat.nix">nat.nix</filename> - for an example. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <option>virtualisation.writableStore</option> - </term> - <listitem> - <para> - By default, the Nix store in the VM is not writable. If you enable this - option, a writable union file system is mounted on top of the Nix store - to make it appear writable. This is necessary for tests that run Nix - operations that modify the store. - </para> - </listitem> - </varlistentry> - </variablelist> - For more options, see the module - <filename -xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix">qemu-vm.nix</filename>. - </para> - - <para> - The test script is a sequence of Python statements that perform various - actions, such as starting VMs, executing commands in the VMs, and so on. Each - virtual machine is represented as an object stored in the variable - <literal><replaceable>name</replaceable></literal> if this is also the - identifier of the machine in the declarative config. - If you didn't specify multiple machines using the <literal>nodes</literal> - attribute, it is just <literal>machine</literal>. - The following example starts the machine, waits until it has finished booting, - then executes a command and checks that the output is more-or-less correct: -<programlisting> -machine.start() -machine.wait_for_unit("default.target") -if not "Linux" in machine.succeed("uname"): - raise Exception("Wrong OS") -</programlisting> - The first line is actually unnecessary; machines are implicitly started when - you first execute an action on them (such as <literal>wait_for_unit</literal> - or <literal>succeed</literal>). If you have multiple machines, you can speed - up the test by starting them in parallel: -<programlisting> -start_all() -</programlisting> - </para> - - <para> - The following methods are available on machine objects: - <variablelist> - <varlistentry> - <term> - <methodname>start</methodname> - </term> - <listitem> - <para> - Start the virtual machine. This method is asynchronous — it does not - wait for the machine to finish booting. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>shutdown</methodname> - </term> - <listitem> - <para> - Shut down the machine, waiting for the VM to exit. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>crash</methodname> - </term> - <listitem> - <para> - Simulate a sudden power failure, by telling the VM to exit immediately. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>block</methodname> - </term> - <listitem> - <para> - Simulate unplugging the Ethernet cable that connects the machine to the - other machines. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>unblock</methodname> - </term> - <listitem> - <para> - Undo the effect of <methodname>block</methodname>. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>screenshot</methodname> - </term> - <listitem> - <para> - Take a picture of the display of the virtual machine, in PNG format. The - screenshot is linked from the HTML log. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>get_screen_text</methodname> - </term> - <listitem> - <para> - Return a textual representation of what is currently visible on the - machine's screen using optical character recognition. - </para> - <note> - <para> - This requires passing <option>enableOCR</option> to the test attribute - set. - </para> - </note> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>send_monitor_command</methodname> - </term> - <listitem> - <para> - Send a command to the QEMU monitor. This is rarely used, but allows doing - stuff such as attaching virtual USB disks to a running machine. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>send_key</methodname> - </term> - <listitem> - <para> - Simulate pressing keys on the virtual keyboard, e.g., - <literal>send_key("ctrl-alt-delete")</literal>. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>send_chars</methodname> - </term> - <listitem> - <para> - Simulate typing a sequence of characters on the virtual keyboard, e.g., - <literal>send_chars("foobar\n")</literal> will type the string - <literal>foobar</literal> followed by the Enter key. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>execute</methodname> - </term> - <listitem> - <para> - Execute a shell command, returning a list - <literal>(<replaceable>status</replaceable>, - <replaceable>stdout</replaceable>)</literal>. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>succeed</methodname> - </term> - <listitem> - <para> - Execute a shell command, raising an exception if the exit status is not - zero, otherwise returning the standard output. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>fail</methodname> - </term> - <listitem> - <para> - Like <methodname>succeed</methodname>, but raising an exception if the - command returns a zero status. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_until_succeeds</methodname> - </term> - <listitem> - <para> - Repeat a shell command with 1-second intervals until it succeeds. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_until_fails</methodname> - </term> - <listitem> - <para> - Repeat a shell command with 1-second intervals until it fails. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_for_unit</methodname> - </term> - <listitem> - <para> - Wait until the specified systemd unit has reached the “active” state. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_for_file</methodname> - </term> - <listitem> - <para> - Wait until the specified file exists. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_for_open_port</methodname> - </term> - <listitem> - <para> - Wait until a process is listening on the given TCP port (on - <literal>localhost</literal>, at least). - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_for_closed_port</methodname> - </term> - <listitem> - <para> - Wait until nobody is listening on the given TCP port. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_for_x</methodname> - </term> - <listitem> - <para> - Wait until the X11 server is accepting connections. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_for_text</methodname> - </term> - <listitem> - <para> - Wait until the supplied regular expressions matches the textual contents - of the screen by using optical character recognition (see - <methodname>get_screen_text</methodname>). - </para> - <note> - <para> - This requires passing <option>enableOCR</option> to the test attribute - set. - </para> - </note> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_for_console_text</methodname> - </term> - <listitem> - <para> - Wait until the supplied regular expressions match a line of the serial - console output. This method is useful when OCR is not possibile or - accurate enough. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>wait_for_window</methodname> - </term> - <listitem> - <para> - Wait until an X11 window has appeared whose name matches the given - regular expression, e.g., <literal>wait_for_window("Terminal")</literal>. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>copy_from_host</methodname> - </term> - <listitem> - <para> - Copies a file from host to machine, e.g., - <literal>copy_from_host("myfile", "/etc/my/important/file")</literal>. - </para> - <para> - The first argument is the file on the host. The file needs to be - accessible while building the nix derivation. The second argument is the - location of the file on the machine. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <methodname>systemctl</methodname> - </term> - <listitem> - <para> - Runs <literal>systemctl</literal> commands with optional support for - <literal>systemctl --user</literal> - </para> - <para> -<programlisting> -machine.systemctl("list-jobs --no-pager") # runs `systemctl list-jobs --no-pager` -machine.systemctl("list-jobs --no-pager", "any-user") # spawns a shell for `any-user` and runs `systemctl --user list-jobs --no-pager` -</programlisting> - </para> - </listitem> - </varlistentry> - </variablelist> - </para> - - <para> - To test user units declared by <literal>systemd.user.services</literal> the - optional <literal>user</literal> argument can be used: -<programlisting> -machine.start() -machine.wait_for_x() -machine.wait_for_unit("xautolock.service", "x-session-user") -</programlisting> - This applies to <literal>systemctl</literal>, <literal>get_unit_info</literal>, - <literal>wait_for_unit</literal>, <literal>start_job</literal> and - <literal>stop_job</literal>. - </para> - - <para> - For faster dev cycles it's also possible to disable the code-linters (this shouldn't - be commited though): -<programlisting> -import ./make-test-python.nix { - skipLint = true; - machine = - { config, pkgs, ... }: - { <replaceable>configuration…</replaceable> - }; - - testScript = - '' - <replaceable>Python code…</replaceable> - ''; -} -</programlisting> - </para> -</section> |