path: root/doc/functions.xml

<chapter xmlns="http://docbook.org/ns/docbook"
	 xmlns:xlink="http://www.w3.org/1999/xlink"
	 xml:id="chap-functions">

<title>Functions reference</title>

<para>
  The nixpkgs repository has several utility functions to manipulate Nix expressions.
</para>

<section xml:id="sec-pkgs-overridePackages">
  <title>pkgs.overridePackages</title>

  <para>
    This function inside the nixpkgs expression (<varname>pkgs</varname>)
    can be used to override the set of packages itself.
  </para>
  <para>
    Warning: this function is expensive and must not be used from within
    the nixpkgs repository.
  </para>
  <para>
    Example usage:

    <programlisting>let
  pkgs = import &lt;nixpkgs&gt; {};
  newpkgs = pkgs.overridePackages (self: super: {
    foo = super.foo.override { ... };
  };
in ...</programlisting>
  </para>

  <para>
    The resulting <varname>newpkgs</varname> will have the new <varname>foo</varname>
    expression, and all other expressions depending on <varname>foo</varname> will also
    use the new <varname>foo</varname> expression.
  </para>

  <para>
    The behavior of this function is similar to <link 
    linkend="sec-modify-via-packageOverrides">config.packageOverrides</link>.
  </para>

  <para>
    The <varname>self</varname> parameter refers to the final package set with the
    applied overrides. Using this parameter may lead to infinite recursion if not
    used consciously.
  </para>

  <para>
    The <varname>super</varname> parameter refers to the old package set.
    It's equivalent to <varname>pkgs</varname> in the above example.
  </para>

</section>

<section xml:id="sec-pkg-override">
  <title>&lt;pkg&gt;.override</title>

  <para>
    The function <varname>override</varname> is usually available for all the
    derivations in the nixpkgs expression (<varname>pkgs</varname>).
  </para>
  <para>
    It is used to override the arguments passed to a function.
  </para>
  <para>
    Example usages:

    <programlisting>pkgs.foo.override { arg1 = val1; arg2 = val2; ... }</programlisting>
    <programlisting>pkgs.overridePackages (self: super: {
  foo = super.foo.override { barSupport = true ; };
})</programlisting>
    <programlisting>mypkg = pkgs.callPackage ./mypkg.nix {
  mydep = pkgs.mydep.override { ... };
})</programlisting>
  </para>

  <para>
    In the first example, <varname>pkgs.foo</varname> is the result of a function call
    with some default arguments, usually a derivation.
    Using <varname>pkgs.foo.override</varname> will call the same function with
    the given new arguments.
  </para>

</section>

<section xml:id="sec-pkg-overrideDerivation">
  <title>&lt;pkg&gt;.overrideDerivation</title>

  <warning>
    <para>Do not use this function in Nixpkgs. Because it breaks
    package abstraction and doesn’t provide error checking for
    function arguments, it is only intended for ad-hoc customisation
    (such as in <filename>~/.nixpkgs/config.nix</filename>).</para>
  </warning>

  <para>
    The function <varname>overrideDerivation</varname> is usually available for all the
    derivations in the nixpkgs expression (<varname>pkgs</varname>).
  </para> 
  <para>
    It is used to create a new derivation by overriding the attributes of
    the original derivation according to the given function.
  </para>

  <para>
    Example usage:

    <programlisting>mySed = pkgs.gnused.overrideDerivation (oldAttrs: {
  name = "sed-4.2.2-pre";
  src = fetchurl {
    url = ftp://alpha.gnu.org/gnu/sed/sed-4.2.2-pre.tar.bz2;
    sha256 = "11nq06d131y4wmf3drm0yk502d2xc6n5qy82cg88rb9nqd2lj41k";
  };
  patches = [];
});</programlisting>
  </para>

  <para>
    In the above example, the name, src and patches of the derivation
    will be overridden, while all other attributes will be retained from the
    original derivation.
  </para>

  <para>
    The argument <varname>oldAttrs</varname> is used to refer to the attribute set of
    the original derivation.
  </para>

</section>

<section xml:id="sec-lib-makeOverridable">
  <title>lib.makeOverridable</title>

  <para>
    The function <varname>lib.makeOverridable</varname> is used to make the result
    of a function easily customizable. This utility only makes sense for functions
    that accept an argument set and return an attribute set.
  </para>

  <para>
    Example usage:

    <programlisting>f = { a, b }: { result = a+b; }
c = lib.makeOverridable f { a = 1; b = 2; }</programlisting>

  </para>

  <para>
    The variable <varname>c</varname> is the value of the <varname>f</varname> function
    applied with some default arguments. Hence the value of <varname>c.result</varname>
    is <literal>3</literal>, in this example.
  </para>

  <para>
    The variable <varname>c</varname> however also has some additional functions, like
    <link linkend="sec-pkg-override">c.override</link> which can be used to
    override the default arguments. In this example the value of
    <varname>(c.override { a = 4; }).result</varname> is 6.
  </para>

</section>


<section xml:id="sec-fhs-environments">
  <title>buildFHSChrootEnv/buildFHSUserEnv</title>

  <para>
    <function>buildFHSChrootEnv</function> and
    <function>buildFHSUserEnv</function> provide a way to build and run
    FHS-compatible lightweight sandboxes. They get their own isolated root with
    binded <filename>/nix/store</filename>, so their footprint in terms of disk
    space needed is quite small. This allows one to run software which is hard or
    unfeasible to patch for NixOS -- 3rd-party source trees with FHS assumptions,
    games distributed as tarballs, software with integrity checking and/or external
    self-updated binaries.
  </para>

  <para>
    <function>buildFHSChrootEnv</function> allows to create persistent
    environments, which can be constructed, deconstructed and entered by
    multiple users at once. A downside is that it requires
    <literal>root</literal> access for both those who create and destroy and
    those who enter it. It can be useful to create environments for daemons that
    one can enter and observe.
  </para>

  <para>
    <function>buildFHSUserEnv</function> uses Linux namespaces feature to create
    temporary lightweight environments which are destroyed after all child
    processes exit. It does not require root access, and can be useful to create
    sandboxes and wrap applications.
  </para>

  <para>
    Those functions both rely on <function>buildFHSEnv</function>, which creates
    an actual directory structure given a list of necessary packages and extra
    build commands.
    <function>buildFHSChrootEnv</function> and <function>buildFHSUserEnv</function>
    both accept those arguments which are passed to
    <function>buildFHSEnv</function>:
  </para>

  <variablelist>
    <varlistentry>
    <term><literal>name</literal></term>

    <listitem><para>Environment name.</para></listitem>
    </varlistentry>

    <varlistentry>
    <term><literal>targetPkgs</literal></term>

    <listitem><para>Packages to be installed for the main host's architecture
    (i.e. x86_64 on x86_64 installations).</para></listitem>
    </varlistentry>

    <varlistentry>
    <term><literal>multiPkgs</literal></term>

    <listitem><para>Packages to be installed for all architectures supported by
    a host (i.e. i686 and x86_64 on x86_64 installations).</para></listitem>
    </varlistentry>

    <varlistentry>
    <term><literal>extraBuildCommands</literal></term>

    <listitem><para>Additional commands to be executed for finalizing the
    directory structure.</para></listitem>
    </varlistentry>

    <varlistentry>
    <term><literal>extraBuildCommandsMulti</literal></term>

    <listitem><para>Like <literal>extraBuildCommandsMulti</literal>, but
    executed only on multilib architectures.</para></listitem>
    </varlistentry>
  </variablelist>

  <para>
    Additionally, <function>buildFHSUserEnv</function> accepts
    <literal>runScript</literal> parameter, which is a command that would be
    executed inside the sandbox and passed all the command line arguments. It
    default to <literal>bash</literal>.
  </para>
  <para>
    It also uses <literal>CHROOTENV_EXTRA_BINDS</literal> environment variable
    for binding extra directories in the sandbox to outside places. The format of
    the variable is <literal>/mnt=test-mnt:/data</literal>, where
    <literal>/mnt</literal> would be mounted as <literal>/test-mnt</literal>
    and <literal>/data</literal> would be mounted as <literal>/data</literal>.
    <literal>extraBindMounts</literal> array argument to
    <function>buildFHSUserEnv</function> function is prepended to this variable.
    Latter entries take priority if defined several times -- i.e. in case of
    <literal>/data=data1:/data=data2</literal> the actual bind path would be
    <literal>/data2</literal>.
  </para>
  <para>
    One can create a simple environment using a <literal>shell.nix</literal>
    like that:
  </para>

<programlisting><![CDATA[
{ pkgs ? import <nixpkgs> {} }:

(pkgs.buildFHSUserEnv {
  name = "simple-x11-env";
  targetPkgs = pkgs: (with pkgs;
    [ udev
      alsaLib
    ]) ++ (with pkgs.xorg;
    [ libX11
      libXcursor
      libXrandr
    ]);
  multiPkgs = pkgs: (with pkgs;
    [ udev
      alsaLib
    ]);
  runScript = "bash";
}).env
]]></programlisting>

  <para>
    Running <literal>nix-shell</literal> would then drop you into a shell with
    these libraries and binaries available. You can use this to run
    closed-source applications which expect FHS structure without hassles:
    simply change <literal>runScript</literal> to the application path,
    e.g. <filename>./bin/start.sh</filename> -- relative paths are supported.
  </para>
</section>

<section xml:id="sec-pkgs-dockerTools">
 <title>pkgs.dockerTools</title>

 <para>
  <varname>pkgs.dockerTools</varname> is a set of functions for creating and
  manipulating Docker images according to the
  <link xlink:href="https://github.com/docker/docker/blob/master/image/spec/v1.md#docker-image-specification-v100">
   Docker Image Specification v1.0.0
  </link>. Docker itself is not used to perform any of the operations done by these
  functions.
 </para>

 <warning>
  <para>
   The <varname>dockerTools</varname> API is unstable and may be subject to
   backwards-incompatible changes in the future.
  </para>
 </warning>

 <section xml:id="ssec-pkgs-dockerTools-buildImage">
  <title>buildImage</title>

  <para>
   This function is analogous to the <command>docker build</command> command,
   in that can used to build a Docker-compatible repository tarball containing
   a single image with one or multiple layers. As such, the result
   is suitable for being loaded in Docker with <command>docker load</command>.
  </para>

  <para>
   The parameters of <varname>buildImage</varname> with relative example values are
   described below:
  </para>

  <example xml:id='ex-dockerTools-buildImage'><title>Docker build</title>
  <programlisting>
  buildImage {
    name = "redis"; <co xml:id='ex-dockerTools-buildImage-1' />
    tag = "latest"; <co xml:id='ex-dockerTools-buildImage-2' />
    
    fromImage = someBaseImage; <co xml:id='ex-dockerTools-buildImage-3' />
    fromImageName = null; <co xml:id='ex-dockerTools-buildImage-4' />
    fromImageTag = "latest"; <co xml:id='ex-dockerTools-buildImage-5' />
    
    contents = pkgs.redis; <co xml:id='ex-dockerTools-buildImage-6' />
    runAsRoot = '' <co xml:id='ex-dockerTools-buildImage-runAsRoot' />
      #!${stdenv.shell}
      mkdir -p /data
    '';

    config = { <co xml:id='ex-dockerTools-buildImage-8' />
      Cmd = [ "/bin/redis-server" ];
      WorkingDir = "/data";
      Volumes = {
        "/data" = {};
      };
    };
  }
  </programlisting>
  </example>

  <para>The above example will build a Docker image <literal>redis/latest</literal>
   from the given base image. Loading and running this image in Docker results in
   <literal>redis-server</literal> being started automatically.
  </para>

  <calloutlist>
   <callout arearefs='ex-dockerTools-buildImage-1'>
    <para>
     <varname>name</varname> specifies the name of the resulting image.
     This is the only required argument for <varname>buildImage</varname>.
    </para>
   </callout>

   <callout arearefs='ex-dockerTools-buildImage-2'>
    <para>
     <varname>tag</varname> specifies the tag of the resulting image.
     By default it's <literal>latest</literal>.
    </para>
   </callout>

   <callout arearefs='ex-dockerTools-buildImage-3'>
    <para>
     <varname>fromImage</varname> is the repository tarball containing the base image.
     It must be a valid Docker image, such as exported by <command>docker save</command>.
     By default it's <literal>null</literal>, which can be seen as equivalent
     to <literal>FROM scratch</literal> of a <filename>Dockerfile</filename>.
    </para>
   </callout>
   
   <callout arearefs='ex-dockerTools-buildImage-4'>
    <para>
     <varname>fromImageName</varname> can be used to further specify
     the base image within the repository, in case it contains multiple images.
     By default it's <literal>null</literal>, in which case
     <varname>buildImage</varname> will peek the first image available
     in the repository.
    </para>
   </callout>

   <callout arearefs='ex-dockerTools-buildImage-5'>
    <para>
     <varname>fromImageTag</varname> can be used to further specify the tag
     of the base image within the repository, in case an image contains multiple tags.
     By default it's <literal>null</literal>, in which case
     <varname>buildImage</varname> will peek the first tag available for the base image.
    </para>
   </callout>

   <callout arearefs='ex-dockerTools-buildImage-6'>
    <para>
     <varname>contents</varname> is a derivation that will be copied in the new
     layer of the resulting image. This can be similarly seen as
     <command>ADD contents/ /</command> in a <filename>Dockerfile</filename>.
     By default it's <literal>null</literal>.
    </para>
   </callout>

   <callout arearefs='ex-dockerTools-buildImage-runAsRoot'>
    <para>
     <varname>runAsRoot</varname> is a bash script that will run as root
     in an environment that overlays the existing layers of the base image with
     the new resulting layer, including the previously copied
     <varname>contents</varname> derivation.
     This can be similarly seen as
     <command>RUN ...</command> in a <filename>Dockerfile</filename>.
     
     <note>
      <para>
       Using this parameter requires the <literal>kvm</literal>
       device to be available.
      </para>
     </note>
    </para>
   </callout>

   <callout arearefs='ex-dockerTools-buildImage-8'>
    <para>
     <varname>config</varname> is used to specify the configuration of the
     containers that will be started off the built image in Docker.
     The available options are listed in the
     <link xlink:href="https://github.com/docker/docker/blob/master/image/spec/v1.md#container-runconfig-field-descriptions">
      Docker Image Specification v1.0.0
     </link>.
    </para>
   </callout>

  </calloutlist>

  <para>
   After the new layer has been created, its closure
   (to which <varname>contents</varname>, <varname>config</varname> and
   <varname>runAsRoot</varname> contribute) will be copied in the layer itself.
   Only new dependencies that are not already in the existing layers will be copied.
  </para>

  <para>
   At the end of the process, only one new single layer will be produced and
   added to the resulting image.
  </para>

  <para>
   The resulting repository will only list the single image
   <varname>image/tag</varname>. In the case of <xref linkend='ex-dockerTools-buildImage'/>
   it would be <varname>redis/latest</varname>.
  </para>

  <para>
   It is possible to inspect the arguments with which an image was built
   using its <varname>buildArgs</varname> attribute.
  </para>

 </section>

 <section xml:id="ssec-pkgs-dockerTools-fetchFromRegistry">
  <title>pullImage</title>

  <para>
   This function is analogous to the <command>docker pull</command> command,
   in that can be used to fetch a Docker image from a Docker registry.
   Currently only registry <literal>v1</literal> is supported.
   By default <link xlink:href="https://hub.docker.com/">Docker Hub</link>
   is used to pull images.
  </para>

  <para>
   Its parameters are described in the example below:
  </para>

  <example xml:id='ex-dockerTools-pullImage'><title>Docker pull</title>
  <programlisting>
  pullImage {
    imageName = "debian"; <co xml:id='ex-dockerTools-pullImage-1' />
    imageTag = "jessie"; <co xml:id='ex-dockerTools-pullImage-2' />
    imageId = null; <co xml:id='ex-dockerTools-pullImage-3' />
    sha256 = "1bhw5hkz6chrnrih0ymjbmn69hyfriza2lr550xyvpdrnbzr4gk2"; <co xml:id='ex-dockerTools-pullImage-4' />

    indexUrl = "https://index.docker.io"; <co xml:id='ex-dockerTools-pullImage-5' />
    registryUrl = "https://registry-1.docker.io";
    registryVersion = "v1";
  }
  </programlisting>
  </example>

  <calloutlist>
   <callout arearefs='ex-dockerTools-pullImage-1'>
    <para>
     <varname>imageName</varname> specifies the name of the image to be downloaded,
     which can also include the registry namespace (e.g. <literal>library/debian</literal>).
     This argument is required.
    </para>
   </callout>
   
   <callout arearefs='ex-dockerTools-pullImage-2'>
    <para>
     <varname>imageTag</varname> specifies the tag of the image to be downloaded.
     By default it's <literal>latest</literal>.
    </para>
   </callout>

   <callout arearefs='ex-dockerTools-pullImage-3'>
    <para>
     <varname>imageId</varname>, if specified this exact image will be fetched, instead
     of <varname>imageName/imageTag</varname>. However, the resulting repository
     will still be named <varname>imageName/imageTag</varname>.
     By default it's <literal>null</literal>.
    </para>
   </callout>

   <callout arearefs='ex-dockerTools-pullImage-4'>
    <para>
     <varname>sha256</varname> is the checksum of the whole fetched image.
     This argument is required.
    </para>

    <note>
     <para>The checksum is computed on the unpacked directory, not on the final tarball.</para>
    </note>

   </callout>

   <callout arearefs='ex-dockerTools-pullImage-5'>
    <para>
     In the above example the default values are shown for the variables <varname>indexUrl</varname>,
     <varname>registryUrl</varname> and <varname>registryVersion</varname>.
     Hence by default the Docker.io registry is used to pull the images.
    </para>
   </callout>
  </calloutlist>
   
 </section>
  
 <section xml:id="ssec-pkgs-dockerTools-exportImage">
  <title>exportImage</title>

  <para>
   This function is analogous to the <command>docker export</command> command,
   in that can used to flatten a Docker image that contains multiple layers.
   It is in fact the result of the merge of all the layers of the image.
   As such, the result is suitable for being imported in Docker
   with <command>docker import</command>.
  </para>

  <note>
   <para>
    Using this function requires the <literal>kvm</literal>
    device to be available.
   </para>
  </note>

  <para>
   The parameters of <varname>exportImage</varname> are the following:
  </para>

  <example xml:id='ex-dockerTools-exportImage'><title>Docker export</title>
  <programlisting>
  exportImage {
    fromImage = someLayeredImage;
    fromImageName = null;
    fromImageTag = null;
    
    name = someLayeredImage.name;
  }
  </programlisting>
  </example>

  <para>
   The parameters relative to the base image have the same synopsis as
   described in <xref linkend='ssec-pkgs-dockerTools-buildImage'/>, except that
   <varname>fromImage</varname> is the only required argument in this case.
  </para>

  <para>
   The <varname>name</varname> argument is the name of the derivation output,
   which defaults to <varname>fromImage.name</varname>.
  </para>
 </section>

 <section xml:id="ssec-pkgs-dockerTools-shadowSetup">
  <title>shadowSetup</title>

  <para>
   This constant string is a helper for setting up the base files for managing
   users and groups, only if such files don't exist already.
   It is suitable for being used in a
   <varname>runAsRoot</varname> <xref linkend='ex-dockerTools-buildImage-runAsRoot'/> script for cases like
   in the example below:
  </para>
  
  <example xml:id='ex-dockerTools-shadowSetup'><title>Shadow base files</title>
  <programlisting>
  buildImage {
    name = "shadow-basic";

    runAsRoot = ''
      #!${stdenv.shell}
      ${shadowSetup}
      groupadd -r redis
      useradd -r -g redis redis
      mkdir /data
      chown redis:redis /data
    '';
  }
  </programlisting>
  </example>

  <para>
   Creating base files like <literal>/etc/passwd</literal> or
   <literal>/etc/login.defs</literal> are necessary for shadow-utils to
   manipulate users and groups.
  </para>
  
 </section>
 
</section>

</chapter>