diff options
Diffstat (limited to 'doc')
37 files changed, 1701 insertions, 626 deletions
diff --git a/doc/Makefile b/doc/Makefile index cdef493502b..49f361ebb60 100644 --- a/doc/Makefile +++ b/doc/Makefile @@ -1,4 +1,4 @@ -MD_TARGETS=$(addsuffix .xml, $(basename $(wildcard ./*.md ./**/*.md))) +MD_TARGETS=$(addsuffix .xml, $(basename $(shell find . -type f -regex '.*\.md$$'))) .PHONY: all all: validate format out/html/index.html out/epub/manual.epub diff --git a/doc/builders/images/appimagetools.xml b/doc/builders/images/appimagetools.xml index 0767a509a43..45c5619abd9 100644 --- a/doc/builders/images/appimagetools.xml +++ b/doc/builders/images/appimagetools.xml @@ -63,7 +63,7 @@ type2.AppImage: ELF 64-bit LSB executable, x86-64, version 1 (SYSV) (Lepton 3.x) appimageTools.wrapType2 { # or wrapType1 name = "patchwork"; <co xml:id='ex-appimageTools-wrapping-1' /> src = fetchurl { <co xml:id='ex-appimageTools-wrapping-2' /> - url = https://github.com/ssbc/patchwork/releases/download/v3.11.4/Patchwork-3.11.4-linux-x86_64.AppImage; + url = "https://github.com/ssbc/patchwork/releases/download/v3.11.4/Patchwork-3.11.4-linux-x86_64.AppImage"; sha256 = "1blsprpkvm0ws9b96gb36f0rbf8f5jgmw4x6dsb1kswr4ysf591s"; }; extraPkgs = pkgs: with pkgs; [ ]; <co xml:id='ex-appimageTools-wrapping-3' /> diff --git a/doc/builders/images/dockertools.xml b/doc/builders/images/dockertools.xml index e7f37fdaaf0..126698d0a9e 100644 --- a/doc/builders/images/dockertools.xml +++ b/doc/builders/images/dockertools.xml @@ -166,7 +166,7 @@ hello latest de2bf4786de6 About a minute ago 25.2MB <title>buildLayeredImage</title> <para> - Create a Docker image with many of the store paths being on their own layer to improve sharing between images. + Create a Docker image with many of the store paths being on their own layer to improve sharing between images. The image is realized into the Nix store as a gzipped tarball. Depending on the intended usage, many users might prefer to use <function>streamLayeredImage</function> instead, which this function uses internally. </para> <variablelist> @@ -327,6 +327,27 @@ pkgs.dockerTools.buildLayeredImage { </section> </section> + <section xml:id="ssec-pkgs-dockerTools-streamLayeredImage"> + <title>streamLayeredImage</title> + + <para> + Builds a script which, when run, will stream an uncompressed tarball of a Docker image to stdout. The arguments to this function are as for <function>buildLayeredImage</function>. This method of constructing an image does not realize the image into the Nix store, so it saves on IO and disk/cache space, particularly with large images. + </para> + + <para> + The image produced by running the output script can be piped directly into <command>docker load</command>, to load it into the local docker daemon: + <screen><![CDATA[ +$(nix-build) | docker load + ]]></screen> + </para> + <para> + Alternatively, the image be piped via <command>gzip</command> into <command>skopeo</command>, e.g. to copy it into a registry: + <screen><![CDATA[ +$(nix-build) | gzip --fast | skopeo copy docker-archive:/dev/stdin docker://some_docker_registry/myimage:tag + ]]></screen> + </para> + </section> + <section xml:id="ssec-pkgs-dockerTools-fetchFromRegistry"> <title>pullImage</title> diff --git a/doc/builders/packages/cataclysm-dda.section.md b/doc/builders/packages/cataclysm-dda.section.md new file mode 100644 index 00000000000..ae2ee56a010 --- /dev/null +++ b/doc/builders/packages/cataclysm-dda.section.md @@ -0,0 +1,94 @@ +# Cataclysm: Dark Days Ahead + +## How to install Cataclysm DDA + +To install the latest stable release of Cataclysm DDA to your profile, execute +`nix-env -f "<nixpkgs>" -iA cataclysm-dda`. For the curses build (build +without tiles), install `cataclysmDDA.stable.curses`. Note: `cataclysm-dda` is +an alias to `cataclysmDDA.stable.tiles`. + +If you like access to a development build of your favorite git revision, +override `cataclysm-dda-git` (or `cataclysmDDA.git.curses` if you like curses +build): + +```nix +cataclysm-dda-git.override { + version = "YYYY-MM-DD"; + rev = "YOUR_FAVORITE_REVISION"; + sha256 = "CHECKSUM_OF_THE_REVISION"; +} +``` + +The sha256 checksum can be obtained by + +```sh +nix-prefetch-url --unpack "https://github.com/CleverRaven/Cataclysm-DDA/archive/${YOUR_FAVORITE_REVISION}.tar.gz" +``` + +The default configuration directory is `~/.cataclysm-dda`. If you prefer +`$XDG_CONFIG_HOME/cataclysm-dda`, override the derivation: + +```nix +cataclysm-dda.override { + useXdgDir = true; +} +``` + +## Customizing with mods + +To install Cataclysm DDA with mods of your choice, you can use `withMods` +attribute: + +```nix +cataclysm-dda.withMods (mods: with mods; [ + tileset.UndeadPeople +]) +``` + +All mods, soundpacks, and tilesets available in nixpkgs are found in +`cataclysmDDA.pkgs`. + +Here is an example to modify existing mods and/or add more mods not available +in nixpkgs: + +```nix +let + customMods = self: super: lib.recursiveUpdate super { + # Modify existing mod + tileset.UndeadPeople = super.tileset.UndeadPeople.overrideAttrs (old: { + # If you like to apply a patch to the tileset for example + patches = [ ./path/to/your.patch ]; + }); + + # Add another mod + mod.Awesome = cataclysmDDA.buildMod { + modName = "Awesome"; + version = "0.x"; + src = fetchFromGitHub { + owner = "Someone"; + repo = "AwesomeMod"; + rev = "..."; + sha256 = "..."; + }; + # Path to be installed in the unpacked source (default: ".") + modRoot = "contents/under/this/path/will/be/installed"; + }; + + # Add another soundpack + soundpack.Fantastic = cataclysmDDA.buildSoundPack { + # ditto + }; + + # Add another tileset + tileset.SuperDuper = cataclysmDDA.buildTileSet { + # ditto + }; + }; +in +cataclysm-dda.withMods (mods: with mods.extend customMods; [ + tileset.UndeadPeople + mod.Awesome + soundpack.Fantastic + tileset.SuperDuper +]) +``` diff --git a/doc/builders/packages/citrix.xml b/doc/builders/packages/citrix.xml index c629dc9ee50..16f1bc6f8f2 100644 --- a/doc/builders/packages/citrix.xml +++ b/doc/builders/packages/citrix.xml @@ -4,34 +4,36 @@ <title>Citrix Workspace</title> <para> - <note> - <para> - Please note that the <literal>citrix_receiver</literal> package has been deprecated since its development was <link xlink:href="https://docs.citrix.com/en-us/citrix-workspace-app.html">discontinued by upstream</link> and has been replaced by <link xlink:href="https://www.citrix.com/products/workspace-app/">the citrix workspace app</link>. - </para> - </note> - <link xlink:href="https://www.citrix.com/products/receiver/">Citrix Receiver</link> and <link xlink:href="https://www.citrix.com/products/workspace-app/">Citrix Workspace App</link> are a remote desktop viewers which provide access to <link xlink:href="https://www.citrix.com/products/xenapp-xendesktop/">XenDesktop</link> installations. + The <link xlink:href="https://www.citrix.com/products/workspace-app/">Citrix Workspace App</link> is a remote desktop viewer which provides access to <link xlink:href="https://www.citrix.com/products/xenapp-xendesktop/">XenDesktop</link> installations. </para> <section xml:id="sec-citrix-base"> <title>Basic usage</title> <para> - The tarball archive needs to be downloaded manually as the license agreements of the vendor for <link xlink:href="https://www.citrix.com/downloads/citrix-receiver/">Citrix Receiver</link> or <link xlink:href="https://www.citrix.de/downloads/workspace-app/linux/workspace-app-for-linux-latest.html">Citrix Workspace</link> need to be accepted first. Then run <command>nix-prefetch-url file://$PWD/linuxx64-$version.tar.gz</command>. With the archive available in the store the package can be built and installed with Nix. + The tarball archive needs to be downloaded manually as the license agreements of the vendor for <link xlink:href="https://www.citrix.de/downloads/workspace-app/linux/workspace-app-for-linux-latest.html">Citrix Workspace</link> needs to be accepted first. Then run <command>nix-prefetch-url file://$PWD/linuxx64-$version.tar.gz</command>. With the archive available in the store the package can be built and installed with Nix. </para> + </section> - <warning> - <title>Caution with <command>nix-shell</command> installs</title> - <para> - It's recommended to install <literal>Citrix Receiver</literal> and/or <literal>Citrix Workspace</literal> using <literal>nix-env -i</literal> or globally to ensure that the <literal>.desktop</literal> files are installed properly into <literal>$XDG_CONFIG_DIRS</literal>. Otherwise it won't be possible to open <literal>.ica</literal> files automatically from the browser to start a Citrix connection. - </para> - </warning> + <section xml:id="sec-citrix-selfservice"> + <title>Citrix Selfservice</title> + <para> + The <link xlink:href="https://support.citrix.com/article/CTX200337">selfservice</link> is an application managing Citrix desktops and applications. Please note that this feature only works with at least <package>citrix_workspace_20_06_0</package> and later versions. + </para> + <para> + In order to set this up, you first have to <link xlink:href="https://its.uiowa.edu/support/article/102186">download the <literal>.cr</literal> file from the Netscaler Gateway</link>. After that you can configure the <command>selfservice</command> like this: + <screen> + <prompt>$ </prompt>storebrowse -C ~/Downloads/receiverconfig.cr + <prompt>$ </prompt>selfservice + </screen> + </para> </section> <section xml:id="sec-citrix-custom-certs"> <title>Custom certificates</title> <para> - The <literal>Citrix Workspace App</literal> in <literal>nixpkgs</literal> trust several certificates <link xlink:href="https://curl.haxx.se/docs/caextract.html">from the Mozilla database</link> by default. However several companies using Citrix might require their own corporate certificate. On distros with imperative packaging these certs can be stored easily in <link xlink:href="https://developer-docs.citrix.com/projects/receiver-for-linux-command-reference/en/13.7/"><literal>$ICAROOT</literal></link>, however this directory is a store path in <literal>nixpkgs</literal>. In order to work around this issue the package provides a simple mechanism to add custom certificates without rebuilding the entire package using <literal>symlinkJoin</literal>: + The <literal>Citrix Workspace App</literal> in <literal>nixpkgs</literal> trusts several certificates <link xlink:href="https://curl.haxx.se/docs/caextract.html">from the Mozilla database</link> by default. However several companies using Citrix might require their own corporate certificate. On distros with imperative packaging these certs can be stored easily in <link xlink:href="https://developer-docs.citrix.com/projects/receiver-for-linux-command-reference/en/13.7/"><literal>$ICAROOT</literal></link>, however this directory is a store path in <literal>nixpkgs</literal>. In order to work around this issue the package provides a simple mechanism to add custom certificates without rebuilding the entire package using <literal>symlinkJoin</literal>: <programlisting> <![CDATA[with import <nixpkgs> { config.allowUnfree = true; }; let extraCerts = [ ./custom-cert-1.pem ./custom-cert-2.pem /* ... */ ]; in diff --git a/doc/builders/packages/index.xml b/doc/builders/packages/index.xml index 4e109bd1c59..e20b0c689a8 100644 --- a/doc/builders/packages/index.xml +++ b/doc/builders/packages/index.xml @@ -18,6 +18,7 @@ <xi:include href="opengl.xml" /> <xi:include href="shell-helpers.xml" /> <xi:include href="steam.xml" /> + <xi:include href="cataclysm-dda.section.xml" /> <xi:include href="urxvt.xml" /> <xi:include href="weechat.xml" /> <xi:include href="xorg.xml" /> diff --git a/doc/builders/packages/steam.xml b/doc/builders/packages/steam.xml index 8dfede59ac1..59673328bac 100644 --- a/doc/builders/packages/steam.xml +++ b/doc/builders/packages/steam.xml @@ -45,13 +45,7 @@ <title>How to play</title> <para> - For 64-bit systems it's important to have -<programlisting>hardware.opengl.driSupport32Bit = true;</programlisting> - in your <filename>/etc/nixos/configuration.nix</filename>. You'll also need -<programlisting>hardware.pulseaudio.support32Bit = true;</programlisting> - if you are using PulseAudio - this will enable 32bit ALSA apps integration. To use the Steam controller or other Steam supported controllers such as the DualShock 4 or Nintendo Switch Pro, you need to add -<programlisting>hardware.steam-hardware.enable = true;</programlisting> - to your configuration. + Use <programlisting>programs.steam.enable = true;</programlisting> if you want to add steam to systemPackages and also enable a few workarrounds aswell as Steam controller support or other Steam supported controllers such as the DualShock 4 or Nintendo Switch Pr. </para> </section> diff --git a/doc/builders/packages/urxvt.xml b/doc/builders/packages/urxvt.xml index f85680cecc4..135cc82a0b5 100644 --- a/doc/builders/packages/urxvt.xml +++ b/doc/builders/packages/urxvt.xml @@ -50,7 +50,7 @@ map (p: p.name) pkgs.rxvt-unicode.plugins In addition to <literal>plugins</literal> the options <literal>extraDeps</literal> and <literal>perlDeps</literal> can be used to install extra packages. - <literal>extraDeps</literal> can be used, for example, to provide + <literal>extraDeps</literal> can be used, for example, to provide <literal>xsel</literal> (a clipboard manager) to the clipboard plugin, without installing it globally: <programlisting>rxvt-unicode.override { configure = { availablePlugins, ... }: { diff --git a/doc/contributing/quick-start.xml b/doc/contributing/quick-start.xml index 8e4e914d451..09d60834ec2 100644 --- a/doc/contributing/quick-start.xml +++ b/doc/contributing/quick-start.xml @@ -111,7 +111,7 @@ </para> <para> The exact syntax and semantics of the Nix expression language, including the built-in function, are described in the Nix manual in the <link - xlink:href="http://hydra.nixos.org/job/nix/trunk/tarball/latest/download-by-type/doc/manual/#chap-writing-nix-expressions">chapter on writing Nix expressions</link>. + xlink:href="https://hydra.nixos.org/job/nix/trunk/tarball/latest/download-by-type/doc/manual/#chap-writing-nix-expressions">chapter on writing Nix expressions</link>. </para> </listitem> <listitem> diff --git a/doc/contributing/submitting-changes.xml b/doc/contributing/submitting-changes.xml index 73222c1a49a..a88965f5cc6 100644 --- a/doc/contributing/submitting-changes.xml +++ b/doc/contributing/submitting-changes.xml @@ -407,23 +407,47 @@ Additional information. <section xml:id="submitting-changes-stable-release-branches"> <title>Stable release branches</title> - <itemizedlist> + <para> + For cherry-picking a commit to a stable release branch (<quote>backporting</quote>), use <literal>git cherry-pick -x <original commit></literal> so that the original commit id is included in the commit. + </para> + + <para> + Add a reason for the backport by using <literal>git cherry-pick -xe <original commit></literal> instead when it is not obvious from the original commit message. It is not needed when it’s a minor version update that includes security and bug fixes but don’t add new features or when the commit fixes an otherwise broken package. + </para> + + <para> + Here is an example of a cherry-picked commit message with good reason description: + </para> + +<screen> +zfs: Keep trying root import until it works + +Works around #11003. + +(cherry picked from commit 98b213a11041af39b39473906b595290e2a4e2f9) + +Reason: several people cannot boot with ZFS on NVMe +</screen> + + <para> + Other examples of reasons are: + </para> + + <itemizedlist spacing="compact"> <listitem> <para> - If you're cherry-picking a commit to a stable release branch (“backporting”), always use <command>git cherry-pick -xe</command> and ensure the message contains a clear description about why this needs to be included in the stable branch. + Previously the build would fail due to, e.g., <literal>getaddrinfo</literal> not being defined </para> + </listitem> + <listitem> <para> - An example of a cherry-picked commit would look like this: + The previous download links were all broken + </para> + </listitem> + <listitem> + <para> + Crash when starting on some X11 systems </para> -<screen> -nixos: Refactor the world. - -The original commit message describing the reason why the world was torn apart. - -(cherry picked from commit abcdef) -Reason: I just had a gut feeling that this would also be wanted by people from -the stone age. -</screen> </listitem> </itemizedlist> </section> diff --git a/doc/languages-frameworks/agda.section.md b/doc/languages-frameworks/agda.section.md new file mode 100644 index 00000000000..9ce046d05b6 --- /dev/null +++ b/doc/languages-frameworks/agda.section.md @@ -0,0 +1,106 @@ +--- +title: Agda +author: Alex Rice (alexarice) +date: 2020-01-06 +--- +# Agda + +## How to use Agda + +Agda can be installed from `agda`: +``` +$ nix-env -iA agda +``` + +To use agda with libraries, the `agda.withPackages` function can be used. This function either takes: ++ A list of packages, ++ or a function which returns a list of packages when given the `agdaPackages` attribute set, ++ or an attribute set containing a list of packages and a GHC derivation for compilation (see below). + +For example, suppose we wanted a version of agda which has access to the standard library. This can be obtained with the expressions: + +``` +agda.withPackages [ agdaPackages.standard-library ] +``` + +or + +``` +agda.withPackages (p: [ p.standard-library ]) +``` + +or can be called as in the [Compiling Agda](#compiling-agda) section. + +If you want to use a library in your home directory (for instance if it is a development version) then typecheck it manually (using `agda.withPackages` if necessary) and then override the `src` attribute of the package to point to your local repository. + +Agda will not by default use these libraries. To tell agda to use the library we have some options: +- Call `agda` with the library flag: +``` +$ agda -l standard-library -i . MyFile.agda +``` +- Write a `my-library.agda-lib` file for the project you are working on which may look like: +``` +name: my-library +include: . +depend: standard-library +``` +- Create the file `~/.agda/defaults` and add any libraries you want to use by default. + +More information can be found in the [official Agda documentation on library management](https://agda.readthedocs.io/en/v2.6.1/tools/package-system.html). + +## Compiling Agda +Agda modules can be compiled with the `--compile` flag. A version of `ghc` with `ieee` is made available to the Agda program via the `--with-compiler` flag. +This can be overridden by a different version of `ghc` as follows: + +``` +agda.withPackages { + pkgs = [ ... ]; + ghc = haskell.compiler.ghcHEAD; +} +``` + +## Writing Agda packages +To write a nix derivation for an agda library, first check that the library has a `*.agda-lib` file. + +A derivation can then be written using `agdaPackages.mkDerivation`. This has similar arguments to `stdenv.mkDerivation` with the following additions: ++ `everythingFile` can be used to specify the location of the `Everything.agda` file, defaulting to `./Everything.agda`. If this file does not exist then either it should be patched in or the `buildPhase` should be overridden (see below). ++ `libraryName` should be the name that appears in the `*.agda-lib` file, defaulting to `pname`. ++ `libraryFile` should be the file name of the `*.agda-lib` file, defaulting to `${libraryName}.agda-lib`. + +### Building Agda packages +The default build phase for `agdaPackages.mkDerivation` simply runs `agda` on the `Everything.agda` file. +If something else is needed to build the package (e.g. `make`) then the `buildPhase` should be overridden. +Additionally, a `preBuild` or `configurePhase` can be used if there are steps that need to be done prior to checking the `Everything.agda` file. +`agda` and the Agda libraries contained in `buildInputs` are made available during the build phase. + +### Installing Agda packages +The default install phase copies agda source files, agda interface files (`*.agdai`) and `*.agda-lib` files to the output directory. +This can be overridden. + +By default, agda sources are files ending on `.agda`, or literate agda files ending on `.lagda`, `.lagda.tex`, `.lagda.org`, `.lagda.md`, `.lagda.rst`. The list of recognised agda source extensions can be extended by setting the `extraExtensions` config variable. + +To add an agda package to `nixpkgs`, the derivation should be written to `pkgs/development/libraries/agda/${library-name}/` and an entry should be added to `pkgs/top-level/agda-packages.nix`. Here it is called in a scope with access to all other agda libraries, so the top line of the `default.nix` can look like: +``` +{ mkDerivation, standard-library, fetchFromGitHub }: +``` +and `mkDerivation` should be called instead of `agdaPackages.mkDerivation`. Here is an example skeleton derivation for iowa-stdlib: + +``` +mkDerivation { + version = "1.5.0"; + pname = "iowa-stdlib"; + + src = ... + + libraryFile = ""; + libraryName = "IAL-1.3"; + + buildPhase = '' + patchShebangs find-deps.sh + make + ''; +} +``` +This library has a file called `.agda-lib`, and so we give an empty string to `libraryFile` as nothing precedes `.agda-lib` in the filename. This file contains `name: IAL-1.3`, and so we let `libraryName = "IAL-1.3"`. This library does not use an `Everything.agda` file and instead has a Makefile, so there is no need to set `everythingFile` and we set a custom `buildPhase`. + +When writing an agda package it is essential to make sure that no `.agda-lib` file gets added to the store as a single file (for example by using `writeText`). This causes agda to think that the nix store is a agda library and it will attempt to write to it whenever it typechecks something. See [https://github.com/agda/agda/issues/4613](https://github.com/agda/agda/issues/4613). diff --git a/doc/languages-frameworks/android.section.md b/doc/languages-frameworks/android.section.md index 23547085b38..f4f6c086a80 100644 --- a/doc/languages-frameworks/android.section.md +++ b/doc/languages-frameworks/android.section.md @@ -167,7 +167,7 @@ parameters that the SDK composition function (the function shown in the previous section) supports. This build function is particularly useful when it is desired to use -[Hydra](http://nixos.org/hydra): the Nix-based continuous integration solution +[Hydra](https://nixos.org/hydra): the Nix-based continuous integration solution to build Android apps. An Android APK gets exposed as a build product and can be installed on any Android device with a web browser by navigating to the build result page. @@ -186,7 +186,7 @@ with import <nixpkgs> {}; androidenv.emulateApp { name = "emulate-MyAndroidApp"; platformVersion = "28"; - abiVersion = "x86_64"; # armeabi-v7a, mips, x86 + abiVersion = "x86"; # armeabi-v7a, mips, x86_64 systemImageType = "google_apis_playstore"; } ``` diff --git a/doc/languages-frameworks/beam.xml b/doc/languages-frameworks/beam.xml index 278535237c2..1d307e1d6dc 100644 --- a/doc/languages-frameworks/beam.xml +++ b/doc/languages-frameworks/beam.xml @@ -68,7 +68,7 @@ <title>How to Install BEAM Packages</title> <para> - BEAM builders are not registered at the top level, simply because they are not relevant to the vast majority of Nix users. + BEAM builders are not registered at the top level, simply because they are not relevant to the vast majority of Nix users. To install any of those builders into your profile, refer to them by their attribute path <literal>beamPackages.rebar3</literal>: </para> diff --git a/doc/languages-frameworks/emscripten.section.md b/doc/languages-frameworks/emscripten.section.md index 80e1094809a..3663f962d5f 100644 --- a/doc/languages-frameworks/emscripten.section.md +++ b/doc/languages-frameworks/emscripten.section.md @@ -21,11 +21,11 @@ Modes of use of `emscripten`: * **Declarative usage**: This mode is far more power full since this makes use of `nix` for dependency management of emscripten libraries and targets by using the `mkDerivation` which is implemented by `pkgs.emscriptenStdenv` and `pkgs.buildEmscriptenPackage`. The source for the packages is in `pkgs/top-level/emscripten-packages.nix` and the abstraction behind it in `pkgs/development/em-modules/generic/default.nix`. - * build and install all packages: - * `nix-env -iA emscriptenPackages` - - * dev-shell for zlib implementation hacking: - * `nix-shell -A emscriptenPackages.zlib` + * build and install all packages: + * `nix-env -iA emscriptenPackages` + + * dev-shell for zlib implementation hacking: + * `nix-shell -A emscriptenPackages.zlib` ## Imperative usage @@ -90,7 +90,7 @@ See the `zlib` example: libz.so.${old.version} -I . -o example.js echo "Using node to execute the test" - ${pkgs.nodejs}/bin/node ./example.js + ${pkgs.nodejs}/bin/node ./example.js set +x if [ $? -ne 0 ]; then @@ -112,7 +112,7 @@ See the `zlib` example: ### Usage 2: pkgs.buildEmscriptenPackage -This `xmlmirror` example features a emscriptenPackage which is defined completely from this context and no `pkgs.zlib.override` is used. +This `xmlmirror` example features a emscriptenPackage which is defined completely from this context and no `pkgs.zlib.override` is used. xmlmirror = pkgs.buildEmscriptenPackage rec { name = "xmlmirror"; @@ -163,7 +163,7 @@ This `xmlmirror` example features a emscriptenPackage which is defined completel checkPhase = '' ''; - }; + }; ### Declarative debugging @@ -182,4 +182,3 @@ Use `nix-shell -I nixpkgs=/some/dir/nixpkgs -A emscriptenPackages.libz` and from Using this toolchain makes it easy to leverage `nix` from NixOS, MacOSX or even Windows (WSL+ubuntu+nix). This toolchain is reproducible, behaves like the rest of the packages from nixpkgs and contains a set of well working examples to learn and adapt from. If in trouble, ask the maintainers. - diff --git a/doc/languages-frameworks/gnome.xml b/doc/languages-frameworks/gnome.xml index 7671714d8a9..159216ca981 100644 --- a/doc/languages-frameworks/gnome.xml +++ b/doc/languages-frameworks/gnome.xml @@ -40,6 +40,23 @@ </para> </section> + <section xml:id="ssec-icon-theme-packaging"> + <title>Packaging icon themes</title> + + <para> + Icon themes may inherit from other icon themes. The inheritance is specified using the <literal>Inherits</literal> key in the <filename>index.theme</filename> file distributed with the icon theme. According to the <link xlink:href="https://specifications.freedesktop.org/icon-theme-spec/icon-theme-spec-latest.html">icon theme specification</link>, icons not provided by the theme are looked for in its parent icon themes. Therefore the parent themes should be installed as dependencies for a more complete experience regarding the icon sets used. + </para> + + <para> + The package <package>hicolor-icon-theme</package> provides a setup hook which makes symbolic links for the parent themes into the directory <filename>share/icons</filename> of the current theme directory in the nix store, making sure they can be found at runtime. For that to work the packages providing parent icon themes should be listed as propagated build dependencies, together with <package>hicolor-icon-theme</package>. + </para> + + <para> + Also make sure that <filename>icon-theme.cache</filename> is installed for each theme provided by the package, and set <code>dontDropIconThemeCache</code> to <code>true</code> so that the cache file is not removed by the <package>gtk3</package> setup hook. + </para> + + </section> + <section xml:id="ssec-gnome-themes"> <title>GTK Themes</title> diff --git a/doc/languages-frameworks/go.xml b/doc/languages-frameworks/go.xml index 84b146532b1..26414f23c7d 100644 --- a/doc/languages-frameworks/go.xml +++ b/doc/languages-frameworks/go.xml @@ -36,10 +36,14 @@ pet = buildGoModule rec { sha256 = "0m2fzpqxk7hrbxsgqplkg7h2p7gv6s1miymv3gvw0cz039skag0s"; }; - modSha256 = "1879j77k96684wi554rkjxydrj8g3hpp0kvxz03sd8dmwr3lh83j"; <co xml:id='ex-buildGoModule-1' /> + vendorSha256 = "1879j77k96684wi554rkjxydrj8g3hpp0kvxz03sd8dmwr3lh83j"; <co xml:id='ex-buildGoModule-1' /> subPackages = [ "." ]; <co xml:id='ex-buildGoModule-2' /> + deleteVendor = true; <co xml:id='ex-buildGoModule-3' /> + + runVend = true; <co xml:id='ex-buildGoModule-4' /> + meta = with lib; { description = "Simple command-line snippet manager, written in Go"; homepage = "https://github.com/knqyf263/pet"; @@ -56,7 +60,7 @@ pet = buildGoModule rec { <calloutlist> <callout arearefs='ex-buildGoModule-1'> <para> - <varname>modSha256</varname> is the hash of the output of the intermediate fetcher derivation. + <varname>vendorSha256</varname> is the hash of the output of the intermediate fetcher derivation. </para> </callout> <callout arearefs='ex-buildGoModule-2'> @@ -64,16 +68,26 @@ pet = buildGoModule rec { <varname>subPackages</varname> limits the builder from building child packages that have not been listed. If <varname>subPackages</varname> is not specified, all child packages will be built. </para> </callout> + <callout arearefs='ex-buildGoModule-3'> + <para> + <varname>deleteVendor</varname> removes the pre-existing vendor directory and fetches the dependencies. This should only be used if the dependencies included in the vendor folder are broken or incomplete. + </para> + </callout> + <callout arearefs='ex-buildGoModule-4'> + <para> + <varname>runVend</varname> runs the vend command to generate the vendor directory. This is useful if your code depends on c code and go mod tidy does not include the needed sources to build. + </para> + </callout> </calloutlist> </para> <para> - <varname>modSha256</varname> can also take <varname>null</varname> as an input. + <varname>vendorSha256</varname> can also take <varname>null</varname> as an input. - When `null` is used as a value, the derivation won't be a - fixed-output derivation but disable the build sandbox instead. This can be useful outside - of nixpkgs where re-generating the modSha256 on each mod.sum changes is cumbersome, - but will fail to build by Hydra, as builds with a disabled sandbox are discouraged. + When `null` is used as a value, rather than fetching the dependencies + and vendoring them, we use the vendoring included within the source repo. + If you'd like to not have to update this field on dependency changes, + run `go mod vendor` in your source repo and set 'vendorSha256 = null;' </para> </section> @@ -192,18 +206,6 @@ deis = buildGoPackage rec { </para> <para> - <varname>buildGoPackage</varname> produces <xref linkend='chap-multiple-output' xrefstyle="select: title" /> where <varname>bin</varname> includes program binaries. You can test build a Go binary as follows: -<screen> -<prompt>$ </prompt>nix-build -A deis.bin -</screen> - or build all outputs with: -<screen> -<prompt>$ </prompt>nix-build -A deis.all -</screen> - <varname>bin</varname> output will be installed by default with <varname>nix-env -i</varname> or <varname>systemPackages</varname>. - </para> - - <para> You may use Go packages installed into the active Nix profiles by adding the following to your ~/.bashrc: <screen> for p in $NIX_PROFILES; do diff --git a/doc/languages-frameworks/haskell.section.md b/doc/languages-frameworks/haskell.section.md index 944c17a137e..cba4d0561b0 100644 --- a/doc/languages-frameworks/haskell.section.md +++ b/doc/languages-frameworks/haskell.section.md @@ -101,17 +101,15 @@ to compile your Haskell packages with any GHC version you please. The following command displays the complete list of available compilers: ``` $ nix-env -f "<nixpkgs>" -qaP -A haskell.compiler -haskell.compiler.ghc8101 ghc-8.10.0.20191210 -haskell.compiler.integer-simple.ghc8101 ghc-8.10.0.20191210 -haskell.compiler.ghcHEAD ghc-8.10.20191119 -haskell.compiler.integer-simple.ghcHEAD ghc-8.10.20191119 +haskell.compiler.ghc8101 ghc-8.10.1 +haskell.compiler.integer-simple.ghc8101 ghc-8.10.1 +haskell.compiler.ghcHEAD ghc-8.11.20200505 +haskell.compiler.integer-simple.ghcHEAD ghc-8.11.20200505 haskell.compiler.ghc822Binary ghc-8.2.2-binary haskell.compiler.ghc844 ghc-8.4.4 haskell.compiler.ghc863Binary ghc-8.6.3-binary haskell.compiler.ghc865 ghc-8.6.5 haskell.compiler.integer-simple.ghc865 ghc-8.6.5 -haskell.compiler.ghc881 ghc-8.8.1 -haskell.compiler.integer-simple.ghc881 ghc-8.8.1 haskell.compiler.ghc882 ghc-8.8.2 haskell.compiler.integer-simple.ghc882 ghc-8.8.2 haskell.compiler.ghc883 ghc-8.8.3 @@ -369,7 +367,7 @@ automatically select the right version of GHC and other build tools to build, test and execute apps in an existing project downloaded from somewhere on the Internet. Pass the `--nix` flag to any `stack` command to do so, e.g. ```shell -git clone --recursive https://github.com/yesodweb/wai +git clone --recurse-submodules https://github.com/yesodweb/wai.git cd wai stack --nix build ``` diff --git a/doc/languages-frameworks/index.xml b/doc/languages-frameworks/index.xml index 9364c764bbf..728a38c264a 100644 --- a/doc/languages-frameworks/index.xml +++ b/doc/languages-frameworks/index.xml @@ -5,6 +5,7 @@ <para> The <link linkend="chap-stdenv">standard build environment</link> makes it easy to build typical Autotools-based packages with very little code. Any other kind of package can be accomodated by overriding the appropriate phases of <literal>stdenv</literal>. However, there are specialised functions in Nixpkgs to easily build packages for other programming languages, such as Perl or Haskell. These are described in this chapter. </para> + <xi:include href="agda.section.xml" /> <xi:include href="android.section.xml" /> <xi:include href="beam.xml" /> <xi:include href="bower.xml" /> @@ -17,10 +18,11 @@ <xi:include href="idris.section.xml" /> <xi:include href="ios.section.xml" /> <xi:include href="java.xml" /> - <xi:include href="lua.xml" /> + <xi:include href="lua.section.xml" /> <xi:include href="node.section.xml" /> <xi:include href="ocaml.xml" /> <xi:include href="perl.xml" /> + <xi:include href="php.section.xml" /> <xi:include href="python.section.xml" /> <xi:include href="qt.xml" /> <xi:include href="r.section.xml" /> diff --git a/doc/languages-frameworks/ios.section.md b/doc/languages-frameworks/ios.section.md index 5a81795c004..768e0690b96 100644 --- a/doc/languages-frameworks/ios.section.md +++ b/doc/languages-frameworks/ios.section.md @@ -18,7 +18,7 @@ The primary objective of this project is to use the Nix expression language to specify how iOS apps can be built from source code, and to automatically spawn iOS simulator instances for testing. -This component also makes it possible to use [Hydra](http://nixos.org/hydra), +This component also makes it possible to use [Hydra](https://nixos.org/hydra), the Nix-based continuous integration server to regularly build iOS apps and to do wireless ad-hoc installations of enterprise IPAs on iOS devices through Hydra. diff --git a/doc/languages-frameworks/lua.section.md b/doc/languages-frameworks/lua.section.md new file mode 100644 index 00000000000..a0e9917b8ec --- /dev/null +++ b/doc/languages-frameworks/lua.section.md @@ -0,0 +1,252 @@ +--- +title: Lua +author: Matthieu Coudron +date: 2019-02-05 +--- + +# User's Guide to Lua Infrastructure + +## Using Lua + +### Overview of Lua + +Several versions of the Lua interpreter are available: luajit, lua 5.1, 5.2, 5.3. +The attribute `lua` refers to the default interpreter, it is also possible to refer to specific versions, e.g. `lua5_2` refers to Lua 5.2. + +Lua libraries are in separate sets, with one set per interpreter version. + +The interpreters have several common attributes. One of these attributes is +`pkgs`, which is a package set of Lua libraries for this specific +interpreter. E.g., the `busted` package corresponding to the default interpreter +is `lua.pkgs.busted`, and the lua 5.2 version is `lua5_2.pkgs.busted`. +The main package set contains aliases to these package sets, e.g. +`luaPackages` refers to `lua5_1.pkgs` and `lua52Packages` to +`lua5_2.pkgs`. + +### Installing Lua and packages + +#### Lua environment defined in separate `.nix` file + +Create a file, e.g. `build.nix`, with the following expression +```nix +with import <nixpkgs> {}; + +lua5_2.withPackages (ps: with ps; [ busted luafilesystem ]) +``` +and install it in your profile with +```shell +nix-env -if build.nix +``` +Now you can use the Lua interpreter, as well as the extra packages (`busted`, +`luafilesystem`) that you added to the environment. + +#### Lua environment defined in `~/.config/nixpkgs/config.nix` + +If you prefer to, you could also add the environment as a package override to the Nixpkgs set, e.g. +using `config.nix`, +```nix +{ # ... + + packageOverrides = pkgs: with pkgs; { + myLuaEnv = lua5_2.withPackages (ps: with ps; [ busted luafilesystem ]); + }; +} +``` +and install it in your profile with +```shell +nix-env -iA nixpkgs.myLuaEnv +``` +The environment is is installed by referring to the attribute, and considering +the `nixpkgs` channel was used. + +#### Lua environment defined in `/etc/nixos/configuration.nix` + +For the sake of completeness, here's another example how to install the environment system-wide. + +```nix +{ # ... + + environment.systemPackages = with pkgs; [ + (lua.withPackages(ps: with ps; [ busted luafilesystem ])) + ]; +} +``` + +### How to override a Lua package using overlays? + +Use the following overlay template: + +```nix +final: prev: +{ + + lua = prev.lua.override { + packageOverrides = luaself: luaprev: { + + luarocks-nix = luaprev.luarocks-nix.overrideAttrs(oa: { + pname = "luarocks-nix"; + src = /home/my_luarocks/repository; + }); + }; + + luaPackages = lua.pkgs; +} +``` + +### Temporary Lua environment with `nix-shell` + + +There are two methods for loading a shell with Lua packages. The first and recommended method +is to create an environment with `lua.buildEnv` or `lua.withPackages` and load that. E.g. +```sh +$ nix-shell -p 'lua.withPackages(ps: with ps; [ busted luafilesystem ])' +``` +opens a shell from which you can launch the interpreter +```sh +[nix-shell:~] lua +``` +The other method, which is not recommended, does not create an environment and requires you to list the packages directly, + +```sh +$ nix-shell -p lua.pkgs.busted lua.pkgs.luafilesystem +``` +Again, it is possible to launch the interpreter from the shell. +The Lua interpreter has the attribute `pkgs` which contains all Lua libraries for that specific interpreter. + + +## Developing with Lua + +Now that you know how to get a working Lua environment with Nix, it is time +to go forward and start actually developing with Lua. There are two ways to +package lua software, either it is on luarocks and most of it can be taken care +of by the luarocks2nix converter or the packaging has to be done manually. +Let's present the luarocks way first and the manual one in a second time. + +### Packaging a library on luarocks + +[Luarocks.org](www.luarocks.org) is the main repository of lua packages. +The site proposes two types of packages, the rockspec and the src.rock +(equivalent of a [rockspec](https://github.com/luarocks/luarocks/wiki/Rockspec-format) but with the source). +These packages can have different build types such as `cmake`, `builtin` etc . + +Luarocks-based packages are generated in pkgs/development/lua-modules/generated-packages.nix from +the whitelist maintainers/scripts/luarocks-packages.csv and updated by running maintainers/scripts/update-luarocks-packages. + +[luarocks2nix](https://github.com/nix-community/luarocks) is a tool capable of generating nix derivations from both rockspec and src.rock (and favors the src.rock). +The automation only goes so far though and some packages need to be customized. +These customizations go in `pkgs/development/lua-modules/overrides.nix`. +For instance if the rockspec defines `external_dependencies`, these need to be manually added in in its rockspec file then it won't work. + +You can try converting luarocks packages to nix packages with the command `nix-shell -p luarocks-nix` and then `luarocks nix PKG_NAME`. +Nix rely on luarocks to install lua packages, basically it runs: +`luarocks make --deps-mode=none --tree $out` + +#### Packaging a library manually + +You can develop your package as you usually would, just don't forget to wrap it +within a `toLuaModule` call, for instance +```nix +mynewlib = toLuaModule ( stdenv.mkDerivation { ... }); +``` + +There is also the `buildLuaPackage` function that can be used when lua modules +are not packaged for luarocks. You can see a few examples at `pkgs/top-level/lua-packages.nix`. + +## Lua Reference + +### Lua interpreters + +Versions 5.1, 5.2 and 5.3 of the lua interpreter are available as +respectively `lua5_1`, `lua5_2` and `lua5_3`. Luajit is available too. +The Nix expressions for the interpreters can be found in `pkgs/development/interpreters/lua-5`. + + +#### Attributes on lua interpreters packages + +Each interpreter has the following attributes: + +- `interpreter`. Alias for `${pkgs.lua}/bin/lua`. +- `buildEnv`. Function to build lua interpreter environments with extra packages bundled together. See section *lua.buildEnv function* for usage and documentation. +- `withPackages`. Simpler interface to `buildEnv`. +- `pkgs`. Set of Lua packages for that specific interpreter. The package set can be modified by overriding the interpreter and passing `packageOverrides`. + + +#### `buildLuarocksPackage` function + +The `buildLuarocksPackage` function is implemented in `pkgs/development/interpreters/lua-5/build-lua-package.nix` +The following is an example: +```nix +luaposix = buildLuarocksPackage { + pname = "luaposix"; + version = "34.0.4-1"; + + src = fetchurl { + url = "https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/luaposix-34.0.4-1.src.rock"; + sha256 = "0yrm5cn2iyd0zjd4liyj27srphvy0gjrjx572swar6zqr4dwjqp2"; + }; + disabled = (luaOlder "5.1") || (luaAtLeast "5.4"); + propagatedBuildInputs = [ bit32 lua std_normalize ]; + + meta = with stdenv.lib; { + homepage = "https://github.com/luaposix/luaposix/"; + description = "Lua bindings for POSIX"; + maintainers = with maintainers; [ vyp lblasc ]; + license.fullName = "MIT/X11"; + }; +}; +``` + +The `buildLuarocksPackage` delegates most tasks to luarocks: + +* it adds `luarocks` as an unpacker for `src.rock` files (zip files really). +* configurePhase` writes a temporary luarocks configuration file which location +is exported via the environment variable `LUAROCKS_CONFIG`. +* the `buildPhase` does nothing. +* `installPhase` calls `luarocks make --deps-mode=none --tree $out` to build and +install the package +* In the `postFixup` phase, the `wrapLuaPrograms` bash function is called to + wrap all programs in the `$out/bin/*` directory to include `$PATH` + environment variable and add dependent libraries to script's `LUA_PATH` and + `LUA_CPATH`. + +By default `meta.platforms` is set to the same value as the interpreter unless overridden otherwise. + +#### `buildLuaApplication` function + +The `buildLuaApplication` function is practically the same as `buildLuaPackage`. +The difference is that `buildLuaPackage` by default prefixes the names of the packages with the version of the interpreter. +Because with an application we're not interested in multiple version the prefix is dropped. + +#### lua.withPackages function + +The `lua.withPackages` takes a function as an argument that is passed the set of lua packages and returns the list of packages to be included in the environment. +Using the `withPackages` function, the previous example for the luafilesystem environment can be written like this: +```nix +with import <nixpkgs> {}; + +lua.withPackages (ps: [ps.luafilesystem]) +``` + +`withPackages` passes the correct package set for the specific interpreter version as an argument to the function. In the above example, `ps` equals `luaPackages`. +But you can also easily switch to using `lua5_2`: +```nix +with import <nixpkgs> {}; + +lua5_2.withPackages (ps: [ps.lua]) +``` + +Now, `ps` is set to `lua52Packages`, matching the version of the interpreter. + + +### Possible Todos + +* export/use version specific variables such as `LUA_PATH_5_2`/`LUAROCKS_CONFIG_5_2` +* let luarocks check for dependencies via exporting the different rocktrees in temporary config + +### Lua Contributing guidelines + +Following rules should be respected: + +* Make sure libraries build for all Lua interpreters. +* Commit names of Lua libraries should reflect that they are Lua libraries, so write for example `luaPackages.luafilesystem: 1.11 -> 1.12`. + diff --git a/doc/languages-frameworks/lua.xml b/doc/languages-frameworks/lua.xml deleted file mode 100644 index bcca6b73753..00000000000 --- a/doc/languages-frameworks/lua.xml +++ /dev/null @@ -1,36 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xml:id="sec-language-lua"> - <title>Lua</title> - - <para> - Lua packages are built by the <varname>buildLuaPackage</varname> function. This function is implemented in <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/lua-modules/generic/default.nix"> <filename>pkgs/development/lua-modules/generic/default.nix</filename></link> and works similarly to <varname>buildPerlPackage</varname>. (See <xref linkend="sec-language-perl"/> for details.) - </para> - - <para> - Lua packages are defined in <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/lua-packages.nix"><filename>pkgs/top-level/lua-packages.nix</filename></link>. Most of them are simple. For example: -<programlisting> -fileSystem = buildLuaPackage { - name = "filesystem-1.6.2"; - src = fetchurl { - url = "https://github.com/keplerproject/luafilesystem/archive/v1_6_2.tar.gz"; - sha256 = "1n8qdwa20ypbrny99vhkmx8q04zd2jjycdb5196xdhgvqzk10abz"; - }; - meta = { - homepage = "https://github.com/keplerproject/luafilesystem"; - hydraPlatforms = stdenv.lib.platforms.linux; - maintainers = with maintainers; [ flosse ]; - }; -}; -</programlisting> - </para> - - <para> - Though, more complicated package should be placed in a seperate file in <link - xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/lua-modules"><filename>pkgs/development/lua-modules</filename></link>. - </para> - - <para> - Lua packages accept additional parameter <varname>disabled</varname>, which defines the condition of disabling package from luaPackages. For example, if package has <varname>disabled</varname> assigned to <literal>lua.luaversion != "5.1"</literal>, it will not be included in any luaPackages except lua51Packages, making it only be built for lua 5.1. - </para> -</section> diff --git a/doc/languages-frameworks/node.section.md b/doc/languages-frameworks/node.section.md index 4dc95fc1dd9..c1f4294711a 100644 --- a/doc/languages-frameworks/node.section.md +++ b/doc/languages-frameworks/node.section.md @@ -12,10 +12,9 @@ When it is desired to use NPM libraries in a development project, use the `node2nix` generator directly on the `package.json` configuration file of the project. -The package set also provides support for multiple Node.js versions. The policy -is that a new package should be added to the collection for the latest stable LTS -release (which is currently 10.x), unless there is an explicit reason to support -a different release. +The package set provides support for the official stable Node.js versions. +The latest stable LTS release in `nodePackages`, as well as the latest stable +Current release in `nodePackages_latest`. If your package uses native addons, you need to examine what kind of native build system it uses. Here are some examples: @@ -26,7 +25,7 @@ build system it uses. Here are some examples: After you have identified the correct system, you need to override your package expression while adding in build system as a build input. For example, `dat` -requires `node-gyp-build`, so we override its expression in `default-v10.nix`: +requires `node-gyp-build`, so we override its expression in `default.nix`: ```nix dat = nodePackages.dat.override (oldAttrs: { @@ -36,14 +35,14 @@ dat = nodePackages.dat.override (oldAttrs: { To add a package from NPM to nixpkgs: - 1. Modify `pkgs/development/node-packages/node-packages-v10.json` to add, update - or remove package entries. (Or `pkgs/development/node-packages/node-packages-v8.json` - for packages depending on Node.js 8.x) + 1. Modify `pkgs/development/node-packages/node-packages.json` to add, update + or remove package entries to have it included in `nodePackages` and + `nodePackages_latest`. 2. Run the script: `(cd pkgs/development/node-packages && ./generate.sh)`. 3. Build your new package to test your changes: `cd /path/to/nixpkgs && nix-build -A nodePackages.<new-or-updated-package>`. - To build against a specific Node.js version (e.g. 10.x): - `nix-build -A nodePackages_10_x.<new-or-updated-package>` + To build against the latest stable Current Node.js version (e.g. 14.x): + `nix-build -A nodePackages_latest.<new-or-updated-package>` 4. Add and commit all modified and generated files. For more information about the generation process, consult the diff --git a/doc/languages-frameworks/perl.xml b/doc/languages-frameworks/perl.xml index d9b6b2721c6..ff0f350e99c 100644 --- a/doc/languages-frameworks/perl.xml +++ b/doc/languages-frameworks/perl.xml @@ -3,159 +3,193 @@ xml:id="sec-language-perl"> <title>Perl</title> - <para> - Nixpkgs provides a function <varname>buildPerlPackage</varname>, a generic package builder function for any Perl package that has a standard <varname>Makefile.PL</varname>. It’s implemented in <link -xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/perl-modules/generic"><filename>pkgs/development/perl-modules/generic</filename></link>. - </para> - - <para> - Perl packages from CPAN are defined in <link -xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/perl-packages.nix"><filename>pkgs/top-level/perl-packages.nix</filename></link>, rather than <filename>pkgs/all-packages.nix</filename>. Most Perl packages are so straight-forward to build that they are defined here directly, rather than having a separate function for each package called from <filename>perl-packages.nix</filename>. However, more complicated packages should be put in a separate file, typically in <filename>pkgs/development/perl-modules</filename>. Here is an example of the former: -<programlisting> -ClassC3 = buildPerlPackage rec { - name = "Class-C3-0.21"; - src = fetchurl { - url = "mirror://cpan/authors/id/F/FL/FLORA/${name}.tar.gz"; - sha256 = "1bl8z095y4js66pwxnm7s853pi9czala4sqc743fdlnk27kq94gz"; - }; -}; -</programlisting> - Note the use of <literal>mirror://cpan/</literal>, and the <literal>${name}</literal> in the URL definition to ensure that the name attribute is consistent with the source that we’re actually downloading. Perl packages are made available in <filename>all-packages.nix</filename> through the variable <varname>perlPackages</varname>. For instance, if you have a package that needs <varname>ClassC3</varname>, you would typically write -<programlisting> -foo = import ../path/to/foo.nix { - inherit stdenv fetchurl ...; - inherit (perlPackages) ClassC3; -}; -</programlisting> - in <filename>all-packages.nix</filename>. You can test building a Perl package as follows: -<screen> -<prompt>$ </prompt>nix-build -A perlPackages.ClassC3 -</screen> - <varname>buildPerlPackage</varname> adds <literal>perl-</literal> to the start of the name attribute, so the package above is actually called <literal>perl-Class-C3-0.21</literal>. So to install it, you can say: -<screen> -<prompt>$ </prompt>nix-env -i perl-Class-C3 -</screen> - (Of course you can also install using the attribute name: <literal>nix-env -i -A perlPackages.ClassC3</literal>.) - </para> - - <para> - So what does <varname>buildPerlPackage</varname> do? It does the following: - <orderedlist> - <listitem> - <para> - In the configure phase, it calls <literal>perl Makefile.PL</literal> to generate a Makefile. You can set the variable <varname>makeMakerFlags</varname> to pass flags to <filename>Makefile.PL</filename> - </para> - </listitem> - <listitem> - <para> - It adds the contents of the <envar>PERL5LIB</envar> environment variable to <literal>#! .../bin/perl</literal> line of Perl scripts as <literal>-I<replaceable>dir</replaceable></literal> flags. This ensures that a script can find its dependencies. (This can cause this shebang line to become too long for Darwin to handle; see the note below.) - </para> - </listitem> - <listitem> - <para> - In the fixup phase, it writes the propagated build inputs (<varname>propagatedBuildInputs</varname>) to the file <filename>$out/nix-support/propagated-user-env-packages</filename>. <command>nix-env</command> recursively installs all packages listed in this file when you install a package that has it. This ensures that a Perl package can find its dependencies. - </para> - </listitem> - </orderedlist> - </para> - - <para> - <varname>buildPerlPackage</varname> is built on top of <varname>stdenv</varname>, so everything can be customised in the usual way. For instance, the <literal>BerkeleyDB</literal> module has a <varname>preConfigure</varname> hook to generate a configuration file used by <filename>Makefile.PL</filename>: -<programlisting> -{ buildPerlPackage, fetchurl, db }: - -buildPerlPackage rec { - name = "BerkeleyDB-0.36"; - - src = fetchurl { - url = "mirror://cpan/authors/id/P/PM/PMQS/${name}.tar.gz"; - sha256 = "07xf50riarb60l1h6m2dqmql8q5dij619712fsgw7ach04d8g3z1"; - }; - - preConfigure = '' - echo "LIB = ${db.out}/lib" > config.in - echo "INCLUDE = ${db.dev}/include" >> config.in - ''; -} -</programlisting> - </para> - - <para> - Dependencies on other Perl packages can be specified in the <varname>buildInputs</varname> and <varname>propagatedBuildInputs</varname> attributes. If something is exclusively a build-time dependency, use <varname>buildInputs</varname>; if it’s (also) a runtime dependency, use <varname>propagatedBuildInputs</varname>. For instance, this builds a Perl module that has runtime dependencies on a bunch of other modules: -<programlisting> -ClassC3Componentised = buildPerlPackage rec { - name = "Class-C3-Componentised-1.0004"; - src = fetchurl { - url = "mirror://cpan/authors/id/A/AS/ASH/${name}.tar.gz"; - sha256 = "0xql73jkcdbq4q9m0b0rnca6nrlvf5hyzy8is0crdk65bynvs8q1"; - }; - propagatedBuildInputs = [ - ClassC3 ClassInspector TestException MROCompat - ]; -}; -</programlisting> - </para> - - <para> - On Darwin, if a script has too many <literal>-I<replaceable>dir</replaceable></literal> flags in its first line (its “shebang line”), it will not run. This can be worked around by calling the <literal>shortenPerlShebang</literal> function from the <literal>postInstall</literal> phase: -<programlisting> -{ stdenv, buildPerlPackage, fetchurl, shortenPerlShebang }: - -ImageExifTool = buildPerlPackage { - pname = "Image-ExifTool"; - version = "11.50"; - - src = fetchurl { - url = "https://www.sno.phy.queensu.ca/~phil/exiftool/Image-ExifTool-11.50.tar.gz"; - sha256 = "0d8v48y94z8maxkmw1rv7v9m0jg2dc8xbp581njb6yhr7abwqdv3"; - }; - - buildInputs = stdenv.lib.optional stdenv.isDarwin shortenPerlShebang; - postInstall = stdenv.lib.optional stdenv.isDarwin '' - shortenPerlShebang $out/bin/exiftool - ''; -}; -</programlisting> - This will remove the <literal>-I</literal> flags from the shebang line, rewrite them in the <literal>use lib</literal> form, and put them on the next line instead. This function can be given any number of Perl scripts as arguments; it will modify them in-place. - </para> - - <section xml:id="ssec-generation-from-CPAN"> - <title>Generation from CPAN</title> + <section xml:id="ssec-perl-running"> + <title>Running perl programs on the shell</title> <para> - Nix expressions for Perl packages can be generated (almost) automatically from CPAN. This is done by the program <command>nix-generate-from-cpan</command>, which can be installed as follows: + When executing a Perl script, it is possible you get an error such as <literal>./myscript.pl: bad interpreter: /usr/bin/perl: no such file or directory</literal>. This happens when the script expects Perl to be installed at <filename>/usr/bin/perl</filename>, which is not the case when using Perl from nixpkgs. You can fix the script by changing the first line to: + <programlisting> + #!/usr/bin/env perl + </programlisting> + to take the Perl installation from the <literal>PATH</literal> environment variable, or invoke Perl directly with: + <screen> + <prompt>$ </prompt>perl ./myscript.pl + </screen> </para> -<screen> -<prompt>$ </prompt>nix-env -i nix-generate-from-cpan -</screen> + <para> + When the script is using a Perl library that is not installed globally, you might get an error such as <literal>Can't locate DB_File.pm in @INC (you may need to install the DB_File module)</literal>. In that case, you can use <command>nix-shell</command> to start an ad-hoc shell with that library installed, for instance: + <screen> + <prompt>$ </prompt>nix-shell -p perl perlPackages.DBFile --run ./myscript.pl + </screen> + </para> <para> - This program takes a Perl module name, looks it up on CPAN, fetches and unpacks the corresponding package, and prints a Nix expression on standard output. For example: -<screen> -<prompt>$ </prompt>nix-generate-from-cpan XML::Simple - XMLSimple = buildPerlPackage rec { - name = "XML-Simple-2.22"; - src = fetchurl { - url = "mirror://cpan/authors/id/G/GR/GRANTM/${name}.tar.gz"; - sha256 = "b9450ef22ea9644ae5d6ada086dc4300fa105be050a2030ebd4efd28c198eb49"; - }; - propagatedBuildInputs = [ XMLNamespaceSupport XMLSAX XMLSAXExpat ]; - meta = { - description = "An API for simple XML files"; - license = with stdenv.lib.licenses; [ artistic1 gpl1Plus ]; - }; - }; -</screen> - The output can be pasted into <filename>pkgs/top-level/perl-packages.nix</filename> or wherever else you need it. + If you are always using the script in places where <command>nix-shell</command> is available, you can embed the <command>nix-shell</command> invocation in the shebang like this: + <programlisting> + #!/usr/bin/env nix-shell + #! nix-shell -i perl -p perl perlPackages.DBFile + </programlisting> </para> </section> - <section xml:id="ssec-perl-cross-compilation"> - <title>Cross-compiling modules</title> + <section xml:id="ssec-perl-packaging"> + <title>Packaging Perl programs</title> + + <para> + Nixpkgs provides a function <varname>buildPerlPackage</varname>, a generic package builder function for any Perl package that has a standard <varname>Makefile.PL</varname>. It’s implemented in <link + xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/perl-modules/generic"><filename>pkgs/development/perl-modules/generic</filename></link>. + </para> + + <para> + Perl packages from CPAN are defined in <link + xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/perl-packages.nix"><filename>pkgs/top-level/perl-packages.nix</filename></link>, rather than <filename>pkgs/all-packages.nix</filename>. Most Perl packages are so straight-forward to build that they are defined here directly, rather than having a separate function for each package called from <filename>perl-packages.nix</filename>. However, more complicated packages should be put in a separate file, typically in <filename>pkgs/development/perl-modules</filename>. Here is an example of the former: + <programlisting> + ClassC3 = buildPerlPackage rec { + name = "Class-C3-0.21"; + src = fetchurl { + url = "mirror://cpan/authors/id/F/FL/FLORA/${name}.tar.gz"; + sha256 = "1bl8z095y4js66pwxnm7s853pi9czala4sqc743fdlnk27kq94gz"; + }; + }; + </programlisting> + Note the use of <literal>mirror://cpan/</literal>, and the <literal>${name}</literal> in the URL definition to ensure that the name attribute is consistent with the source that we’re actually downloading. Perl packages are made available in <filename>all-packages.nix</filename> through the variable <varname>perlPackages</varname>. For instance, if you have a package that needs <varname>ClassC3</varname>, you would typically write + <programlisting> + foo = import ../path/to/foo.nix { + inherit stdenv fetchurl ...; + inherit (perlPackages) ClassC3; + }; + </programlisting> + in <filename>all-packages.nix</filename>. You can test building a Perl package as follows: + <screen> + <prompt>$ </prompt>nix-build -A perlPackages.ClassC3 + </screen> + <varname>buildPerlPackage</varname> adds <literal>perl-</literal> to the start of the name attribute, so the package above is actually called <literal>perl-Class-C3-0.21</literal>. So to install it, you can say: + <screen> + <prompt>$ </prompt>nix-env -i perl-Class-C3 + </screen> + (Of course you can also install using the attribute name: <literal>nix-env -i -A perlPackages.ClassC3</literal>.) + </para> + + <para> + So what does <varname>buildPerlPackage</varname> do? It does the following: + <orderedlist> + <listitem> + <para> + In the configure phase, it calls <literal>perl Makefile.PL</literal> to generate a Makefile. You can set the variable <varname>makeMakerFlags</varname> to pass flags to <filename>Makefile.PL</filename> + </para> + </listitem> + <listitem> + <para> + It adds the contents of the <envar>PERL5LIB</envar> environment variable to <literal>#! .../bin/perl</literal> line of Perl scripts as <literal>-I<replaceable>dir</replaceable></literal> flags. This ensures that a script can find its dependencies. (This can cause this shebang line to become too long for Darwin to handle; see the note below.) + </para> + </listitem> + <listitem> + <para> + In the fixup phase, it writes the propagated build inputs (<varname>propagatedBuildInputs</varname>) to the file <filename>$out/nix-support/propagated-user-env-packages</filename>. <command>nix-env</command> recursively installs all packages listed in this file when you install a package that has it. This ensures that a Perl package can find its dependencies. + </para> + </listitem> + </orderedlist> + </para> + + <para> + <varname>buildPerlPackage</varname> is built on top of <varname>stdenv</varname>, so everything can be customised in the usual way. For instance, the <literal>BerkeleyDB</literal> module has a <varname>preConfigure</varname> hook to generate a configuration file used by <filename>Makefile.PL</filename>: + <programlisting> + { buildPerlPackage, fetchurl, db }: + + buildPerlPackage rec { + name = "BerkeleyDB-0.36"; + + src = fetchurl { + url = "mirror://cpan/authors/id/P/PM/PMQS/${name}.tar.gz"; + sha256 = "07xf50riarb60l1h6m2dqmql8q5dij619712fsgw7ach04d8g3z1"; + }; + + preConfigure = '' + echo "LIB = ${db.out}/lib" > config.in + echo "INCLUDE = ${db.dev}/include" >> config.in + ''; + } + </programlisting> + </para> <para> - Nixpkgs has experimental support for cross-compiling Perl modules. In many cases, it will just work out of the box, even for modules with native extensions. Sometimes, however, the Makefile.PL for a module may (indirectly) import a native module. In that case, you will need to make a stub for that module that will satisfy the Makefile.PL and install it into <filename>lib/perl5/site_perl/cross_perl/${perl.version}</filename>. See the <varname>postInstall</varname> for <varname>DBI</varname> for an example. + Dependencies on other Perl packages can be specified in the <varname>buildInputs</varname> and <varname>propagatedBuildInputs</varname> attributes. If something is exclusively a build-time dependency, use <varname>buildInputs</varname>; if it’s (also) a runtime dependency, use <varname>propagatedBuildInputs</varname>. For instance, this builds a Perl module that has runtime dependencies on a bunch of other modules: + <programlisting> + ClassC3Componentised = buildPerlPackage rec { + name = "Class-C3-Componentised-1.0004"; + src = fetchurl { + url = "mirror://cpan/authors/id/A/AS/ASH/${name}.tar.gz"; + sha256 = "0xql73jkcdbq4q9m0b0rnca6nrlvf5hyzy8is0crdk65bynvs8q1"; + }; + propagatedBuildInputs = [ + ClassC3 ClassInspector TestException MROCompat + ]; + }; + </programlisting> </para> + + <para> + On Darwin, if a script has too many <literal>-I<replaceable>dir</replaceable></literal> flags in its first line (its “shebang line”), it will not run. This can be worked around by calling the <literal>shortenPerlShebang</literal> function from the <literal>postInstall</literal> phase: + <programlisting> + { stdenv, buildPerlPackage, fetchurl, shortenPerlShebang }: + + ImageExifTool = buildPerlPackage { + pname = "Image-ExifTool"; + version = "11.50"; + + src = fetchurl { + url = "https://www.sno.phy.queensu.ca/~phil/exiftool/Image-ExifTool-11.50.tar.gz"; + sha256 = "0d8v48y94z8maxkmw1rv7v9m0jg2dc8xbp581njb6yhr7abwqdv3"; + }; + + buildInputs = stdenv.lib.optional stdenv.isDarwin shortenPerlShebang; + postInstall = stdenv.lib.optional stdenv.isDarwin '' + shortenPerlShebang $out/bin/exiftool + ''; + }; + </programlisting> + This will remove the <literal>-I</literal> flags from the shebang line, rewrite them in the <literal>use lib</literal> form, and put them on the next line instead. This function can be given any number of Perl scripts as arguments; it will modify them in-place. + </para> + + <section xml:id="ssec-generation-from-CPAN"> + <title>Generation from CPAN</title> + + <para> + Nix expressions for Perl packages can be generated (almost) automatically from CPAN. This is done by the program <command>nix-generate-from-cpan</command>, which can be installed as follows: + </para> + + <screen> + <prompt>$ </prompt>nix-env -i nix-generate-from-cpan + </screen> + + <para> + This program takes a Perl module name, looks it up on CPAN, fetches and unpacks the corresponding package, and prints a Nix expression on standard output. For example: + <screen> + <prompt>$ </prompt>nix-generate-from-cpan XML::Simple + XMLSimple = buildPerlPackage rec { + name = "XML-Simple-2.22"; + src = fetchurl { + url = "mirror://cpan/authors/id/G/GR/GRANTM/${name}.tar.gz"; + sha256 = "b9450ef22ea9644ae5d6ada086dc4300fa105be050a2030ebd4efd28c198eb49"; + }; + propagatedBuildInputs = [ XMLNamespaceSupport XMLSAX XMLSAXExpat ]; + meta = { + description = "An API for simple XML files"; + license = with stdenv.lib.licenses; [ artistic1 gpl1Plus ]; + }; + }; + </screen> + The output can be pasted into <filename>pkgs/top-level/perl-packages.nix</filename> or wherever else you need it. + </para> + </section> + + <section xml:id="ssec-perl-cross-compilation"> + <title>Cross-compiling modules</title> + + <para> + Nixpkgs has experimental support for cross-compiling Perl modules. In many cases, it will just work out of the box, even for modules with native extensions. Sometimes, however, the Makefile.PL for a module may (indirectly) import a native module. In that case, you will need to make a stub for that module that will satisfy the Makefile.PL and install it into <filename>lib/perl5/site_perl/cross_perl/${perl.version}</filename>. See the <varname>postInstall</varname> for <varname>DBI</varname> for an example. + </para> + </section> </section> </section> diff --git a/doc/languages-frameworks/php.section.md b/doc/languages-frameworks/php.section.md new file mode 100644 index 00000000000..763beeb5935 --- /dev/null +++ b/doc/languages-frameworks/php.section.md @@ -0,0 +1,137 @@ +# PHP {#sec-php} + +## User Guide {#ssec-php-user-guide} + +### Overview {#ssec-php-user-guide-overview} + +Several versions of PHP are available on Nix, each of which having a +wide variety of extensions and libraries available. + +The different versions of PHP that nixpkgs provides are located under +attributes named based on major and minor version number; e.g., +`php74` is PHP 7.4. + +Only versions of PHP that are supported by upstream for the entirety +of a given NixOS release will be included in that release of +NixOS. See [PHP Supported +Versions](https://www.php.net/supported-versions.php). + +The attribute `php` refers to the version of PHP considered most +stable and thoroughly tested in nixpkgs for any given release of +NixOS - not necessarily the latest major release from upstream. + +All available PHP attributes are wrappers around their respective +binary PHP package and provide commonly used extensions this way. The +real PHP 7.4 package, i.e. the unwrapped one, is available as +`php74.unwrapped`; see the next section for more details. + +Interactive tools built on PHP are put in `php.packages`; composer is +for example available at `php.packages.composer`. + +Most extensions that come with PHP, as well as some popular +third-party ones, are available in `php.extensions`; for example, the +opcache extension shipped with PHP is available at +`php.extensions.opcache` and the third-party ImageMagick extension at +`php.extensions.imagick`. + +### Installing PHP with extensions {#ssec-php-user-guide-installing-with-extensions} + +A PHP package with specific extensions enabled can be built using +`php.withExtensions`. This is a function which accepts an anonymous +function as its only argument; the function should accept two named +parameters: `enabled` - a list of currently enabled extensions and +`all` - the set of all extensions, and return a list of wanted +extensions. For example, a PHP package with all default extensions and +ImageMagick enabled: + +```nix +php.withExtensions ({ enabled, all }: + enabled ++ [ all.imagick ]) +``` + +To exclude some, but not all, of the default extensions, you can +filter the `enabled` list like this: + +```nix +php.withExtensions ({ enabled, all }: + (lib.filter (e: e != php.extensions.opcache) enabled) + ++ [ all.imagick ]) +``` + +To build your list of extensions from the ground up, you can simply +ignore `enabled`: + +```nix +php.withExtensions ({ all, ... }: with all; [ imagick opcache ]) +``` + +`php.withExtensions` provides extensions by wrapping a minimal php +base package, providing a `php.ini` file listing all extensions to be +loaded. You can access this package through the `php.unwrapped` +attribute; useful if you, for example, need access to the `dev` +output. The generated `php.ini` file can be accessed through the +`php.phpIni` attribute. + +If you want a PHP build with extra configuration in the `php.ini` +file, you can use `php.buildEnv`. This function takes two named and +optional parameters: `extensions` and `extraConfig`. `extensions` +takes an extension specification equivalent to that of +`php.withExtensions`, `extraConfig` a string of additional `php.ini` +configuration parameters. For example, a PHP package with the opcache +and ImageMagick extensions enabled, and `memory_limit` set to `256M`: + +```nix +php.buildEnv { + extensions = { all, ... }: with all; [ imagick opcache ]; + extraConfig = "memory_limit=256M"; +} +``` + +#### Example setup for `phpfpm` {#ssec-php-user-guide-installing-with-extensions-phpfpm} + +You can use the previous examples in a `phpfpm` pool called `foo` as +follows: + +```nix +let + myPhp = php.withExtensions ({ all, ... }: with all; [ imagick opcache ]); +in { + services.phpfpm.pools."foo".phpPackage = myPhp; +}; +``` + +```nix +let + myPhp = php.buildEnv { + extensions = { all, ... }: with all; [ imagick opcache ]; + extraConfig = "memory_limit=256M"; + }; +in { + services.phpfpm.pools."foo".phpPackage = myPhp; +}; +``` + +#### Example usage with `nix-shell` {#ssec-php-user-guide-installing-with-extensions-nix-shell} + +This brings up a temporary environment that contains a PHP interpreter +with the extensions `imagick` and `opcache` enabled: + +```sh +nix-shell -p 'php.withExtensions ({ all, ... }: with all; [ imagick opcache ])' +``` + +### Installing PHP packages with extensions {#ssec-php-user-guide-installing-packages-with-extensions} + +All interactive tools use the PHP package you get them from, so all +packages at `php.packages.*` use the `php` package with its default +extensions. Sometimes this default set of extensions isn't enough and +you may want to extend it. A common case of this is the `composer` +package: a project may depend on certain extensions and `composer` +won't work with that project unless those extensions are loaded. + +Example of building `composer` with additional extensions: +```nix +(php.withExtensions ({ all, enabled }: + enabled ++ (with all; [ imagick redis ])) +).packages.composer +``` diff --git a/doc/languages-frameworks/python.section.md b/doc/languages-frameworks/python.section.md index 650ed342858..f189ce31448 100644 --- a/doc/languages-frameworks/python.section.md +++ b/doc/languages-frameworks/python.section.md @@ -9,7 +9,7 @@ Several versions of the Python interpreter are available on Nix, as well as a high amount of packages. The attribute `python` refers to the default interpreter, which is currently CPython 2.7. It is also possible to refer to -specific versions, e.g. `python35` refers to CPython 3.5, and `pypy` refers to +specific versions, e.g. `python38` refers to CPython 3.8, and `pypy` refers to the default PyPy interpreter. Python is used a lot, and in different ways. This affects also how it is @@ -25,10 +25,10 @@ however, are in separate sets, with one set per interpreter version. The interpreters have several common attributes. One of these attributes is `pkgs`, which is a package set of Python libraries for this specific interpreter. E.g., the `toolz` package corresponding to the default interpreter -is `python.pkgs.toolz`, and the CPython 3.5 version is `python35.pkgs.toolz`. +is `python.pkgs.toolz`, and the CPython 3.8 version is `python38.pkgs.toolz`. The main package set contains aliases to these package sets, e.g. -`pythonPackages` refers to `python.pkgs` and `python35Packages` to -`python35.pkgs`. +`pythonPackages` refers to `python.pkgs` and `python38Packages` to +`python38.pkgs`. #### Installing Python and packages @@ -36,121 +36,191 @@ The Nix and NixOS manuals explain how packages are generally installed. In the case of Python and Nix, it is important to make a distinction between whether the package is considered an application or a library. -Applications on Nix are typically installed into your user -profile imperatively using `nix-env -i`, and on NixOS declaratively by adding the -package name to `environment.systemPackages` in `/etc/nixos/configuration.nix`. -Dependencies such as libraries are automatically installed and should not be -installed explicitly. +Applications on Nix are typically installed into your user profile imperatively +using `nix-env -i`, and on NixOS declaratively by adding the package name to +`environment.systemPackages` in `/etc/nixos/configuration.nix`. Dependencies +such as libraries are automatically installed and should not be installed +explicitly. -The same goes for Python applications and libraries. Python applications can be -installed in your profile. But Python libraries you would like to use for -development cannot be installed, at least not individually, because they won't -be able to find each other resulting in import errors. Instead, it is possible -to create an environment with `python.buildEnv` or `python.withPackages` where -the interpreter and other executables are able to find each other and all of the -modules. +The same goes for Python applications. Python applications can be installed in +your profile, and will be wrapped to find their exact library dependencies, +without impacting other applications or polluting your user environment. -In the following examples we create an environment with Python 3.5, `numpy` and -`toolz`. As you may imagine, there is one limitation here, and that's that -you can install only one environment at a time. You will notice the complaints -about collisions when you try to install a second environment. +But Python libraries you would like to use for development cannot be installed, +at least not individually, because they won't be able to find each other +resulting in import errors. Instead, it is possible to create an environment +with `python.buildEnv` or `python.withPackages` where the interpreter and other +executables are wrapped to be able to find each other and all of the modules. -##### Environment defined in separate `.nix` file +In the following examples we will start by creating a simple, ad-hoc environment +with a nix-shell that has `numpy` and `toolz` in Python 3.8; then we will create +a re-usable environment in a single-file Python script; then we will create a +full Python environment for development with this same environment. -Create a file, e.g. `build.nix`, with the following expression -```nix -with import <nixpkgs> {}; - -python35.withPackages (ps: with ps; [ numpy toolz ]) -``` -and install it in your profile with -```shell -nix-env -if build.nix -``` -Now you can use the Python interpreter, as well as the extra packages (`numpy`, -`toolz`) that you added to the environment. +Philosphically, this should be familiar to users who are used to a `venv` style +of development: individual projects create their own Python environments without +impacting the global environment or each other. -##### Environment defined in `~/.config/nixpkgs/config.nix` +#### Ad-hoc temporary Python environment with `nix-shell` -If you prefer you could also add the environment as a package override to the -Nixpkgs set, e.g. using `config.nix`, +The simplest way to start playing with the way nix wraps and sets up Python +environments is with `nix-shell` at the cmdline. These environments create a +temporary shell session with a Python and a *precise* list of packages (plus +their runtime dependencies), with no other Python packages in the Python +interpreter's scope. -```nix -{ # ... +To create a Python 3.8 session with `numpy` and `toolz` available, run: - packageOverrides = pkgs: with pkgs; { - myEnv = python35.withPackages (ps: with ps; [ numpy toolz ]); - }; -} +```sh +$ nix-shell -p 'python38.withPackages(ps: with ps; [ numpy toolz ])' ``` -and install it in your profile with -```shell -nix-env -iA nixpkgs.myEnv -``` +By default `nix-shell` will start a `bash` session with this interpreter in our +`PATH`, so if we then run: -The environment is is installed by referring to the attribute, and considering -the `nixpkgs` channel was used. +``` +[nix-shell:~/src/nixpkgs]$ python3 +Python 3.8.1 (default, Dec 18 2019, 19:06:26) +[GCC 9.2.0] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> import numpy; import toolz +``` -##### Environment defined in `/etc/nixos/configuration.nix` +Note that no other modules are in scope, even if they were imperatively +installed into our user environment as a dependency of a Python application: -For the sake of completeness, here's another example how to install the -environment system-wide. +``` +>>> import requests +Traceback (most recent call last): + File "<stdin>", line 1, in <module> +ModuleNotFoundError: No module named 'requests' +``` -```nix -{ # ... +We can add as many additional modules onto the `nix-shell` as we need, and we +will still get 1 wrapped Python interpreter. We can start the interpreter +directly like so: - environment.systemPackages = with pkgs; [ - (python35.withPackages(ps: with ps; [ numpy toolz ])) - ]; -} +```sh +$ nix-shell -p 'python38.withPackages(ps: with ps; [ numpy toolz requests ])' --run python3 +these derivations will be built: + /nix/store/xbdsrqrsfa1yva5s7pzsra8k08gxlbz1-python3-3.8.1-env.drv +building '/nix/store/xbdsrqrsfa1yva5s7pzsra8k08gxlbz1-python3-3.8.1-env.drv'... +created 277 symlinks in user environment +Python 3.8.1 (default, Dec 18 2019, 19:06:26) +[GCC 9.2.0] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> import requests +>>> ``` -#### Temporary Python environment with `nix-shell` +Notice that this time it built a new Python environment, which now includes +`requests`. Building an environment just creates wrapper scripts that expose the +selected dependencies to the interpreter while re-using the actual modules. This +means if any other env has installed `requests` or `numpy` in a different +context, we don't need to recompile them -- we just recompile the wrapper script +that sets up an interpreter pointing to them. This matters much more for "big" +modules like `pytorch` or `tensorflow`. + +Module names usually match their names on [pypi.org](https://pypi.org/), but +you can use the [Nixpkgs search website](https://nixos.org/nixos/packages.html) +to find them as well (along with non-python packages). + +At this point we can create throwaway experimental Python environments with +arbitrary dependencies. This is a good way to get a feel for how the Python +interpreter and dependencies work in Nix and NixOS, but to do some actual +development, we'll want to make it a bit more persistent. + +##### Running Python scripts and using `nix-shell` as shebang + +Sometimes, we have a script whose header looks like this: + +```python +#!/usr/bin/env python3 +import numpy as np +a = np.array([1,2]) +b = np.array([3,4]) +print(f"The dot product of {a} and {b} is: {np.dot(a, b)}") +``` -The examples in the previous section showed how to install a Python environment -into a profile. For development you may need to use multiple environments. -`nix-shell` gives the possibility to temporarily load another environment, akin -to `virtualenv`. +Executing this script requires a `python3` that has `numpy`. Using what we learned +in the previous section, we could startup a shell and just run it like so: -There are two methods for loading a shell with Python packages. The first and -recommended method is to create an environment with `python.buildEnv` or -`python.withPackages` and load that. E.g. +``` +nix-shell -p 'python38.withPackages(ps: with ps; [ numpy ])' --run 'python3 foo.py' +The dot product of [1 2] and [3 4] is: 11 +``` -```sh -$ nix-shell -p 'python35.withPackages(ps: with ps; [ numpy toolz ])' +But if we maintain the script ourselves, and if there are more dependencies, it +may be nice to encode those dependencies in source to make the script re-usable +without that bit of knowledge. That can be done by using `nix-shell` as a +[shebang](https://en.wikipedia.org/wiki/Shebang_(Unix), like so: + +```python +#!/usr/bin/env nix-shell +#!nix-shell -i python3 -p "python3.withPackages(ps: [ ps.numpy ])" +import numpy as np +a = np.array([1,2]) +b = np.array([3,4]) +print(f"The dot product of {a} and {b} is: {np.dot(a, b)}") ``` -opens a shell from which you can launch the interpreter +Then we simply execute it, without requiring any environment setup at all! ```sh -[nix-shell:~] python3 +$ ./foo.py +The dot product of [1 2] and [3 4] is: 11 ``` -The other method, which is not recommended, does not create an environment and -requires you to list the packages directly, - -```sh -$ nix-shell -p python35.pkgs.numpy python35.pkgs.toolz +If the dependencies are not available on the host where `foo.py` is executed, it +will build or download them from a Nix binary cache prior to starting up, prior +that it is executed on a machine with a multi-user nix installation. + +This provides a way to ship a self bootstrapping Python script, akin to a +statically linked binary, where it can be run on any machine (provided nix is +installed) without having to assume that `numpy` is installed globally on the +system. + +By default it is pulling the import checkout of Nixpkgs itself from our nix +channel, which is nice as it cache aligns with our other package builds, but we +can make it fully reproducible by pinning the `nixpkgs` import: + +```python +#!/usr/bin/env nix-shell +#!nix-shell -i python3 -p "python3.withPackages(ps: [ ps.numpy ])" +#!nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/d373d80b1207d52621961b16aa4a3438e4f98167.tar.gz +import numpy as np +a = np.array([1,2]) +b = np.array([3,4]) +print(f"The dot product of {a} and {b} is: {np.dot(a, b)}") ``` -Again, it is possible to launch the interpreter from the shell. The Python -interpreter has the attribute `pkgs` which contains all Python libraries for -that specific interpreter. +This will execute with the exact same versions of Python 3.8, numpy, and system +dependencies a year from now as it does today, because it will always use +exactly git commit `d373d80b1207d52621961b16aa4a3438e4f98167` of Nixpkgs for all +of the package versions. + +This is also a great way to ensure the script executes identically on different +servers. ##### Load environment from `.nix` expression -As explained in the Nix manual, `nix-shell` can also load an -expression from a `.nix` file. Say we want to have Python 3.5, `numpy` -and `toolz`, like before, in an environment. Consider a `shell.nix` file -with + +We've now seen how to create an ad-hoc temporary shell session, and how to +create a single script with Python dependencies, but in the course of normal +development we're usually working in an entire package repository. + +As explained in the Nix manual, `nix-shell` can also load an expression from a +`.nix` file. Say we want to have Python 3.8, `numpy` and `toolz`, like before, +in an environment. We can add a `shell.nix` file describing our dependencies: ```nix with import <nixpkgs> {}; - -(python35.withPackages (ps: [ps.numpy ps.toolz])).env +(python38.withPackages (ps: [ps.numpy ps.toolz])).env ``` -Executing `nix-shell` gives you again a Nix shell from which you can run Python. +And then at the command line, just typing `nix-shell` produces the same +environment as before. In a normal project, we'll likely have many more +dependencies; this can provide a way for developers to share the environments +with each other and with CI builders. What's happening here? @@ -158,9 +228,9 @@ What's happening here? imports the `<nixpkgs>` function, `{}` calls it and the `with` statement brings all attributes of `nixpkgs` in the local scope. These attributes form the main package set. -2. Then we create a Python 3.5 environment with the `withPackages` function. +2. Then we create a Python 3.8 environment with the `withPackages` function, as before. 3. The `withPackages` function expects us to provide a function as an argument - that takes the set of all python packages and returns a list of packages to + that takes the set of all Python packages and returns a list of packages to include in the environment. Here, we select the packages `numpy` and `toolz` from the package set. @@ -168,59 +238,106 @@ To combine this with `mkShell` you can: ```nix with import <nixpkgs> {}; - let - pythonEnv = python35.withPackages (ps: [ + pythonEnv = python38.withPackages (ps: [ ps.numpy ps.toolz ]); in mkShell { buildInputs = [ pythonEnv - hello + + black + mypy + + libffi + openssl ]; } ``` -##### Execute command with `--run` -A convenient option with `nix-shell` is the `--run` -option, with which you can execute a command in the `nix-shell`. We can -e.g. directly open a Python shell +This will create a unified environment that has not just our Python interpreter +and its Python dependencies, but also tools like `black` or `mypy` and libraries +like `libffi` the `openssl` in scope. This is generic and can span any number of +tools or languages across the Nixpkgs ecosystem. -```sh -$ nix-shell -p python35Packages.numpy python35Packages.toolz --run "python3" +##### Installing environments globally on the system + +Up to now, we've been creating environments scoped to an ad-hoc shell session, +or a single script, or a single project. This is generally advisable, as it +avoids pollution across contexts. + +However, sometimes we know we will often want a Python with some basic packages, +and want this available without having to enter into a shell or build context. +This can be useful to have things like vim/emacs editors and plugins or shell +tools "just work" without having to set them up, or when running other software +that expects packages to be installed globally. + +To create your own custom environment, create a file in `~/.config/nixpkgs/overlays/` +that looks like this: + +```nix +# ~/.config/nixpkgs/overlays/myEnv.nix +self: super: { + myEnv = super.buildEnv { + name = "myEnv"; + paths = [ + # A Python 3 interpreter with some packages + (self.python3.withPackages ( + ps: with ps; [ + pyflakes + pytest + python-language-server + ] + )) + + # Some other packages we'd like as part of this env + self.mypy + self.black + self.ripgrep + self.tmux + ]; + }; +} ``` -or run a script +You can then build and install this to your profile with: ```sh -$ nix-shell -p python35Packages.numpy python35Packages.toolz --run "python3 myscript.py" +nix-env -iA myEnv ``` -##### `nix-shell` as shebang -In fact, for the second use case, there is a more convenient method. You can add -a [shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) to your script -specifying which dependencies `nix-shell` needs. With the following shebang, you -can just execute `./myscript.py`, and it will make available all dependencies -and run the script in the `python3` shell. +One limitation of this is that you can only have 1 Python env installed +globally, since they conflict on the `python` to load out of your `PATH`. -```py -#! /usr/bin/env nix-shell -#! nix-shell -i python3 -p "python3.withPackages(ps: [ps.numpy])" +If you get a conflict or prefer to keep the setup clean, you can have `nix-env` +atomically *uninstall* all other imperatively installed packages and replace +your profile with just `myEnv` by using the `--replace` flag. + +##### Environment defined in `/etc/nixos/configuration.nix` -import numpy +For the sake of completeness, here's how to install the environment system-wide +on NixOS. + +```nix +{ # ... -print(numpy.__version__) + environment.systemPackages = with pkgs; [ + (python38.withPackages(ps: with ps; [ numpy toolz ])) + ]; +} ``` ### Developing with Python -Now that you know how to get a working Python environment with Nix, it is time -to go forward and start actually developing with Python. We will first have a -look at how Python packages are packaged on Nix. Then, we will look at how you -can use development mode with your code. +Above, we were mostly just focused on use cases and what to do to get started +creating working Python environments in nix. + +Now that you know the basics to be up and running, it is time to take a step +back and take a deeper look at at how Python packages are packaged on Nix. Then, +we will look at how you can use development mode with your code. -#### Packaging a library +#### Python library packages in Nixpkgs With Nix all packages are built by functions. The main function in Nix for building Python libraries is `buildPythonPackage`. Let's see how we can build the @@ -231,11 +348,11 @@ building Python libraries is `buildPythonPackage`. Let's see how we can build th buildPythonPackage rec { pname = "toolz"; - version = "0.7.4"; + version = "0.10.0"; src = fetchPypi { inherit pname version; - sha256 = "43c2c9e5e7a16b6c88ba3088a9bfc82f7db8e13378be7c78d6c14a5f8ed05afd"; + sha256 = "08fdd5ef7c96480ad11c12d472de21acd32359996f69a5259299b540feba4560"; }; doCheck = false; @@ -260,8 +377,9 @@ information. The output of the function is a derivation. An expression for `toolz` can be found in the Nixpkgs repository. As explained in the introduction of this Python section, a derivation of `toolz` is available -for each interpreter version, e.g. `python35.pkgs.toolz` refers to the `toolz` -derivation corresponding to the CPython 3.5 interpreter. +for each interpreter version, e.g. `python38.pkgs.toolz` refers to the `toolz` +derivation corresponding to the CPython 3.8 interpreter. + The above example works when you're directly working on `pkgs/top-level/python-packages.nix` in the Nixpkgs repository. Often though, you will want to test a Nix expression outside of the Nixpkgs tree. @@ -273,13 +391,13 @@ and adds it along with a `numpy` package to a Python environment. with import <nixpkgs> {}; ( let - my_toolz = python35.pkgs.buildPythonPackage rec { + my_toolz = python38.pkgs.buildPythonPackage rec { pname = "toolz"; - version = "0.7.4"; + version = "0.10.0"; - src = python35.pkgs.fetchPypi { + src = python38.pkgs.fetchPypi { inherit pname version; - sha256 = "43c2c9e5e7a16b6c88ba3088a9bfc82f7db8e13378be7c78d6c14a5f8ed05afd"; + sha256 = "08fdd5ef7c96480ad11c12d472de21acd32359996f69a5259299b540feba4560"; }; doCheck = false; @@ -290,12 +408,12 @@ with import <nixpkgs> {}; }; }; - in python35.withPackages (ps: [ps.numpy my_toolz]) + in python38.withPackages (ps: [ps.numpy my_toolz]) ).env ``` Executing `nix-shell` will result in an environment in which you can use -Python 3.5 and the `toolz` package. As you can see we had to explicitly mention +Python 3.8 and the `toolz` package. As you can see we had to explicitly mention for which Python version we want to build a package. So, what did we do here? Well, we took the Nix expression that we used earlier @@ -312,7 +430,7 @@ Our example, `toolz`, does not have any dependencies on other Python packages or system libraries. According to the manual, `buildPythonPackage` uses the arguments `buildInputs` and `propagatedBuildInputs` to specify dependencies. If something is exclusively a build-time dependency, then the dependency should be -included as a `buildInput`, but if it is (also) a runtime dependency, then it +included in `buildInputs`, but if it is (also) a runtime dependency, then it should be added to `propagatedBuildInputs`. Test dependencies are considered build-time dependencies and passed to `checkInputs`. @@ -412,7 +530,7 @@ buildPythonPackage rec { meta = with lib; { description = "A pythonic wrapper around FFTW, the FFT library, presenting a unified interface for all the supported transforms"; - homepage = http://hgomersall.github.com/pyFFTW; + homepage = "http://hgomersall.github.com/pyFFTW"; license = with licenses; [ bsd2 bsd3 ]; maintainers = with maintainers; [ fridh ]; }; @@ -420,13 +538,129 @@ buildPythonPackage rec { ``` Note also the line `doCheck = false;`, we explicitly disabled running the test-suite. +#### Testing Python Packages + +It is highly encouraged to have testing as part of the package build. This +helps to avoid situations where the package was able to build and install, +but is not usable at runtime. Currently, all packages will use the `test` +command provided by the setup.py (i.e. `python setup.py test`). However, +this is currently deprecated https://github.com/pypa/setuptools/pull/1878 +and your package should provide its own checkPhase. + +*NOTE:* The `checkPhase` for python maps to the `installCheckPhase` on a +normal derivation. This is due to many python packages not behaving well +to the pre-installed version of the package. Version info, and natively +compiled extensions generally only exist in the install directory, and +thus can cause issues when a test suite asserts on that behavior. -#### Develop local package +*NOTE:* Tests should only be disabled if they don't agree with nix +(e.g. external dependencies, network access, flakey tests), however, +as many tests should be enabled as possible. Failing tests can still be +a good indication that the package is not in a valid state. -As a Python developer you're likely aware of [development mode](http://setuptools.readthedocs.io/en/latest/setuptools.html#development-mode) (`python setup.py develop`); -instead of installing the package this command creates a special link to the project code. -That way, you can run updated code without having to reinstall after each and every change you make. -Development mode is also available. Let's see how you can use it. +#### Using pytest + +Pytest is the most common test runner for python repositories. A trivial +test run would be: +``` + checkInputs = [ pytest ]; + checkPhase = "pytest"; +``` + +However, many repositories' test suites do not translate well to nix's build +sandbox, and will generally need many tests to be disabled. + +To filter tests using pytest, one can do the following: +``` + checkInputs = [ pytest ]; + # avoid tests which need additional data or touch network + checkPhase = '' + pytest tests/ --ignore=tests/integration -k 'not download and not update' + ''; +``` + +`--ignore` will tell pytest to ignore that file or directory from being +collected as part of a test run. This is useful is a file uses a package +which is not available in nixpkgs, thus skipping that test file is much +easier than having to create a new package. + +`-k` is used to define a predicate for test names. In this example, we are +filtering out tests which contain `download` or `update` in their test case name. +Only one `-k` argument is allows, and thus a long predicate should be concatenated +with "\" and wrapped to the next line. + +*NOTE:* In pytest==6.0.1, the use of "\" to continue a line (e.g. `-k 'not download \'`) has +been removed, in this case, it's recommended to use `pytestCheckHook`. + +#### Using pytestCheckHook + +`pytestCheckHook` is a convenient hook which will substitute the setuptools +`test` command for a checkPhase which runs `pytest`. This is also beneficial +when a package may need many items disabled to run the test suite. + +Using the example above, the analagous pytestCheckHook usage would be: +``` + checkInputs = [ pytestCheckHook ]; + + # requires additional data + pytestFlagsArray = [ "tests/" "--ignore=tests/integration" ]; + + disabledTests = [ + # touches network + "download" + "update" + ]; +``` + +This is expecially useful when tests need to be conditionallydisabled, +for example: + +``` + disabledTests = [ + # touches network + "download" + "update" + ] ++ lib.optionals (pythonAtLeast "3.8") [ + # broken due to python3.8 async changes + "async" + ] ++ lib.optionals stdenv.isDarwin [ + # can fail when building with other packages + "socket" + ]; +``` +Trying to concatenate the related strings to disable tests in a regular checkPhase +would be much harder to read. This also enables us to comment on why specific tests +are disabled. + +#### Using pythonImportsCheck + +Although unit tests are highly prefered to valid correctness of a package. Not +all packages have test suites that can be ran easily, and some have none at all. +To help ensure the package still works, `pythonImportsCheck` can attempt to import +the listed modules. + +``` + pythonImportsCheck = [ "requests" "urllib" ]; +``` +roughly translates to: +``` + postCheck = '' + PYTHONPATH=$out/${python.sitePackages}:$PYTHONPATH + python -c "import requests; import urllib" + ''; +``` +However, this is done in it's own phase, and not dependent on whether `doCheck = true;` + +This can also be useful in verifying that the package doesn't assume commonly +present packages (e.g. `setuptools`) + +### Develop local package + +As a Python developer you're likely aware of [development mode](http://setuptools.readthedocs.io/en/latest/setuptools.html#development-mode) +(`python setup.py develop`); instead of installing the package this command +creates a special link to the project code. That way, you can run updated code +without having to reinstall after each and every change you make. Development +mode is also available. Let's see how you can use it. In the previous Nix expression the source was fetched from an url. We can also refer to a local source instead using `src = ./path/to/source/tree;` @@ -435,7 +669,7 @@ If we create a `shell.nix` file which calls `buildPythonPackage`, and if `src` is a local source, and if the local source has a `setup.py`, then development mode is activated. -In the following example we create a simple environment that has a Python 3.5 +In the following example we create a simple environment that has a Python 3.8 version of our package in it, as well as its dependencies and other packages we like to have in the environment, all specified with `propagatedBuildInputs`. Indeed, we can just add any package we like to have in our environment to @@ -443,7 +677,7 @@ Indeed, we can just add any package we like to have in our environment to ```nix with import <nixpkgs> {}; -with python35Packages; +with python38Packages; buildPythonPackage rec { name = "mypackage"; @@ -455,7 +689,6 @@ buildPythonPackage rec { It is important to note that due to how development mode is implemented on Nix it is not possible to have multiple packages simultaneously in development mode. - ### Organising your packages So far we discussed how you can use Python on Nix, and how you can develop with @@ -481,11 +714,11 @@ We first create a function that builds `toolz` in `~/path/to/toolz/release.nix` buildPythonPackage rec { pname = "toolz"; - version = "0.7.4"; + version = "0.10.0"; src = fetchPypi { inherit pname version; - sha256 = "43c2c9e5e7a16b6c88ba3088a9bfc82f7db8e13378be7c78d6c14a5f8ed05afd"; + sha256 = "08fdd5ef7c96480ad11c12d472de21acd32359996f69a5259299b540feba4560"; }; meta = with lib; { @@ -497,17 +730,17 @@ buildPythonPackage rec { } ``` -It takes an argument `buildPythonPackage`. -We now call this function using `callPackage` in the definition of our environment +It takes an argument `buildPythonPackage`. We now call this function using +`callPackage` in the definition of our environment ```nix with import <nixpkgs> {}; ( let toolz = callPackage /path/to/toolz/release.nix { - buildPythonPackage = python35Packages.buildPythonPackage; + buildPythonPackage = python38Packages.buildPythonPackage; }; - in python35.withPackages (ps: [ ps.numpy toolz ]) + in python38.withPackages (ps: [ ps.numpy toolz ]) ).env ``` @@ -515,17 +748,17 @@ Important to remember is that the Python version for which the package is made depends on the `python` derivation that is passed to `buildPythonPackage`. Nix tries to automatically pass arguments when possible, which is why generally you don't explicitly define which `python` derivation should be used. In the above -example we use `buildPythonPackage` that is part of the set `python35Packages`, -and in this case the `python35` interpreter is automatically used. +example we use `buildPythonPackage` that is part of the set `python38Packages`, +and in this case the `python38` interpreter is automatically used. ## Reference ### Interpreters -Versions 2.7, 3.5, 3.6, 3.7 and 3.8 of the CPython interpreter are available as -respectively `python27`, `python35`, `python36`, `python37` and `python38`. The +Versions 2.7, 3.6, 3.7 and 3.8 of the CPython interpreter are available as +respectively `python27`, `python36`, `python37` and `python38`. The aliases `python2` and `python3` correspond to respectively `python27` and -`python37`. The default interpreter, `python`, maps to `python2`. The PyPy +`python38`. The default interpreter, `python`, maps to `python2`. The PyPy interpreters compatible with Python 2.7 and 3 are available as `pypy27` and `pypy3`, with aliases `pypy2` mapping to `pypy27` and `pypy` mapping to `pypy2`. The Nix expressions for the interpreters can be found in @@ -548,7 +781,7 @@ Each interpreter has the following attributes: - `buildEnv`. Function to build python interpreter environments with extra packages bundled together. See section *python.buildEnv function* for usage and documentation. - `withPackages`. Simpler interface to `buildEnv`. See section *python.withPackages function* for usage and documentation. - `sitePackages`. Alias for `lib/${libPrefix}/site-packages`. -- `executable`. Name of the interpreter executable, e.g. `python3.7`. +- `executable`. Name of the interpreter executable, e.g. `python3.8`. - `pkgs`. Set of Python packages for that specific interpreter. The package set can be modified by overriding the interpreter and passing `packageOverrides`. ### Building packages and applications @@ -571,15 +804,16 @@ attribute set is created for each available Python interpreter. The available sets are * `pkgs.python27Packages` -* `pkgs.python35Packages` * `pkgs.python36Packages` * `pkgs.python37Packages` +* `pkgs.python38Packages` +* `pkgs.python39Packages` * `pkgs.pypyPackages` and the aliases * `pkgs.python2Packages` pointing to `pkgs.python27Packages` -* `pkgs.python3Packages` pointing to `pkgs.python37Packages` +* `pkgs.python3Packages` pointing to `pkgs.python38Packages` * `pkgs.pythonPackages` pointing to `pkgs.python2Packages` #### `buildPythonPackage` function @@ -643,12 +877,9 @@ following are specific to `buildPythonPackage`: appears more than once in dependency tree. Default is `true`. * `disabled` ? false: If `true`, package is not built for the particular Python interpreter version. -* `dontWrapPythonPrograms ? false`: Skip wrapping of python programs. +* `dontWrapPythonPrograms ? false`: Skip wrapping of Python programs. * `permitUserSite ? false`: Skip setting the `PYTHONNOUSERSITE` environment variable in wrapped programs. -* `installFlags ? []`: A list of strings. Arguments to be passed to `pip - install`. To pass options to `python setup.py install`, use - `--install-option`. E.g., `installFlags=["--install-option='--cpp_implementation'"]`. * `format ? "setuptools"`: Format of the source. Valid options are `"setuptools"`, `"pyproject"`, `"flit"`, `"wheel"`, and `"other"`. `"setuptools"` is for when the source has a `setup.py` and `setuptools` is @@ -662,8 +893,11 @@ following are specific to `buildPythonPackage`: variables which will be available when the binary is run. For example, `makeWrapperArgs = ["--set FOO BAR" "--set BAZ QUX"]`. * `namePrefix`: Prepends text to `${name}` parameter. In case of libraries, this - defaults to `"python3.5-"` for Python 3.5, etc., and in case of applications + defaults to `"python3.8-"` for Python 3.8, etc., and in case of applications to `""`. +* `pipInstallFlags ? []`: A list of strings. Arguments to be passed to `pip + install`. To pass options to `python setup.py install`, use + `--install-option`. E.g., `pipInstallFlags=["--install-option='--cpp_implementation'"]`. * `pythonPath ? []`: List of packages to be added into `$PYTHONPATH`. Packages in `pythonPath` are not propagated (contrary to `propagatedBuildInputs`). * `preShellHook`: Hook to execute commands before `shellHook`. @@ -730,7 +964,7 @@ Another difference is that `buildPythonPackage` by default prefixes the names of the packages with the version of the interpreter. Because this is irrelevant for applications, the prefix is omitted. -When packaging a python application with `buildPythonApplication`, it should be +When packaging a Python application with `buildPythonApplication`, it should be called with `callPackage` and passed `python` or `pythonPackages` (possibly specifying an interpreter version), like this: @@ -761,7 +995,7 @@ luigi = callPackage ../applications/networking/cluster/luigi { }; ``` Since the package is an application, a consumer doesn't need to care about -python versions or modules, which is why they don't go in `pythonPackages`. +Python versions or modules, which is why they don't go in `pythonPackages`. #### `toPythonApplication` function @@ -875,7 +1109,7 @@ thus be also written like this: ```nix with import <nixpkgs> {}; -(python36.withPackages (ps: [ps.numpy ps.requests])).env +(python38.withPackages (ps: [ps.numpy ps.requests])).env ``` In contrast to `python.buildEnv`, `python.withPackages` does not support the @@ -898,14 +1132,15 @@ are used in `buildPythonPackage`. - `pipBuildHook` to build a wheel using `pip` and PEP 517. Note a build system (e.g. `setuptools` or `flit`) should still be added as `nativeBuildInput`. - `pipInstallHook` to install wheels. -- `pytestCheckHook` to run tests with `pytest`. +- `pytestCheckHook` to run tests with `pytest`. See [example usage](#using-pytestcheckhook). - `pythonCatchConflictsHook` to check whether a Python package is not already existing. - `pythonImportsCheckHook` to check whether importing the listed modules works. - `pythonRemoveBinBytecode` to remove bytecode from the `/bin` folder. - `setuptoolsBuildHook` to build a wheel using `setuptools`. - `setuptoolsCheckHook` to run tests with `python setup.py test`. - `venvShellHook` to source a Python 3 `venv` at the `venvDir` location. A - `venv` is created if it does not yet exist. + `venv` is created if it does not yet exist. `postVenvCreation` can be used to + to run commands only after venv is first created. - `wheelUnpackHook` to move a wheel to the correct folder so it can be installed with the `pipInstallHook`. @@ -932,7 +1167,7 @@ pythonPackages.buildPythonPackage { Running `nix-shell` with no arguments should give you the environment in which the package would be built with `nix-build`. -Shortcut to setup environments with C headers/libraries and python packages: +Shortcut to setup environments with C headers/libraries and Python packages: ```shell nix-shell -p pythonPackages.pyramid zlib libjpeg git @@ -960,10 +1195,9 @@ has security implications and is relevant for those using Python in a When the environment variable `DETERMINISTIC_BUILD` is set, all bytecode will have timestamp 1. The `buildPythonPackage` function sets `DETERMINISTIC_BUILD=1` -and [PYTHONHASHSEED=0](https://docs.python.org/3.5/using/cmdline.html#envvar-PYTHONHASHSEED). +and [PYTHONHASHSEED=0](https://docs.python.org/3.8/using/cmdline.html#envvar-PYTHONHASHSEED). Both are also exported in `nix-shell`. - ### Automatic tests It is recommended to test packages as part of the build process. @@ -976,7 +1210,7 @@ example of such a situation is when `py.test` is used. #### Common issues * Non-working tests can often be deselected. By default `buildPythonPackage` - runs `python setup.py test`. Most python modules follows the standard test + runs `python setup.py test`. Most Python modules follows the standard test protocol where the pytest runner can be used instead. `py.test` supports a `-k` parameter to ignore test methods or classes: @@ -1014,7 +1248,7 @@ with import <nixpkgs> {}; packageOverrides = self: super: { pandas = super.pandas.overridePythonAttrs(old: {name="foo";}); }; - in pkgs.python35.override {inherit packageOverrides;}; + in pkgs.python38.override {inherit packageOverrides;}; in python.withPackages(ps: [ps.pandas])).env ``` @@ -1036,7 +1270,7 @@ with import <nixpkgs> {}; packageOverrides = self: super: { scipy = super.scipy_0_17; }; - in (pkgs.python35.override {inherit packageOverrides;}).withPackages (ps: [ps.blaze]) + in (pkgs.python38.override {inherit packageOverrides;}).withPackages (ps: [ps.blaze]) ).env ``` @@ -1049,12 +1283,12 @@ If you want the whole of Nixpkgs to use your modifications, then you can use ```nix let pkgs = import <nixpkgs> {}; - newpkgs = import pkgs.path { overlays = [ (pkgsself: pkgssuper: { - python27 = let - packageOverrides = self: super: { - numpy = super.numpy_1_10; + newpkgs = import pkgs.path { overlays = [ (self: super: { + python38 = let + packageOverrides = python-self: python-super: { + numpy = python-super.numpy_1_18; }; - in pkgssuper.python27.override {inherit packageOverrides;}; + in super.python38.override {inherit packageOverrides;}; } ) ]; }; in newpkgs.inkscape ``` @@ -1127,14 +1361,14 @@ If you want to create a Python environment for development, then the recommended method is to use `nix-shell`, either with or without the `python.buildEnv` function. -### How to consume python modules using pip in a virtual environment like I am used to on other Operating Systems? +### How to consume Python modules using pip in a virtual environment like I am used to on other Operating Systems? While this approach is not very idiomatic from Nix perspective, it can still be useful when dealing with pre-existing projects or in situations where it's not feasible or desired to write derivations for all required dependencies. This is an example of a `default.nix` for a `nix-shell`, which allows to consume -a virtual environment created by `venv`, and install python modules through +a virtual environment created by `venv`, and install Python modules through `pip` the traditional way. Create this `default.nix` file, together with a `requirements.txt` and simply @@ -1149,7 +1383,7 @@ in pkgs.mkShell rec { name = "impurePythonEnv"; venvDir = "./.venv"; buildInputs = [ - # A python interpreter including the 'venv' module is required to bootstrap + # A Python interpreter including the 'venv' module is required to bootstrap # the environment. pythonPackages.python @@ -1163,7 +1397,7 @@ in pkgs.mkShell rec { pythonPackages.requests # In this particular example, in order to compile any binary extensions they may - # require, the python modules listed in the hypothetical requirements.txt need + # require, the Python modules listed in the hypothetical requirements.txt need # the following packages to be installed locally: taglib openssl @@ -1174,16 +1408,23 @@ in pkgs.mkShell rec { zlib ]; + # Run this command, only after creating the virtual environment + postVenvCreation = '' + unset SOURCE_DATE_EPOCH + pip install -r requirements.txt + ''; + # Now we can execute any commands within the virtual environment. # This is optional and can be left out to run pip manually. postShellHook = '' - pip install -r requirements.txt + # allow pip to install wheels + unset SOURCE_DATE_EPOCH ''; } ``` -In case the supplied venvShellHook is insufficient, or when python 2 support is +In case the supplied venvShellHook is insufficient, or when Python 2 support is needed, you can define your own shell hook and adapt to your needs like in the following example: @@ -1229,7 +1470,7 @@ in pkgs.mkShell rec { ``` Note that the `pip install` is an imperative action. So every time `nix-shell` -is executed it will attempt to download the python modules listed in +is executed it will attempt to download the Python modules listed in requirements.txt. However these will be cached locally within the `virtualenv` folder and not downloaded again. @@ -1290,32 +1531,8 @@ self: super: { ### How to use Intel's MKL with numpy and scipy? -A `site.cfg` is created that configures BLAS based on the `blas` parameter of -the `numpy` derivation. By passing in `mkl`, `numpy` and packages depending on -`numpy` will be built with `mkl`. - -The following is an overlay that configures `numpy` to use `mkl`: - -```nix -self: super: { - python37 = super.python37.override { - packageOverrides = python-self: python-super: { - numpy = python-super.numpy.override { - blas = super.pkgs.mkl; - }; - }; - }; -} -``` - -`mkl` requires an `openmp` implementation when running with multiple processors. -By default, `mkl` will use Intel's `iomp` implementation if no other is -specified, but this is a runtime-only dependency and binary compatible with the -LLVM implementation. To use that one instead, Intel recommends users set it with -`LD_PRELOAD`. - -Note that `mkl` is only available on `x86_64-{linux,darwin}` platforms; -moreover, Hydra is not building and distributing pre-compiled binaries using it. +MKL can be configured using an overlay. See the section "[Using overlays to +configure alternatives](#sec-overlays-alternatives-blas-lapack)". ### What inputs do `setup_requires`, `install_requires` and `tests_require` map to? diff --git a/doc/languages-frameworks/ruby.xml b/doc/languages-frameworks/ruby.xml index b28745fd6e2..9b36801fb96 100644 --- a/doc/languages-frameworks/ruby.xml +++ b/doc/languages-frameworks/ruby.xml @@ -32,7 +32,7 @@ bundlerEnv rec { meta = with lib; { description = "A monitoring framework that aims to be simple, malleable, and scalable"; - homepage = http://sensuapp.org/; + homepage = "http://sensuapp.org/"; license = with licenses; mit; maintainers = with maintainers; [ theuni ]; platforms = platforms.unix; @@ -69,7 +69,7 @@ bundlerApp { meta = with lib; { description = "Tool and libraries for maintaining Ruby gems."; - homepage = https://github.com/nyarly/corundum; + homepage = "https://github.com/nyarly/corundum"; license = licenses.mit; maintainers = [ maintainers.nyarly ]; platforms = platforms.unix; diff --git a/doc/languages-frameworks/rust.section.md b/doc/languages-frameworks/rust.section.md index cec3373cbee..6c51da87cab 100644 --- a/doc/languages-frameworks/rust.section.md +++ b/doc/languages-frameworks/rust.section.md @@ -43,7 +43,6 @@ rustPlatform.buildRustPackage rec { homepage = "https://github.com/BurntSushi/ripgrep"; license = licenses.unlicense; maintainers = [ maintainers.tailhook ]; - platforms = platforms.all; }; } ``` @@ -75,6 +74,90 @@ pkgs.rustPlatform.buildRustPackage { } ``` +### Running package tests + +When using `buildRustPackage`, the `checkPhase` is enabled by default and runs +`cargo test` on the package to build. To make sure that we don't compile the +sources twice and to actually test the artifacts that will be used at runtime, +the tests will be ran in the `release` mode by default. + +However, in some cases the test-suite of a package doesn't work properly in the +`release` mode. For these situations, the mode for `checkPhase` can be changed like +so: + +```nix +rustPlatform.buildRustPackage { + /* ... */ + checkType = "debug"; +} +``` + +Please note that the code will be compiled twice here: once in `release` mode +for the `buildPhase`, and again in `debug` mode for the `checkPhase`. + +#### Tests relying on the structure of the `target/` directory + +Some tests may rely on the structure of the `target/` directory. Those tests +are likely to fail because we use `cargo --target` during the build. This means that +the artifacts +[are stored in `target/<architecture>/release/`](https://doc.rust-lang.org/cargo/guide/build-cache.html), +rather than in `target/release/`. + +This can only be worked around by patching the affected tests accordingly. + +#### Disabling package-tests + +In some instances, it may be necessary to disable testing altogether (with `doCheck = false;`): + +* If no tests exist -- the `checkPhase` should be explicitly disabled to skip + unnecessary build steps to speed up the build. +* If tests are highly impure (e.g. due to network usage). + +There will obviously be some corner-cases not listed above where it's sensible to disable tests. +The above are just guidelines, and exceptions may be granted on a case-by-case basis. + +However, please check if it's possible to disable a problematic subset of the +test suite and leave a comment explaining your reasoning. + +### Building a package in `debug` mode + +By default, `buildRustPackage` will use `release` mode for builds. If a package +should be built in `debug` mode, it can be configured like so: + +```nix +rustPlatform.buildRustPackage { + /* ... */ + buildType = "debug"; +} +``` + +In this scenario, the `checkPhase` will be ran in `debug` mode as well. + +### Custom `build`/`install`-procedures + +Some packages may use custom scripts for building/installing, e.g. with a `Makefile`. +In these cases, it's recommended to override the `buildPhase`/`installPhase`/`checkPhase`. + +Otherwise, some steps may fail because of the modified directory structure of `target/`. + +### Building a crate with an absent or out-of-date Cargo.lock file + +`buildRustPackage` needs a `Cargo.lock` file to get all dependencies in the +source code in a reproducible way. If it is missing or out-of-date one can use +the `cargoPatches` attribute to update or add it. + +``` +{ lib, rustPlatform, fetchFromGitHub }: + +rustPlatform.buildRustPackage rec { + (...) + cargoPatches = [ + # a patch file to add/update Cargo.lock in the source code + ./add-Cargo.lock.patch + ]; +} +``` + ## Compiling Rust crates using Nix instead of Cargo ### Simple operation diff --git a/doc/languages-frameworks/texlive.xml b/doc/languages-frameworks/texlive.xml index 8fa8f963b2f..a581ec5911c 100644 --- a/doc/languages-frameworks/texlive.xml +++ b/doc/languages-frameworks/texlive.xml @@ -149,31 +149,4 @@ EOF ]]></programlisting> </para> </section> - - <section xml:id="sec-language-texlive-known-problems"> - <title>Known problems</title> - - <itemizedlist> - <listitem> - <para> - Some tools are still missing, e.g. luajittex; - </para> - </listitem> - <listitem> - <para> - some apps aren't packaged/tested yet (asymptote, biber, etc.); - </para> - </listitem> - <listitem> - <para> - feature/bug: when a package is rejected by <varname>pkgFilter</varname>, its dependencies are still propagated; - </para> - </listitem> - <listitem> - <para> - in case of any bugs or feature requests, file a github issue or better a pull request and /cc @vcunat. - </para> - </listitem> - </itemizedlist> - </section> </section> diff --git a/doc/languages-frameworks/vim.section.md b/doc/languages-frameworks/vim.section.md index 05a23d26cf2..4911509212e 100644 --- a/doc/languages-frameworks/vim.section.md +++ b/doc/languages-frameworks/vim.section.md @@ -261,12 +261,7 @@ deoplete-fish = super.deoplete-fish.overrideAttrs(old: { Sometimes plugins require an override that must be changed when the plugin is updated. This can cause issues when Vim plugins are auto-updated but the associated override isn't updated. For these plugins, the override should be written so that it specifies all information required to install the plugin, and running `./update.py` doesn't change the derivation for the plugin. Manually updating the override is required to update these types of plugins. An example of such a plugin is `LanguageClient-neovim`. -To add a new plugin: - - 1. run `./update.py` and create a commit named "vimPlugins: Update", - 2. add the new plugin to [vim-plugin-names](/pkgs/misc/vim-plugins/vim-plugin-names) and add overrides if required to [overrides.nix](/pkgs/misc/vim-plugins/overrides.nix), - 3. run `./update.py` again and create a commit named "vimPlugins.[name]: init at [version]" (where `name` and `version` can be found in [generated.nix](/pkgs/misc/vim-plugins/generated.nix)), and - 4. create a pull request. +To add a new plugin, run `./update.py --add "[owner]/[name]"`. **NOTE**: This script automatically commits to your git repository. Be sure to check out a fresh branch before running. ## Important repositories diff --git a/doc/old/cross.txt b/doc/old/cross.txt index ff9fefb04a8..064ca8d37b0 100644 --- a/doc/old/cross.txt +++ b/doc/old/cross.txt @@ -60,7 +60,7 @@ stdenv.mkDerivation { name = "binutils-2.16.1-arm"; builder = ./builder.sh; src = fetchurl { - url = http://ftp.nluug.nl/gnu/binutils/binutils-2.16.1.tar.bz2; + url = "http://ftp.nluug.nl/gnu/binutils/binutils-2.16.1.tar.bz2"; sha256 = "1ian3kwh2vg6hr3ymrv48s04gijs539vzrq62xr76bxbhbwnz2np"; }; inherit noSysDirs; @@ -84,7 +84,7 @@ stdenv.mkDerivation { name = "linux-headers-2.6.13.1-arm"; builder = ./builder.sh; src = fetchurl { - url = http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.13.1.tar.bz2; + url = "http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.13.1.tar.bz2"; sha256 = "12qxmc827fjhaz53kjy7vyrzsaqcg78amiqsb3qm20z26w705lma"; }; } @@ -220,7 +220,7 @@ fi preConfigure=preConfigure preConfigure() { - + # Determine the frontends to build. langs="c" if test -n "$langCC"; then @@ -262,7 +262,7 @@ postInstall() { #if test -z "$profiledCompiler"; then #makeFlags="bootstrap" -#else +#else #makeFlags="profiledbootstrap" #fi diff --git a/doc/preface.chapter.md b/doc/preface.chapter.md index 88ca5e2e3ce..549e42de7aa 100644 --- a/doc/preface.chapter.md +++ b/doc/preface.chapter.md @@ -37,12 +37,12 @@ security updates. More up to date packages and modules are available via the Both `nixos-unstable` and `nixpkgs` follow the `master` branch of the Nixpkgs repository, although both do lag the `master` branch by generally -[a couple of days](https://howoldis.herokuapp.com/). Updates to a channel are +[a couple of days](https://status.nixos.org/). Updates to a channel are distributed as soon as all tests for that channel pass, e.g. [this table](https://hydra.nixos.org/job/nixpkgs/trunk/unstable#tabs-constituents) shows the status of tests for the `nixpkgs` channel. -The tests are conducted by a cluster called [Hydra](http://nixos.org/hydra/), +The tests are conducted by a cluster called [Hydra](https://nixos.org/hydra/), which also builds binary packages from the Nix expressions in Nixpkgs for `x86_64-linux`, `i686-linux` and `x86_64-darwin`. The binaries are made available via a [binary cache](https://cache.nixos.org). diff --git a/doc/release-notes.xml b/doc/release-notes.xml index b85f61da079..a15f5466729 100644 --- a/doc/release-notes.xml +++ b/doc/release-notes.xml @@ -190,7 +190,7 @@ preConfigure = "configureFlagsArray=(\"CFLAGS=-O0 -g\")";</programlisting> The function <function>fetchurl</function> now has support for two different kinds of mirroring of files. First, it has support for <emphasis>content-addressable mirrors</emphasis>. For example, given the <function>fetchurl</function> call <programlisting> fetchurl { - url = http://releases.mozilla.org/<replaceable>...</replaceable>/firefox-2.0.0.6-source.tar.bz2; + url = "http://releases.mozilla.org/<replaceable>...</replaceable>/firefox-2.0.0.6-source.tar.bz2"; sha1 = "eb72f55e4a8bf08e8c6ef227c0ade3d068ba1082"; }</programlisting> <function>fetchurl</function> will first try to download this file from <link @@ -286,7 +286,7 @@ export NIX_MIRRORS_sourceforge=http://osdn.dl.sourceforge.net/sourceforge/</prog <note> <para> This release of Nixpkgs requires <link -xlink:href='http://nixos.org/releases/nix/nix-0.10/'>Nix 0.10</link> or higher. +xlink:href='https://nixos.org/releases/nix/nix-0.10/'>Nix 0.10</link> or higher. </para> </note> @@ -436,7 +436,7 @@ stdenv.mkDerivation { <listitem> <para> Distribution files have been moved to <link - xlink:href="http://nixos.org/" />. + xlink:href="https://nixos.org/" />. </para> </listitem> <listitem> diff --git a/doc/stdenv/meta.xml b/doc/stdenv/meta.xml index 45f7834eb2c..9cef9360002 100644 --- a/doc/stdenv/meta.xml +++ b/doc/stdenv/meta.xml @@ -155,17 +155,17 @@ hello-2.3 A program that produces a familiar, friendly greeting <itemizedlist> <listitem> <para> - Single license referenced by attribute (preferred) <literal>stdenv.lib.licenses.gpl3</literal>. + Single license referenced by attribute (preferred) <literal>stdenv.lib.licenses.gpl3Only</literal>. </para> </listitem> <listitem> <para> - Single license referenced by its attribute shortName (frowned upon) <literal>"gpl3"</literal>. + Single license referenced by its attribute shortName (frowned upon) <literal>"gpl3Only"</literal>. </para> </listitem> <listitem> <para> - Single license referenced by its attribute spdxId (frowned upon) <literal>"GPL-3.0"</literal>. + Single license referenced by its attribute spdxId (frowned upon) <literal>"GPL-3.0-only"</literal>. </para> </listitem> <listitem> diff --git a/doc/stdenv/multiple-output.xml b/doc/stdenv/multiple-output.xml index 83275bb2fbd..51e1cc2e024 100644 --- a/doc/stdenv/multiple-output.xml +++ b/doc/stdenv/multiple-output.xml @@ -106,7 +106,7 @@ </para> <para> - The reason for why <literal>glibc</literal> deviates from the convention is because referencing a library provided by <literal>glibc</literal> is a very common operation among Nix packages. For instance, third-party executables packaged by Nix are typically patched and relinked with the relevant version of <literal>glibc</literal> libraries from Nix packages (please see the documentation on <link xlink:href="https://nixos.org/patchelf.html">patchelf</link> for more details). + The reason for why <literal>glibc</literal> deviates from the convention is because referencing a library provided by <literal>glibc</literal> is a very common operation among Nix packages. For instance, third-party executables packaged by Nix are typically patched and relinked with the relevant version of <literal>glibc</literal> libraries from Nix packages (please see the documentation on <link xlink:href="https://github.com/NixOS/patchelf/blob/master/README">patchelf</link> for more details). </para> </section> diff --git a/doc/stdenv/stdenv.xml b/doc/stdenv/stdenv.xml index 1e97bf6157b..f97c2a145af 100644 --- a/doc/stdenv/stdenv.xml +++ b/doc/stdenv/stdenv.xml @@ -14,7 +14,7 @@ stdenv.mkDerivation { name = "libfoo-1.2.3"; src = fetchurl { - url = http://example.org/libfoo-1.2.3.tar.bz2; + url = "http://example.org/libfoo-1.2.3.tar.bz2"; sha256 = "0x2g1jqygyr5wiwg4ma1nd7w4ydpy82z9gkcv8vh2v8dn3y58v5m"; }; }</programlisting> @@ -145,7 +145,7 @@ genericBuild </listitem> <listitem> <para> - GNU Make. It has been patched to provide <quote>nested</quote> output that can be fed into the <command>nix-log2xml</command> command and <command>log2html</command> stylesheet to create a structured, readable output of the build steps performed by Make. + GNU Make. </para> </listitem> <listitem> @@ -254,7 +254,7 @@ let f(h, h + 1, i) = i + h <variablelist> <title>Variables specifying dependencies</title> - <varlistentry> + <varlistentry xml:id="var-stdenv-depsBuildBuild"> <term> <varname>depsBuildBuild</varname> </term> @@ -267,7 +267,7 @@ let f(h, h + 1, i) = i + h </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-nativeBuildInputs"> <term> <varname>nativeBuildInputs</varname> </term> @@ -280,7 +280,7 @@ let f(h, h + 1, i) = i + h </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-depsBuildTarget"> <term> <varname>depsBuildTarget</varname> </term> @@ -296,7 +296,7 @@ let f(h, h + 1, i) = i + h </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-depsHostHost"> <term> <varname>depsHostHost</varname> </term> @@ -306,7 +306,7 @@ let f(h, h + 1, i) = i + h </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-buildInputs"> <term> <varname>buildInputs</varname> </term> @@ -319,7 +319,7 @@ let f(h, h + 1, i) = i + h </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-depsTargetTarget"> <term> <varname>depsTargetTarget</varname> </term> @@ -329,7 +329,7 @@ let f(h, h + 1, i) = i + h </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-depsBuildBuildPropagated"> <term> <varname>depsBuildBuildPropagated</varname> </term> @@ -339,7 +339,7 @@ let f(h, h + 1, i) = i + h </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-propagatedNativeBuildInputs"> <term> <varname>propagatedNativeBuildInputs</varname> </term> @@ -349,7 +349,7 @@ let f(h, h + 1, i) = i + h </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-depsBuildTargetPropagated"> <term> <varname>depsBuildTargetPropagated</varname> </term> @@ -359,7 +359,7 @@ let f(h, h + 1, i) = i + h </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-depsHostHostPropagated"> <term> <varname>depsHostHostPropagated</varname> </term> @@ -369,7 +369,7 @@ let f(h, h + 1, i) = i + h </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-propagatedBuildInputs"> <term> <varname>propagatedBuildInputs</varname> </term> @@ -379,7 +379,7 @@ let f(h, h + 1, i) = i + h </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-depsTargetTargetPropagated"> <term> <varname>depsTargetTargetPropagated</varname> </term> @@ -396,7 +396,7 @@ let f(h, h + 1, i) = i + h <variablelist> <title>Variables affecting <literal>stdenv</literal> initialisation</title> - <varlistentry> + <varlistentry xml:id="var-stdenv-NIX_DEBUG"> <term> <varname>NIX_DEBUG</varname> </term> @@ -410,7 +410,7 @@ let f(h, h + 1, i) = i + h <variablelist> <title>Attributes affecting build properties</title> - <varlistentry> + <varlistentry xml:id="var-stdenv-enableParallelBuilding"> <term> <varname>enableParallelBuilding</varname> </term> @@ -427,7 +427,7 @@ let f(h, h + 1, i) = i + h <variablelist> <title>Special variables</title> - <varlistentry> + <varlistentry xml:id="var-stdenv-passthru"> <term> <varname>passthru</varname> </term> @@ -504,7 +504,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] There are a number of variables that control what phases are executed and in what order: <variablelist> <title>Variables affecting phase control</title> - <varlistentry> + <varlistentry xml:id="var-stdenv-phases"> <term> <varname>phases</varname> </term> @@ -517,7 +517,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-prePhases"> <term> <varname>prePhases</varname> </term> @@ -527,7 +527,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-preConfigurePhases"> <term> <varname>preConfigurePhases</varname> </term> @@ -537,7 +537,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-preBuildPhases"> <term> <varname>preBuildPhases</varname> </term> @@ -547,7 +547,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-preInstallPhases"> <term> <varname>preInstallPhases</varname> </term> @@ -557,7 +557,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-preFixupPhases"> <term> <varname>preFixupPhases</varname> </term> @@ -567,7 +567,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-preDistPhases"> <term> <varname>preDistPhases</varname> </term> @@ -577,7 +577,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-postPhases"> <term> <varname>postPhases</varname> </term> @@ -635,7 +635,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] <variablelist> <title>Variables controlling the unpack phase</title> - <varlistentry> + <varlistentry xml:id="var-stdenv-src"> <term> <varname>srcs</varname> / <varname>src</varname> </term> @@ -645,7 +645,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-sourceRoot"> <term> <varname>sourceRoot</varname> </term> @@ -655,7 +655,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-setSourceRoot"> <term> <varname>setSourceRoot</varname> </term> @@ -665,7 +665,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-preUnpack"> <term> <varname>preUnpack</varname> </term> @@ -675,7 +675,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-postUnpack"> <term> <varname>postUnpack</varname> </term> @@ -685,7 +685,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontUnpack"> <term> <varname>dontUnpack</varname> </term> @@ -695,7 +695,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontMakeSourcesWritable"> <term> <varname>dontMakeSourcesWritable</varname> </term> @@ -705,7 +705,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-unpackCmd"> <term> <varname>unpackCmd</varname> </term> @@ -727,7 +727,17 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] <variablelist> <title>Variables controlling the patch phase</title> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontPatch"> + <term> + <varname>dontPatch</varname> + </term> + <listitem> + <para> + Set to true to skip the patch phase. + </para> + </listitem> + </varlistentry> + <varlistentry xml:id="var-stdenv-patches"> <term> <varname>patches</varname> </term> @@ -737,7 +747,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-patchFlags"> <term> <varname>patchFlags</varname> </term> @@ -747,7 +757,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-prePatch"> <term> <varname>prePatch</varname> </term> @@ -757,7 +767,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-postPatch"> <term> <varname>postPatch</varname> </term> @@ -779,7 +789,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] <variablelist> <title>Variables controlling the configure phase</title> - <varlistentry> + <varlistentry xml:id="var-stdenv-configureScript"> <term> <varname>configureScript</varname> </term> @@ -789,7 +799,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-configureFlags"> <term> <varname>configureFlags</varname> </term> @@ -799,7 +809,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontConfigure"> <term> <varname>dontConfigure</varname> </term> @@ -809,7 +819,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-configureFlagsArray"> <term> <varname>configureFlagsArray</varname> </term> @@ -819,7 +829,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontAddPrefix"> <term> <varname>dontAddPrefix</varname> </term> @@ -829,7 +839,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-prefix"> <term> <varname>prefix</varname> </term> @@ -839,7 +849,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-prefixKey"> <term> <varname>prefixKey</varname> </term> @@ -849,7 +859,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontAddDisableDepTrack"> <term> <varname>dontAddDisableDepTrack</varname> </term> @@ -859,7 +869,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontFixLibtool"> <term> <varname>dontFixLibtool</varname> </term> @@ -875,7 +885,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontDisableStatic"> <term> <varname>dontDisableStatic</varname> </term> @@ -888,7 +898,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-configurePlatforms"> <term> <varname>configurePlatforms</varname> </term> @@ -903,7 +913,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-preConfigure"> <term> <varname>preConfigure</varname> </term> @@ -913,7 +923,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-postConfigure"> <term> <varname>postConfigure</varname> </term> @@ -935,7 +945,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] <variablelist> <title>Variables controlling the build phase</title> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontBuild"> <term> <varname>dontBuild</varname> </term> @@ -945,7 +955,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-makefile"> <term> <varname>makefile</varname> </term> @@ -955,7 +965,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-makeFlags"> <term> <varname>makeFlags</varname> </term> @@ -973,7 +983,7 @@ makeFlags = [ "PREFIX=$(out)" ]; </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-makeFlagsArray"> <term> <varname>makeFlagsArray</varname> </term> @@ -989,7 +999,7 @@ preBuild = '' </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-buildFlags"> <term> <varname>buildFlags</varname> / <varname>buildFlagsArray</varname> </term> @@ -999,7 +1009,7 @@ preBuild = '' </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-preBuild"> <term> <varname>preBuild</varname> </term> @@ -1009,7 +1019,7 @@ preBuild = '' </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-postBuild"> <term> <varname>postBuild</varname> </term> @@ -1039,7 +1049,7 @@ preBuild = '' <variablelist> <title>Variables controlling the check phase</title> - <varlistentry> + <varlistentry xml:id="var-stdenv-doCheck"> <term> <varname>doCheck</varname> </term> @@ -1057,11 +1067,11 @@ preBuild = '' </term> <listitem> <para> - See the build phase for details. + See the <link xlink:href="#var-stdenv-makeFlags">build phase</link> for details. </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-checkTarget"> <term> <varname>checkTarget</varname> </term> @@ -1071,7 +1081,7 @@ preBuild = '' </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-checkFlags"> <term> <varname>checkFlags</varname> / <varname>checkFlagsArray</varname> </term> @@ -1081,7 +1091,7 @@ preBuild = '' </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-checkInputs"> <term> <varname>checkInputs</varname> </term> @@ -1091,7 +1101,7 @@ preBuild = '' </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-preCheck"> <term> <varname>preCheck</varname> </term> @@ -1101,7 +1111,7 @@ preBuild = '' </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-postCheck"> <term> <varname>postCheck</varname> </term> @@ -1123,7 +1133,7 @@ preBuild = '' <variablelist> <title>Variables controlling the install phase</title> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontInstall"> <term> <varname>dontInstall</varname> </term> @@ -1139,11 +1149,11 @@ preBuild = '' </term> <listitem> <para> - See the build phase for details. + See the <link xlink:href="#var-stdenv-makeFlags">build phase</link> for details. </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-installTargets"> <term> <varname>installTargets</varname> </term> @@ -1155,7 +1165,7 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-installFlags"> <term> <varname>installFlags</varname> / <varname>installFlagsArray</varname> </term> @@ -1165,7 +1175,7 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-preInstall"> <term> <varname>preInstall</varname> </term> @@ -1175,7 +1185,7 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-postInstall"> <term> <varname>postInstall</varname> </term> @@ -1219,7 +1229,7 @@ installTargets = "install-bin install-doc";</programlisting> <variablelist> <title>Variables controlling the fixup phase</title> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontFixup"> <term> <varname>dontFixup</varname> </term> @@ -1229,7 +1239,7 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontStrip"> <term> <varname>dontStrip</varname> </term> @@ -1239,7 +1249,7 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontStripHost"> <term> <varname>dontStripHost</varname> </term> @@ -1249,7 +1259,7 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontStripTarget"> <term> <varname>dontStripTarget</varname> </term> @@ -1259,7 +1269,7 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontMoveSbin"> <term> <varname>dontMoveSbin</varname> </term> @@ -1269,7 +1279,7 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-stripAllList"> <term> <varname>stripAllList</varname> </term> @@ -1279,7 +1289,7 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-stripAllFlags"> <term> <varname>stripAllFlags</varname> </term> @@ -1289,17 +1299,17 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-stripDebugList"> <term> <varname>stripDebugList</varname> </term> <listitem> <para> - List of directories to search for libraries and executables from which only debugging-related symbols should be stripped. It defaults to <literal>lib bin sbin</literal>. + List of directories to search for libraries and executables from which only debugging-related symbols should be stripped. It defaults to <literal>lib lib32 lib64 libexec bin sbin</literal>. </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-stripDebugFlags"> <term> <varname>stripDebugFlags</varname> </term> @@ -1309,7 +1319,7 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontPatchELF"> <term> <varname>dontPatchELF</varname> </term> @@ -1319,7 +1329,7 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontPatchShebangs"> <term> <varname>dontPatchShebangs</varname> </term> @@ -1329,7 +1339,7 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontPruneLibtoolFiles"> <term> <varname>dontPruneLibtoolFiles</varname> </term> @@ -1339,7 +1349,7 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-forceShare"> <term> <varname>forceShare</varname> </term> @@ -1349,7 +1359,7 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-setupHook"> <term> <varname>setupHook</varname> </term> @@ -1360,7 +1370,7 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-preFixup"> <term> <varname>preFixup</varname> </term> @@ -1370,7 +1380,7 @@ installTargets = "install-bin install-doc";</programlisting> </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-postFixup"> <term> <varname>postFixup</varname> </term> @@ -1409,7 +1419,7 @@ set debug-file-directory ~/.nix-profile/lib/debug <variablelist> <title>Variables controlling the installCheck phase</title> - <varlistentry> + <varlistentry xml:id="var-stdenv-doInstallCheck"> <term> <varname>doInstallCheck</varname> </term> @@ -1421,7 +1431,7 @@ set debug-file-directory ~/.nix-profile/lib/debug </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-installCheckTarget"> <term> <varname>installCheckTarget</varname> </term> @@ -1431,7 +1441,7 @@ set debug-file-directory ~/.nix-profile/lib/debug </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-installCheckFlags"> <term> <varname>installCheckFlags</varname> / <varname>installCheckFlagsArray</varname> </term> @@ -1441,7 +1451,7 @@ set debug-file-directory ~/.nix-profile/lib/debug </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-installCheckInputs"> <term> <varname>installCheckInputs</varname> </term> @@ -1451,7 +1461,7 @@ set debug-file-directory ~/.nix-profile/lib/debug </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-preInstallCheck"> <term> <varname>preInstallCheck</varname> </term> @@ -1461,7 +1471,7 @@ set debug-file-directory ~/.nix-profile/lib/debug </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-postInstallCheck"> <term> <varname>postInstallCheck</varname> </term> @@ -1483,7 +1493,7 @@ set debug-file-directory ~/.nix-profile/lib/debug <variablelist> <title>Variables controlling the distribution phase</title> - <varlistentry> + <varlistentry xml:id="var-stdenv-distTarget"> <term> <varname>distTarget</varname> </term> @@ -1493,7 +1503,7 @@ set debug-file-directory ~/.nix-profile/lib/debug </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-distFlags"> <term> <varname>distFlags</varname> / <varname>distFlagsArray</varname> </term> @@ -1503,7 +1513,7 @@ set debug-file-directory ~/.nix-profile/lib/debug </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-tarballs"> <term> <varname>tarballs</varname> </term> @@ -1513,7 +1523,7 @@ set debug-file-directory ~/.nix-profile/lib/debug </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-dontCopyDist"> <term> <varname>dontCopyDist</varname> </term> @@ -1523,7 +1533,7 @@ set debug-file-directory ~/.nix-profile/lib/debug </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-preDist"> <term> <varname>preDist</varname> </term> @@ -1533,7 +1543,7 @@ set debug-file-directory ~/.nix-profile/lib/debug </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="var-stdenv-postDist"> <term> <varname>postDist</varname> </term> @@ -1991,7 +2001,7 @@ addEnvHooks "$hostOffset" myBashFunction </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="setup-hook-autopatchelfhook"> <term> autoPatchelfHook </term> @@ -2000,15 +2010,16 @@ addEnvHooks "$hostOffset" myBashFunction This is a special setup hook which helps in packaging proprietary software in that it automatically tries to find missing shared library dependencies of ELF files based on the given <varname>buildInputs</varname> and <varname>nativeBuildInputs</varname>. </para> <para> - You can also specify a <envar>runtimeDependencies</envar> environment variable which lists dependencies that are unconditionally added to all executables. - </para> - <para> + You can also specify a <varname>runtimeDependencies</varname> variable which lists dependencies to be unconditionally added to <glossterm>rpath</glossterm> of all executables. This is useful for programs that use <citerefentry> <refentrytitle>dlopen</refentrytitle> <manvolnum>3</manvolnum> </citerefentry> to load libraries at runtime. </para> <para> - In certain situations you may want to run the main command (<command>autoPatchelf</command>) of the setup hook on a file or a set of directories instead of unconditionally patching all outputs. This can be done by setting the <envar>dontAutoPatchelf</envar> environment variable to a non-empty value. + In certain situations you may want to run the main command (<command>autoPatchelf</command>) of the setup hook on a file or a set of directories instead of unconditionally patching all outputs. This can be done by setting the <varname>dontAutoPatchelf</varname> environment variable to a non-empty value. + </para> + <para> + By default <command>autoPatchelf</command> will fail as soon as any ELF file requires a dependency which cannot be resolved via the given build inputs. In some situations you might prefer to just leave missing dependencies unpatched and continue to patch the rest. This can be achieved by setting the <envar>autoPatchelfIgnoreMissingDeps</envar> environment variable to a non-empty value. </para> <para> The <command>autoPatchelf</command> command also recognizes a <parameter class="command">--no-recurse</parameter> command line flag, which prevents it from recursing into subdirectories. @@ -2075,6 +2086,16 @@ postInstall = '' </varlistentry> <varlistentry> <term> + validatePkgConfig + </term> + <listitem> + <para> + The <literal>validatePkgConfig</literal> hook validates all pkg-config (<filename>.pc</filename>) files in a package. This helps catching some common errors in pkg-config files, such as undefined variables. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> cmake </term> <listitem> diff --git a/doc/using/configuration.xml b/doc/using/configuration.xml index f4d6e911006..b670f78f28b 100644 --- a/doc/using/configuration.xml +++ b/doc/using/configuration.xml @@ -85,19 +85,19 @@ <title>Installing packages on unsupported systems</title> <para> - There are also two ways to try compiling a package which has been marked as unsuported for the given system. + There are also two ways to try compiling a package which has been marked as unsupported for the given system. </para> <itemizedlist> <listitem> <para> - For allowing the build of a broken package once, you can use an environment variable for a single invocation of the nix tools: + For allowing the build of an unsupported package once, you can use an environment variable for a single invocation of the nix tools: <programlisting>$ export NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM=1</programlisting> </para> </listitem> <listitem> <para> - For permanently allowing broken packages to be built, you may add <literal>allowUnsupportedSystem = true;</literal> to your user's configuration file, like this: + For permanently allowing unsupported packages to be built, you may add <literal>allowUnsupportedSystem = true;</literal> to your user's configuration file, like this: <programlisting> { allowUnsupportedSystem = true; @@ -162,10 +162,10 @@ </programlisting> </para> <para> - The following example configuration blacklists the <literal>gpl3</literal> and <literal>agpl3</literal> licenses: + The following example configuration blacklists the <literal>gpl3Only</literal> and <literal>agpl3Only</literal> licenses: <programlisting> { - blacklistedLicenses = with stdenv.lib.licenses; [ agpl3 gpl3 ]; + blacklistedLicenses = with stdenv.lib.licenses; [ agpl3Only gpl3Only ]; } </programlisting> </para> @@ -387,7 +387,7 @@ fi </screen> <para> - Now just run <literal>source $HOME/.profile</literal> and you can starting loading man pages from your environent. + Now just run <literal>source $HOME/.profile</literal> and you can starting loading man pages from your environment. </para> </section> diff --git a/doc/using/overlays.xml b/doc/using/overlays.xml index 26a888368ab..7f6ee040c7c 100644 --- a/doc/using/overlays.xml +++ b/doc/using/overlays.xml @@ -137,4 +137,144 @@ self: super: Overlays are similar to other methods for customizing Nixpkgs, in particular the <literal>packageOverrides</literal> attribute described in <xref linkend="sec-modify-via-packageOverrides"/>. Indeed, <literal>packageOverrides</literal> acts as an overlay with only the <varname>super</varname> argument. It is therefore appropriate for basic use, but overlays are more powerful and easier to distribute. </para> </section> + <section xml:id="sec-overlays-alternatives"> + <title>Using overlays to configure alternatives</title> + <para> + Certain software packages have different implementations of the + same interface. Other distributions have functionality to switch + between these. For example, Debian provides <link + xlink:href="https://wiki.debian.org/DebianAlternatives">DebianAlternatives</link>. + Nixpkgs has what we call <literal>alternatives</literal>, which + are configured through overlays. + </para> + <section xml:id="sec-overlays-alternatives-blas-lapack"> + <title>BLAS/LAPACK</title> + <para> + In Nixpkgs, we have multiple implementations of the BLAS/LAPACK + numerical linear algebra interfaces. They are: + </para> + <itemizedlist> + <listitem> + <para> + <link xlink:href="https://www.openblas.net/">OpenBLAS</link> + </para> + <para> + The Nixpkgs attribute is <literal>openblas</literal> for + ILP64 (integer width = 64 bits) and + <literal>openblasCompat</literal> for LP64 (integer width = + 32 bits). <literal>openblasCompat</literal> is the default. + </para> + </listitem> + <listitem> + <para> + <link xlink:href="http://www.netlib.org/lapack/">LAPACK + reference</link> (also provides BLAS) + </para> + <para> + The Nixpkgs attribute is <literal>lapack-reference</literal>. + </para> + </listitem> + <listitem> + <para> + <link + xlink:href="https://software.intel.com/en-us/mkl">Intel + MKL</link> (only works on the x86_64 architecture, unfree) + </para> + <para> + The Nixpkgs attribute is <literal>mkl</literal>. + </para> + </listitem> + <listitem> + <para> + <link + xlink:href="https://developer.amd.com/amd-aocl/blas-library/">AMD + BLIS/LIBFLAME</link> (optimized for modern AMD x86_64 CPUs) + </para> + <para> + The AMD BLIS library, with attribute <literal>amd-blis</literal>, + provides a BLAS implementation. The complementary AMD LIBFLAME + library, with attribute <literal>amd-libflame</literal>, provides + a LAPACK implementation. + </para> + </listitem> + </itemizedlist> + <para> + Introduced in <link + xlink:href="https://github.com/NixOS/nixpkgs/pull/83888">PR + #83888</link>, we are able to override the <literal>blas</literal> + and <literal>lapack</literal> packages to use different implementations, + through the <literal>blasProvider</literal> and + <literal>lapackProvider</literal> argument. This can be used + to select a different provider. BLAS providers will have + symlinks in <literal>$out/lib/libblas.so.3</literal> and + <literal>$out/lib/libcblas.so.3</literal> to their respective + BLAS libraries. Likewise, LAPACK providers will have symlinks + in <literal>$out/lib/liblapack.so.3</literal> and + <literal>$out/lib/liblapacke.so.3</literal> to their respective + LAPACK libraries. For example, Intel MKL is both a BLAS and + LAPACK provider. An overlay can be created to use Intel MKL + that looks like: + </para> + <programlisting> +self: super: + +{ + blas = super.blas.override { + blasProvider = self.mkl; + } + lapack = super.lapack.override { + lapackProvider = self.mkl; + } +} + </programlisting> + <para> + This overlay uses Intel’s MKL library for both BLAS and LAPACK + interfaces. Note that the same can be accomplished at runtime + using <literal>LD_LIBRARY_PATH</literal> of + <literal>libblas.so.3</literal> and + <literal>liblapack.so.3</literal>. For instance: + </para> + <programlisting> +$ LD_LIBRARY_PATH=$(nix-build -A mkl)/lib:$LD_LIBRARY_PATH nix-shell -p octave --run octave + </programlisting> + <para> + Intel MKL requires an <literal>openmp</literal> implementation + when running with multiple processors. By default, + <literal>mkl</literal> will use Intel’s <literal>iomp</literal> + implementation if no other is specified, but this is a + runtime-only dependency and binary compatible with the LLVM + implementation. To use that one instead, Intel recommends users + set it with <literal>LD_PRELOAD</literal>. Note that + <literal>mkl</literal> is only available on + <literal>x86_64-linux</literal> and + <literal>x86_64-darwin</literal>. Moreover, Hydra is not + building and distributing pre-compiled binaries using it. + </para> + <para> + For BLAS/LAPACK switching to work correctly, all packages must + depend on <literal>blas</literal> or <literal>lapack</literal>. + This ensures that only one BLAS/LAPACK library is used at one + time. There are two versions versions of BLAS/LAPACK currently + in the wild, <literal>LP64</literal> (integer size = 32 bits) + and <literal>ILP64</literal> (integer size = 64 bits). Some + software needs special flags or patches to work with + <literal>ILP64</literal>. You can check if + <literal>ILP64</literal> is used in Nixpkgs with + <varname>blas.isILP64</varname> and + <varname>lapack.isILP64</varname>. Some software does NOT work + with <literal>ILP64</literal>, and derivations need to specify + an assertion to prevent this. You can prevent + <literal>ILP64</literal> from being used with the following: + </para> + <programlisting> +{ stdenv, blas, lapack, ... }: + +assert (!blas.isILP64) && (!lapack.isILP64); + +stdenv.mkDerivation { + ... +} + </programlisting> + </section> + </section> </chapter> |