diff options
Diffstat (limited to 'doc')
-rw-r--r-- | doc/languages-frameworks/index.xml | 2 | ||||
-rw-r--r-- | doc/languages-frameworks/lua.section.md | 252 | ||||
-rw-r--r-- | doc/languages-frameworks/lua.xml | 36 | ||||
-rw-r--r-- | doc/languages-frameworks/python.section.md | 8 | ||||
-rw-r--r-- | doc/languages-frameworks/rust.section.md | 84 | ||||
-rw-r--r-- | doc/stdenv/stdenv.xml | 18 | ||||
-rw-r--r-- | doc/using/configuration.xml | 2 |
7 files changed, 355 insertions, 47 deletions
diff --git a/doc/languages-frameworks/index.xml b/doc/languages-frameworks/index.xml index 2414956995c..728a38c264a 100644 --- a/doc/languages-frameworks/index.xml +++ b/doc/languages-frameworks/index.xml @@ -18,7 +18,7 @@ <xi:include href="idris.section.xml" /> <xi:include href="ios.section.xml" /> <xi:include href="java.xml" /> - <xi:include href="lua.xml" /> + <xi:include href="lua.section.xml" /> <xi:include href="node.section.xml" /> <xi:include href="ocaml.xml" /> <xi:include href="perl.xml" /> diff --git a/doc/languages-frameworks/lua.section.md b/doc/languages-frameworks/lua.section.md new file mode 100644 index 00000000000..a0e9917b8ec --- /dev/null +++ b/doc/languages-frameworks/lua.section.md @@ -0,0 +1,252 @@ +--- +title: Lua +author: Matthieu Coudron +date: 2019-02-05 +--- + +# User's Guide to Lua Infrastructure + +## Using Lua + +### Overview of Lua + +Several versions of the Lua interpreter are available: luajit, lua 5.1, 5.2, 5.3. +The attribute `lua` refers to the default interpreter, it is also possible to refer to specific versions, e.g. `lua5_2` refers to Lua 5.2. + +Lua libraries are in separate sets, with one set per interpreter version. + +The interpreters have several common attributes. One of these attributes is +`pkgs`, which is a package set of Lua libraries for this specific +interpreter. E.g., the `busted` package corresponding to the default interpreter +is `lua.pkgs.busted`, and the lua 5.2 version is `lua5_2.pkgs.busted`. +The main package set contains aliases to these package sets, e.g. +`luaPackages` refers to `lua5_1.pkgs` and `lua52Packages` to +`lua5_2.pkgs`. + +### Installing Lua and packages + +#### Lua environment defined in separate `.nix` file + +Create a file, e.g. `build.nix`, with the following expression +```nix +with import <nixpkgs> {}; + +lua5_2.withPackages (ps: with ps; [ busted luafilesystem ]) +``` +and install it in your profile with +```shell +nix-env -if build.nix +``` +Now you can use the Lua interpreter, as well as the extra packages (`busted`, +`luafilesystem`) that you added to the environment. + +#### Lua environment defined in `~/.config/nixpkgs/config.nix` + +If you prefer to, you could also add the environment as a package override to the Nixpkgs set, e.g. +using `config.nix`, +```nix +{ # ... + + packageOverrides = pkgs: with pkgs; { + myLuaEnv = lua5_2.withPackages (ps: with ps; [ busted luafilesystem ]); + }; +} +``` +and install it in your profile with +```shell +nix-env -iA nixpkgs.myLuaEnv +``` +The environment is is installed by referring to the attribute, and considering +the `nixpkgs` channel was used. + +#### Lua environment defined in `/etc/nixos/configuration.nix` + +For the sake of completeness, here's another example how to install the environment system-wide. + +```nix +{ # ... + + environment.systemPackages = with pkgs; [ + (lua.withPackages(ps: with ps; [ busted luafilesystem ])) + ]; +} +``` + +### How to override a Lua package using overlays? + +Use the following overlay template: + +```nix +final: prev: +{ + + lua = prev.lua.override { + packageOverrides = luaself: luaprev: { + + luarocks-nix = luaprev.luarocks-nix.overrideAttrs(oa: { + pname = "luarocks-nix"; + src = /home/my_luarocks/repository; + }); + }; + + luaPackages = lua.pkgs; +} +``` + +### Temporary Lua environment with `nix-shell` + + +There are two methods for loading a shell with Lua packages. The first and recommended method +is to create an environment with `lua.buildEnv` or `lua.withPackages` and load that. E.g. +```sh +$ nix-shell -p 'lua.withPackages(ps: with ps; [ busted luafilesystem ])' +``` +opens a shell from which you can launch the interpreter +```sh +[nix-shell:~] lua +``` +The other method, which is not recommended, does not create an environment and requires you to list the packages directly, + +```sh +$ nix-shell -p lua.pkgs.busted lua.pkgs.luafilesystem +``` +Again, it is possible to launch the interpreter from the shell. +The Lua interpreter has the attribute `pkgs` which contains all Lua libraries for that specific interpreter. + + +## Developing with Lua + +Now that you know how to get a working Lua environment with Nix, it is time +to go forward and start actually developing with Lua. There are two ways to +package lua software, either it is on luarocks and most of it can be taken care +of by the luarocks2nix converter or the packaging has to be done manually. +Let's present the luarocks way first and the manual one in a second time. + +### Packaging a library on luarocks + +[Luarocks.org](www.luarocks.org) is the main repository of lua packages. +The site proposes two types of packages, the rockspec and the src.rock +(equivalent of a [rockspec](https://github.com/luarocks/luarocks/wiki/Rockspec-format) but with the source). +These packages can have different build types such as `cmake`, `builtin` etc . + +Luarocks-based packages are generated in pkgs/development/lua-modules/generated-packages.nix from +the whitelist maintainers/scripts/luarocks-packages.csv and updated by running maintainers/scripts/update-luarocks-packages. + +[luarocks2nix](https://github.com/nix-community/luarocks) is a tool capable of generating nix derivations from both rockspec and src.rock (and favors the src.rock). +The automation only goes so far though and some packages need to be customized. +These customizations go in `pkgs/development/lua-modules/overrides.nix`. +For instance if the rockspec defines `external_dependencies`, these need to be manually added in in its rockspec file then it won't work. + +You can try converting luarocks packages to nix packages with the command `nix-shell -p luarocks-nix` and then `luarocks nix PKG_NAME`. +Nix rely on luarocks to install lua packages, basically it runs: +`luarocks make --deps-mode=none --tree $out` + +#### Packaging a library manually + +You can develop your package as you usually would, just don't forget to wrap it +within a `toLuaModule` call, for instance +```nix +mynewlib = toLuaModule ( stdenv.mkDerivation { ... }); +``` + +There is also the `buildLuaPackage` function that can be used when lua modules +are not packaged for luarocks. You can see a few examples at `pkgs/top-level/lua-packages.nix`. + +## Lua Reference + +### Lua interpreters + +Versions 5.1, 5.2 and 5.3 of the lua interpreter are available as +respectively `lua5_1`, `lua5_2` and `lua5_3`. Luajit is available too. +The Nix expressions for the interpreters can be found in `pkgs/development/interpreters/lua-5`. + + +#### Attributes on lua interpreters packages + +Each interpreter has the following attributes: + +- `interpreter`. Alias for `${pkgs.lua}/bin/lua`. +- `buildEnv`. Function to build lua interpreter environments with extra packages bundled together. See section *lua.buildEnv function* for usage and documentation. +- `withPackages`. Simpler interface to `buildEnv`. +- `pkgs`. Set of Lua packages for that specific interpreter. The package set can be modified by overriding the interpreter and passing `packageOverrides`. + + +#### `buildLuarocksPackage` function + +The `buildLuarocksPackage` function is implemented in `pkgs/development/interpreters/lua-5/build-lua-package.nix` +The following is an example: +```nix +luaposix = buildLuarocksPackage { + pname = "luaposix"; + version = "34.0.4-1"; + + src = fetchurl { + url = "https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/luaposix-34.0.4-1.src.rock"; + sha256 = "0yrm5cn2iyd0zjd4liyj27srphvy0gjrjx572swar6zqr4dwjqp2"; + }; + disabled = (luaOlder "5.1") || (luaAtLeast "5.4"); + propagatedBuildInputs = [ bit32 lua std_normalize ]; + + meta = with stdenv.lib; { + homepage = "https://github.com/luaposix/luaposix/"; + description = "Lua bindings for POSIX"; + maintainers = with maintainers; [ vyp lblasc ]; + license.fullName = "MIT/X11"; + }; +}; +``` + +The `buildLuarocksPackage` delegates most tasks to luarocks: + +* it adds `luarocks` as an unpacker for `src.rock` files (zip files really). +* configurePhase` writes a temporary luarocks configuration file which location +is exported via the environment variable `LUAROCKS_CONFIG`. +* the `buildPhase` does nothing. +* `installPhase` calls `luarocks make --deps-mode=none --tree $out` to build and +install the package +* In the `postFixup` phase, the `wrapLuaPrograms` bash function is called to + wrap all programs in the `$out/bin/*` directory to include `$PATH` + environment variable and add dependent libraries to script's `LUA_PATH` and + `LUA_CPATH`. + +By default `meta.platforms` is set to the same value as the interpreter unless overridden otherwise. + +#### `buildLuaApplication` function + +The `buildLuaApplication` function is practically the same as `buildLuaPackage`. +The difference is that `buildLuaPackage` by default prefixes the names of the packages with the version of the interpreter. +Because with an application we're not interested in multiple version the prefix is dropped. + +#### lua.withPackages function + +The `lua.withPackages` takes a function as an argument that is passed the set of lua packages and returns the list of packages to be included in the environment. +Using the `withPackages` function, the previous example for the luafilesystem environment can be written like this: +```nix +with import <nixpkgs> {}; + +lua.withPackages (ps: [ps.luafilesystem]) +``` + +`withPackages` passes the correct package set for the specific interpreter version as an argument to the function. In the above example, `ps` equals `luaPackages`. +But you can also easily switch to using `lua5_2`: +```nix +with import <nixpkgs> {}; + +lua5_2.withPackages (ps: [ps.lua]) +``` + +Now, `ps` is set to `lua52Packages`, matching the version of the interpreter. + + +### Possible Todos + +* export/use version specific variables such as `LUA_PATH_5_2`/`LUAROCKS_CONFIG_5_2` +* let luarocks check for dependencies via exporting the different rocktrees in temporary config + +### Lua Contributing guidelines + +Following rules should be respected: + +* Make sure libraries build for all Lua interpreters. +* Commit names of Lua libraries should reflect that they are Lua libraries, so write for example `luaPackages.luafilesystem: 1.11 -> 1.12`. + diff --git a/doc/languages-frameworks/lua.xml b/doc/languages-frameworks/lua.xml deleted file mode 100644 index bcca6b73753..00000000000 --- a/doc/languages-frameworks/lua.xml +++ /dev/null @@ -1,36 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xml:id="sec-language-lua"> - <title>Lua</title> - - <para> - Lua packages are built by the <varname>buildLuaPackage</varname> function. This function is implemented in <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/lua-modules/generic/default.nix"> <filename>pkgs/development/lua-modules/generic/default.nix</filename></link> and works similarly to <varname>buildPerlPackage</varname>. (See <xref linkend="sec-language-perl"/> for details.) - </para> - - <para> - Lua packages are defined in <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/lua-packages.nix"><filename>pkgs/top-level/lua-packages.nix</filename></link>. Most of them are simple. For example: -<programlisting> -fileSystem = buildLuaPackage { - name = "filesystem-1.6.2"; - src = fetchurl { - url = "https://github.com/keplerproject/luafilesystem/archive/v1_6_2.tar.gz"; - sha256 = "1n8qdwa20ypbrny99vhkmx8q04zd2jjycdb5196xdhgvqzk10abz"; - }; - meta = { - homepage = "https://github.com/keplerproject/luafilesystem"; - hydraPlatforms = stdenv.lib.platforms.linux; - maintainers = with maintainers; [ flosse ]; - }; -}; -</programlisting> - </para> - - <para> - Though, more complicated package should be placed in a seperate file in <link - xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/lua-modules"><filename>pkgs/development/lua-modules</filename></link>. - </para> - - <para> - Lua packages accept additional parameter <varname>disabled</varname>, which defines the condition of disabling package from luaPackages. For example, if package has <varname>disabled</varname> assigned to <literal>lua.luaversion != "5.1"</literal>, it will not be included in any luaPackages except lua51Packages, making it only be built for lua 5.1. - </para> -</section> diff --git a/doc/languages-frameworks/python.section.md b/doc/languages-frameworks/python.section.md index 838426afa04..dc10483ce69 100644 --- a/doc/languages-frameworks/python.section.md +++ b/doc/languages-frameworks/python.section.md @@ -643,7 +643,7 @@ and in this case the `python38` interpreter is automatically used. Versions 2.7, 3.5, 3.6, 3.7 and 3.8 of the CPython interpreter are available as respectively `python27`, `python35`, `python36`, `python37` and `python38`. The aliases `python2` and `python3` correspond to respectively `python27` and -`python37`. The default interpreter, `python`, maps to `python2`. The PyPy +`python38`. The default interpreter, `python`, maps to `python2`. The PyPy interpreters compatible with Python 2.7 and 3 are available as `pypy27` and `pypy3`, with aliases `pypy2` mapping to `pypy27` and `pypy` mapping to `pypy2`. The Nix expressions for the interpreters can be found in @@ -764,9 +764,6 @@ following are specific to `buildPythonPackage`: * `dontWrapPythonPrograms ? false`: Skip wrapping of Python programs. * `permitUserSite ? false`: Skip setting the `PYTHONNOUSERSITE` environment variable in wrapped programs. -* `installFlags ? []`: A list of strings. Arguments to be passed to `pip - install`. To pass options to `python setup.py install`, use - `--install-option`. E.g., `installFlags=["--install-option='--cpp_implementation'"]`. * `format ? "setuptools"`: Format of the source. Valid options are `"setuptools"`, `"pyproject"`, `"flit"`, `"wheel"`, and `"other"`. `"setuptools"` is for when the source has a `setup.py` and `setuptools` is @@ -782,6 +779,9 @@ following are specific to `buildPythonPackage`: * `namePrefix`: Prepends text to `${name}` parameter. In case of libraries, this defaults to `"python3.8-"` for Python 3.8, etc., and in case of applications to `""`. +* `pipInstallFlags ? []`: A list of strings. Arguments to be passed to `pip + install`. To pass options to `python setup.py install`, use + `--install-option`. E.g., `pipInstallFlags=["--install-option='--cpp_implementation'"]`. * `pythonPath ? []`: List of packages to be added into `$PYTHONPATH`. Packages in `pythonPath` are not propagated (contrary to `propagatedBuildInputs`). * `preShellHook`: Hook to execute commands before `shellHook`. diff --git a/doc/languages-frameworks/rust.section.md b/doc/languages-frameworks/rust.section.md index cec3373cbee..ec418c56335 100644 --- a/doc/languages-frameworks/rust.section.md +++ b/doc/languages-frameworks/rust.section.md @@ -75,6 +75,90 @@ pkgs.rustPlatform.buildRustPackage { } ``` +### Running package tests + +When using `buildRustPackage`, the `checkPhase` is enabled by default and runs +`cargo test` on the package to build. To make sure that we don't compile the +sources twice and to actually test the artifacts that will be used at runtime, +the tests will be ran in the `release` mode by default. + +However, in some cases the test-suite of a package doesn't work properly in the +`release` mode. For these situations, the mode for `checkPhase` can be changed like +so: + +```nix +rustPlatform.buildRustPackage { + /* ... */ + checkType = "debug"; +} +``` + +Please note that the code will be compiled twice here: once in `release` mode +for the `buildPhase`, and again in `debug` mode for the `checkPhase`. + +#### Tests relying on the structure of the `target/` directory + +Some tests may rely on the structure of the `target/` directory. Those tests +are likely to fail because we use `cargo --target` during the build. This means that +the artifacts +[are stored in `target/<architecture>/release/`](https://doc.rust-lang.org/cargo/guide/build-cache.html), +rather than in `target/release/`. + +This can only be worked around by patching the affected tests accordingly. + +#### Disabling package-tests + +In some instances, it may be necessary to disable testing altogether (with `doCheck = false;`): + +* If no tests exist -- the `checkPhase` should be explicitly disabled to skip + unnecessary build steps to speed up the build. +* If tests are highly impure (e.g. due to network usage). + +There will obviously be some corner-cases not listed above where it's sensible to disable tests. +The above are just guidelines, and exceptions may be granted on a case-by-case basis. + +However, please check if it's possible to disable a problematic subset of the +test suite and leave a comment explaining your reasoning. + +### Building a package in `debug` mode + +By default, `buildRustPackage` will use `release` mode for builds. If a package +should be built in `debug` mode, it can be configured like so: + +```nix +rustPlatform.buildRustPackage { + /* ... */ + buildType = "debug"; +} +``` + +In this scenario, the `checkPhase` will be ran in `debug` mode as well. + +### Custom `build`/`install`-procedures + +Some packages may use custom scripts for building/installing, e.g. with a `Makefile`. +In these cases, it's recommended to override the `buildPhase`/`installPhase`/`checkPhase`. + +Otherwise, some steps may fail because of the modified directory structure of `target/`. + +### Building a crate with an absent or out-of-date Cargo.lock file + +`buildRustPackage` needs a `Cargo.lock` file to get all dependencies in the +source code in a reproducible way. If it is missing or out-of-date one can use +the `cargoPatches` attribute to update or add it. + +``` +{ lib, rustPlatform, fetchFromGitHub }: + +rustPlatform.buildRustPackage rec { + (...) + cargoPatches = [ + # a patch file to add/update Cargo.lock in the source code + ./add-Cargo.lock.patch + ]; +} +``` + ## Compiling Rust crates using Nix instead of Cargo ### Simple operation diff --git a/doc/stdenv/stdenv.xml b/doc/stdenv/stdenv.xml index 65a343d7edd..ee93f39318b 100644 --- a/doc/stdenv/stdenv.xml +++ b/doc/stdenv/stdenv.xml @@ -2001,7 +2001,7 @@ addEnvHooks "$hostOffset" myBashFunction </para> </listitem> </varlistentry> - <varlistentry> + <varlistentry xml:id="setup-hook-autopatchelfhook"> <term> autoPatchelfHook </term> @@ -2010,15 +2010,13 @@ addEnvHooks "$hostOffset" myBashFunction This is a special setup hook which helps in packaging proprietary software in that it automatically tries to find missing shared library dependencies of ELF files based on the given <varname>buildInputs</varname> and <varname>nativeBuildInputs</varname>. </para> <para> - You can also specify a <envar>runtimeDependencies</envar> environment variable which lists dependencies that are unconditionally added to all executables. - </para> - <para> + You can also specify a <varname>runtimeDependencies</varname> variable which lists dependencies to be unconditionally added to <glossterm>rpath</glossterm> of all executables. This is useful for programs that use <citerefentry> <refentrytitle>dlopen</refentrytitle> <manvolnum>3</manvolnum> </citerefentry> to load libraries at runtime. </para> <para> - In certain situations you may want to run the main command (<command>autoPatchelf</command>) of the setup hook on a file or a set of directories instead of unconditionally patching all outputs. This can be done by setting the <envar>dontAutoPatchelf</envar> environment variable to a non-empty value. + In certain situations you may want to run the main command (<command>autoPatchelf</command>) of the setup hook on a file or a set of directories instead of unconditionally patching all outputs. This can be done by setting the <varname>dontAutoPatchelf</varname> environment variable to a non-empty value. </para> <para> The <command>autoPatchelf</command> command also recognizes a <parameter class="command">--no-recurse</parameter> command line flag, which prevents it from recursing into subdirectories. @@ -2085,6 +2083,16 @@ postInstall = '' </varlistentry> <varlistentry> <term> + validatePkgConfig + </term> + <listitem> + <para> + The <literal>validatePkgConfig</literal> hook validates all pkg-config (<filename>.pc</filename>) files in a package. This helps catching some common errors in pkg-config files, such as undefined variables. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> cmake </term> <listitem> diff --git a/doc/using/configuration.xml b/doc/using/configuration.xml index f4d6e911006..f19eddb5868 100644 --- a/doc/using/configuration.xml +++ b/doc/using/configuration.xml @@ -387,7 +387,7 @@ fi </screen> <para> - Now just run <literal>source $HOME/.profile</literal> and you can starting loading man pages from your environent. + Now just run <literal>source $HOME/.profile</literal> and you can starting loading man pages from your environment. </para> </section> |