diff options
Diffstat (limited to 'doc/stdenv/stdenv.chapter.md')
| -rw-r--r-- | doc/stdenv/stdenv.chapter.md | 112 |
1 files changed, 96 insertions, 16 deletions
diff --git a/doc/stdenv/stdenv.chapter.md b/doc/stdenv/stdenv.chapter.md index 15cb081a04e..26c43bd9e94 100644 --- a/doc/stdenv/stdenv.chapter.md +++ b/doc/stdenv/stdenv.chapter.md @@ -101,25 +101,62 @@ genericBuild ### Building a `stdenv` package in `nix-shell` {#sec-building-stdenv-package-in-nix-shell} -To build a `stdenv` package in a [`nix-shell`](https://nixos.org/manual/nix/unstable/command-ref/nix-shell.html), use +To build a `stdenv` package in a [`nix-shell`](https://nixos.org/manual/nix/unstable/command-ref/nix-shell.html), enter a shell, find the [phases](#sec-stdenv-phases) you wish to build, then invoke `genericBuild` manually: + +Go to an empty directory, invoke `nix-shell` with the desired package, and from inside the shell, set the output variables to a writable directory: ```bash +cd "$(mktemp -d)" nix-shell '<nixpkgs>' -A some_package -eval "${unpackPhase:-unpackPhase}" -cd $sourceRoot -eval "${patchPhase:-patchPhase}" -eval "${configurePhase:-configurePhase}" -eval "${buildPhase:-buildPhase}" +export out=$(pwd)/out +``` + +Next, invoke the desired parts of the build. +First, run the phases that generate a working copy of the sources, which will change directory to the sources for you: + +```bash +phases="${prePhases[*]:-} unpackPhase patchPhase" genericBuild +``` + +Then, run more phases up until the failure is reached. +For example, if the failure is in the build phase, the following phases would be required: + +```bash +phases="${preConfigurePhases[*]:-} configurePhase ${preBuildPhases[*]:-} buildPhase" genericBuild +``` + +Re-run a single phase as many times as necessary to examine the failure like so: + +```bash +phases="buildPhase" genericBuild ``` To modify a [phase](#sec-stdenv-phases), first print it with ```bash +echo "$buildPhase" +``` + +Or, if that is empty, for instance, if it is using a function: + +```bash type buildPhase ``` then change it in a text editor, and paste it back to the terminal. +::: {.note} +This method may have some inconsistencies in environment variables and behaviour compared to a normal build within the [Nix build sandbox](https://nixos.org/manual/nix/unstable/language/derivations#builder-execution). +The following is a non-exhaustive list of such differences: + +- `TMP`, `TMPDIR`, and similar variables likely point to non-empty directories that the build might conflict with files in. +- Output store paths are not writable, so the variables for outputs need to be overridden to writable paths. +- Other environment variables may be inconsistent with a `nix-build` either due to `nix-shell`'s initialization script or due to the use of `nix-shell` without the `--pure` option. + +If the build fails differently inside the shell than in the sandbox, consider using [`breakpointHook`](#breakpointhook) and invoking `nix-build` instead. +The [`--keep-failed`](https://nixos.org/manual/nix/unstable/command-ref/conf-file#opt--keep-failed) option for `nix-build` may also be useful to examine the build directory of a failed build. +::: + ## Tools provided by `stdenv` {#sec-tools-of-stdenv} The standard environment provides the following packages: @@ -282,7 +319,7 @@ let f(h, h + 1, i) = i + (if i <= 0 then h else h) let f(h, h + 1, i) = i + h ``` -This is where “sum-like” comes in from above: We can just sum all of the host offsets to get the host offset of the transitive dependency. The target offset is the transitive dependency is simply the host offset + 1, just as it was with the dependencies composed to make this transitive one; it can be ignored as it doesn’t add any new information. +This is where “sum-like” comes in from above: We can just sum all of the host offsets to get the host offset of the transitive dependency. The target offset is the transitive dependency is the host offset + 1, just as it was with the dependencies composed to make this transitive one; it can be ignored as it doesn’t add any new information. Because of the bounds checks, the uncommon cases are `h = t` and `h + 2 = t`. In the former case, the motivation for `mapOffset` is that since its host and target platforms are the same, no transitive dependency of it should be able to “discover” an offset greater than its reduced target offsets. `mapOffset` effectively “squashes” all its transitive dependencies’ offsets so that none will ever be greater than the target offset of the original `h = t` package. In the other case, `h + 1` is skipped over between the host and target offsets. Instead of squashing the offsets, we need to “rip” them apart so no transitive dependencies’ offset is that one. @@ -491,7 +528,7 @@ If the returned array contains exactly one object (e.g. `[{}]`), all values are ``` ::: -### Recursive attributes in `mkDerivation` {#mkderivation-recursive-attributes} +### Fixed-point arguments of `mkDerivation` {#mkderivation-recursive-attributes} If you pass a function to `mkDerivation`, it will receive as its argument the final arguments, including the overrides when reinvoked via `overrideAttrs`. For example: @@ -612,7 +649,7 @@ Zip files are unpacked using `unzip`. However, `unzip` is not in the standard en #### Directories in the Nix store {#directories-in-the-nix-store} -These are simply copied to the current directory. The hash part of the file name is stripped, e.g. `/nix/store/1wydxgby13cz...-my-sources` would be copied to `my-sources`. +These are copied to the current directory. The hash part of the file name is stripped, e.g. `/nix/store/1wydxgby13cz...-my-sources` would be copied to `my-sources`. Additional file types can be supported by setting the `unpackCmd` variable (see below). @@ -751,7 +788,7 @@ Hook executed at the end of the configure phase. ### The build phase {#build-phase} -The build phase is responsible for actually building the package (e.g. compiling it). The default `buildPhase` simply calls `make` if a file named `Makefile`, `makefile` or `GNUmakefile` exists in the current directory (or the `makefile` is explicitly set); otherwise it does nothing. +The build phase is responsible for actually building the package (e.g. compiling it). The default `buildPhase` calls `make` if a file named `Makefile`, `makefile` or `GNUmakefile` exists in the current directory (or the `makefile` is explicitly set); otherwise it does nothing. #### Variables controlling the build phase {#variables-controlling-the-build-phase} @@ -991,13 +1028,56 @@ Hook executed at the end of the fixup phase. If set to `true`, the standard environment will enable debug information in C/C++ builds. After installation, the debug information will be separated from the executables and stored in the output named `debug`. (This output is enabled automatically; you don’t need to set the `outputs` attribute explicitly.) To be precise, the debug information is stored in `debug/lib/debug/.build-id/XX/YYYY…`, where \<XXYYYY…\> is the \<build ID\> of the binary — a SHA-1 hash of the contents of the binary. Debuggers like GDB use the build ID to look up the separated debug information. -For example, with GDB, you can add +:::{.example #ex-gdb-debug-symbols-socat} -``` -set debug-file-directory ~/.nix-profile/lib/debug +# Enable debug symbols for use with GDB + +To make GDB find debug information for the `socat` package and its dependencies, you can use the following `shell.nix`: + +```nix +let + pkgs = import ./. { + config = {}; + overlays = [ + (final: prev: { + ncurses = prev.ncurses.overrideAttrs { separateDebugInfo = true; }; + readline = prev.readline.overrideAttrs { separateDebugInfo = true; }; + }) + ]; + }; + + myDebugInfoDirs = pkgs.symlinkJoin { + name = "myDebugInfoDirs"; + paths = with pkgs; [ + glibc.debug + ncurses.debug + openssl.debug + readline.debug + ]; + }; +in + pkgs.mkShell { + + NIX_DEBUG_INFO_DIRS = "${pkgs.lib.getLib myDebugInfoDirs}/lib/debug"; + + packages = [ + pkgs.gdb + pkgs.socat + ]; + + shellHook = '' + ${pkgs.lib.getBin pkgs.gdb}/bin/gdb ${pkgs.lib.getBin pkgs.socat}/bin/socat + ''; + } ``` -to `~/.gdbinit`. GDB will then be able to find debug information installed via `nix-env -i`. +This setup works as follows: +- Add [`overlays`](#chap-overlays) to the package set, since debug symbols are disabled for `ncurses` and `readline` by default. +- Create a derivation to combine all required debug symbols under one path with [`symlinkJoin`](#trivial-builder-symlinkJoin). +- Set the environment variable `NIX_DEBUG_INFO_DIRS` in the shell. Nixpkgs patches `gdb` to use it for looking up debug symbols. +- Run `gdb` on the `socat` binary on shell startup in the [`shellHook`](#sec-pkgs-mkShell). Here we use [`lib.getBin`](#function-library-lib.attrsets.getBin) to ensure that the correct derivation output is selected rather than the default one. + +::: ### The installCheck phase {#ssec-installCheck-phase} @@ -1237,7 +1317,7 @@ Nix itself considers a build-time dependency as merely something that should pre In order to alleviate this burden, the setup hook mechanism was written, where any package can include a shell script that \[by convention rather than enforcement by Nix\], any downstream reverse-dependency will source as part of its build process. That allows the downstream dependency to merely specify its dependencies, and lets those dependencies effectively initialize themselves. No boilerplate mirroring the list of dependencies is needed. -The setup hook mechanism is a bit of a sledgehammer though: a powerful feature with a broad and indiscriminate area of effect. The combination of its power and implicit use may be expedient, but isn’t without costs. Nix itself is unchanged, but the spirit of added dependencies being effect-free is violated even if the latter isn’t. For example, if a derivation path is mentioned more than once, Nix itself doesn’t care and simply makes sure the dependency derivation is already built just the same—depending is just needing something to exist, and needing is idempotent. However, a dependency specified twice will have its setup hook run twice, and that could easily change the build environment (though a well-written setup hook will therefore strive to be idempotent so this is in fact not observable). More broadly, setup hooks are anti-modular in that multiple dependencies, whether the same or different, should not interfere and yet their setup hooks may well do so. +The setup hook mechanism is a bit of a sledgehammer though: a powerful feature with a broad and indiscriminate area of effect. The combination of its power and implicit use may be expedient, but isn’t without costs. Nix itself is unchanged, but the spirit of added dependencies being effect-free is violated even if the latter isn’t. For example, if a derivation path is mentioned more than once, Nix itself doesn’t care and makes sure the dependency derivation is already built just the same—depending is just needing something to exist, and needing is idempotent. However, a dependency specified twice will have its setup hook run twice, and that could easily change the build environment (though a well-written setup hook will therefore strive to be idempotent so this is in fact not observable). More broadly, setup hooks are anti-modular in that multiple dependencies, whether the same or different, should not interfere and yet their setup hooks may well do so. The most typical use of the setup hook is actually to add other hooks which are then run (i.e. after all the setup hooks) on each dependency. For example, the C compiler wrapper’s setup hook feeds itself flags for each dependency that contains relevant libraries and headers. This is done by defining a bash function, and appending its name to one of `envBuildBuildHooks`, `envBuildHostHooks`, `envBuildTargetHooks`, `envHostHostHooks`, `envHostTargetHooks`, or `envTargetTargetHooks`. These 6 bash variables correspond to the 6 sorts of dependencies by platform (there’s 12 total but we ignore the propagated/non-propagated axis). @@ -1478,7 +1558,7 @@ This flag can break dynamic shared object loading. For instance, the module syst #### `bindnow` {#bindnow} -Adds the `-z bindnow` linker option. During program load, all dynamic symbols are resolved, allowing for the complete GOT to be marked read-only (due to `relro`). This prevents GOT overwrite attacks. For very large applications, this can incur some performance loss during initial load while symbols are resolved, but this shouldn’t be an issue for daemons. +Adds the `-z now` linker option. During program load, all dynamic symbols are resolved, allowing for the complete GOT to be marked read-only (due to `relro`). This prevents GOT overwrite attacks. For very large applications, this can incur some performance loss during initial load while symbols are resolved, but this shouldn’t be an issue for daemons. This flag can break dynamic shared object loading. For instance, the module systems of Xorg and PHP are incompatible with this flag. Programs incompatible with this flag often fail at runtime due to missing symbols, like: |