nixos/release-combined.nix: Build pkgs/by-name tester

1 files changed, 16 insertions, 1 deletions

diff --git a/pkgs/test/nixpkgs-check-by-name/README.md b/pkgs/test/nixpkgs-check-by-name/README.md
index 16536154a26..754d0a2090d 100644
--- a/pkgs/test/nixpkgs-check-by-name/README.md
+++ b/pkgs/test/nixpkgs-check-by-name/README.md
@@ -5,7 +5,7 @@ This is part of the implementation of [RFC 140](https://github.com/NixOS/rfcs/pu
 
 ## API
 
-This API may be changed over time if the CI making use of it is adjusted to deal with the change appropriately.
+This API may be changed over time if the CI making use of it is adjusted to deal with the change appropriately, see [Hydra builds](#hydra-builds).
 
 - Command line: `nixpkgs-check-by-name <NIXPKGS>`
 - Arguments:
@@ -80,3 +80,18 @@ Tests are declared in [`./tests`](./tests) as subdirectories imitating Nixpkgs w
 - `expected` (optional):
   A file containing the expected standard output.
   The default is expecting an empty standard output.
+
+## Hydra builds
+
+This program will always be available pre-built for `x86_64-linux` on the `nixos-unstable` channel and `nixos-XX.YY` channels.
+This is ensured by including it in the `tested` jobset description in [`nixos/release-combined.nix`](../../../nixos/release-combined.nix).
+
+This allows CI for PRs to development branches `master` and `release-XX.YY` to fetch the pre-built program from the corresponding channel and use that to check the PR. This has the following benefits:
+- It allows CI to check all PRs, even if they would break the CI tooling.
+- It makes the CI check very fast, since no Nix builds need to be done, even for mass rebuilds.
+- It improves security, since we don't have to build potentially untrusted code from PRs.
+  The tool only needs a very minimal Nix evaluation at runtime, which can work with [readonly-mode](https://nixos.org/manual/nix/stable/command-ref/opt-common.html#opt-readonly-mode) and [restrict-eval](https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-restrict-eval).
+- It allows anybody to make updates to the tooling and for those updates to be automatically used by CI without needing a separate release mechanism.
+
+The tradeoff is that there's a delay between updates to the tool and those updates being used by CI.
+This needs to be considered when updating the [API](#api).