| Commit message (Collapse) | Author | Age |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Rename files in `bootstrap-files/` to match jobs in `make-bootstrap-tools-cross`.
Should make automating the bootstrap files easier.
```nix
$ nix repl --file ./pkgs/top-level/release-cross.nix
nix-repl> bootstrapTools
{ aarch64-unknown-linux-gnu = { ... }; aarch64-unknown-linux-musl = { ... }; armv5tel-unknown-linux-gnueabi = { ... }; armv6l-unknown-linux-gnueabihf = { ... }; armv6l-unknown-linux-musleabihf = { ... }; armv7l-unknown-linux-gnueabihf = { ... }; mips64el-unknown-linux-gnuabi64 = {... }; mips64el-unknown-linux-gnuabin32 = { ... }; mipsel-unknown-linux-gnu = { ... }; powerpc64le-unknown-linux-gnu = { ... }; riscv64-unknown-linux-gnu = { ... }; x86_64-unknown-linux-musl = { ... }; }
```
Additionally if non-linux bootstrap files are added to
`make-bootstrap-tools-cross` then there won't be any renaming needed.
|
|\ |
|
| | |
|
| |
| |
| |
| | |
The last use of it was removed in (linux/default.nix: use mipsel.nix instead of longson.nix for mips32)[e8b10284f32b32469d5f54433e5c5a75448b327b].
|
| | |
|
| |
| |
| |
| | |
in stdenv/linux/default.nix
|
|/
|
|
| |
This reverts commit 3f46cdcb5b903fe1c07707fea9ee24ae09d964b2.
|
|
|
|
|
|
|
| |
This reverts commit ab66640da31a80c4cfd38e8f495c23ef2f7e254b, reversing
changes made to c08b005e0f6ce26d0269e9ac7e12747a29f822fb.
The changes were merged without the associated bootstrap updates.
|
|
|
|
|
|
|
|
| |
Include all runtime object files in output package, enabling different
kinds of build modes - non-PIE, PIE, static PIE and profile-generated.
Suggested by @trofi:
https://github.com/NixOS/nixpkgs/pull/252310#issuecomment-1709425791
|
| |
|
|
|
|
| |
Accidentally omitted from #237968.
|
|
|
|
|
|
|
|
| |
Unlike autoreconfHook, updateAutotoolsGnuConfigScriptsHook adds
almost no compilations. Therefore, in the interest of building the
same source code on every platform wherever possible, let's
eliminate the conditional guards around
updateAutotoolsGnuConfigScriptsHook in stdenv.
|
|\
| |
| | |
[staging] gnugrep: 3.7 -> 3.11
|
| | |
|
|\ \
| |/
|/| |
stdenv: don't set NIX_LIB*_IN_SELF_RPATH by default
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The NIX_LIB64|32_IN_SELF_RPATH environment variables control whether
to add lib64 and lib32 to rpaths. However, they're set depending
on the build paltform, not the target platform and thus their values
are incorrect for for cross-builds.
On the other hand, setting them according to the build platform introduce
pointless differences in build outputs; see #221350 for details.
This change fixes the issues by boldly removes the NIX_LIB*_IN_SELF_RPATH
facility altogether, in the hope that it is no longer necessary. They
were introduced in 2009, long before nixpkgs had good support for
cross-builds.
Fixes #221350
|
| | |
|
|\ \
| | |
| | | |
pkgsCross.loongarch64-linux.freshBootstrapTools.bootstrapFiles: fix build
|
| | | |
|
|\ \ \
| | | |
| | | | |
stdenv: always update config script on loongarch64-linux
|
| |/ / |
|
|/ /
| |
| |
| |
| |
| | |
musl now supports RISC-V. Let's centralise musl availability checks
in musl.meta.platforms, so we don't have to keep cleaning up ad-hoc
checks like this all over the tree.
|
|\ \
| | |
| | | |
stdenvBootstrapTools: drop no-longer-needed, now-broken cross clause
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The comment stating that "GCC has certain things built in
statically" is no longer true as of
https://github.com/NixOS/nixpkgs/pull/209870. Moreover, as pointed
out in https://github.com/NixOS/nixpkgs/pull/228130 this
optionalString block no longer builds for
pkgsCross.*.freshBootstrapTools.
It is safe to drop this clause now.
|
|\| | |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
I broke `pkgsMusl` with #209870.
Something odd is happening with `xgcc` (the temporary compiler that
should be used only to compile `gcc`, although we are using it to
compile a temporary `patchelf` too) and `libstdc++`.
The temporary fix in this commit is to use `-static-libstdc++` for
the ephemeral `patchelf` built by `xgcc`. It isn't pretty, but it
appears to work.
Incorporates:
- https://github.com/NixOS/nixpkgs/pull/224945
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The stage before `xgcc` creates the first compiled patchelf
(i.e. not from bootstrapFiles).
The `xgcc` stage was inadvertently switching *back* to using the
patchelf *from* the bootstrapFiles.
The first commit in this PR adds self-checking comments (assertions)
to make it clear where each stage's patchelf comes from.
The second commit fixes the bug, and updates the self-checking
comments.
|
|/ / |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Without the change when I attempt to built `nixpkgs` with weekly
`gcc-13` (it pulls in `flex` as a build input`) I am getting build
failure related to glibc mix caused by glibc loading:
...-binutils-patchelfed-ld-2.40/bin/ld: ...-xgcc-13.0.0/libexec/gcc/x86_64-unknown-linux-gnu/13.0.1/liblto_plugin.so:
error loading plugin: ...-bootstrap-tools/lib/libpthread.so.0: undefined symbol: __libc_vfork, version GLIBC_PRIVATE
The change disables LTO plugin entirely to avoid loading of `glibc` mix.
|
|\ \
| | |
| | |
| | | |
amjoseph-nixpkgs/pr/stdenv/external-gcc-bootstrap
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This commit adds `gcc/common/checksum.nix`, which contains code
common to both gcc11 and gcc12, implementing the `enableChecksum`
feature.
When gcc's built-in bootstrap (`--enable-bootstrap`) is used, gcc
compiles itself three times and compares a hash of the unlinked `.o`
files from the second and third compilation. The
`enableChecksum=true` parameter performs the same comparison as part
of the `postInstall` phase.
Notably, `enableChecksum=true` can be used with `enableBootstrap=false`.
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
|
| | | |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Our bootstrap-files unpacker has always relied on a lot of unstated
assumptions, one of them being that every library has a DT_NEEDED
for librt.so, so patchelf'ing something into the RUNPATH into
librt.so means that it will be searched for every library load in
all of the bootstrap-files.
Unfortunately that assumption is not true for libgcc.
This causes problems, because patchelf links against libgcc (and
against libstdc++, which links against libgcc). So we can't use
patchelf on libgcc, because it needs libgcc, so patchelf doesn't
work until libgcc is patchelfed.
The robust solution here is to use static linking for the copy of
patchelf that is shipped with the bootstrap-files. We don't have to
go all the way to a statically linked libc; just -static-libgcc and
-static-libstdc++ are enough to break the circular dependency.
|
| | |
| | |
| | |
| | |
| | |
| | | |
We do not want to preserve the symlinks from libgcc_s, since they
point to another outpath. We want to copy from that outpath rather
than link to it.
|
| | |
| | |
| | |
| | |
| | |
| | | |
Now that we've dropped the
gcc-links-statically-to-lib{isl,mpfr,mpc,gmp} hack, our gcc needs
libisl.so. Let's add it to the bootstrap-files.
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
|
| |/
| |
| |
| |
| | |
This commit has no effect on eval. It simply factors out a common
subexpression.
|
|/
|
|
|
|
|
|
|
|
|
|
|
| |
Right now our bootstrapFiles-selecting algorithm uses the
`loongson2f.nix` bootstrapFiles (which were not built by Hydra).
These bootstrapFiles don't work anymore. They were added in 2010 by
40405d03ac383c8c6e45dec59eaad307d03e4c42.
This commit causes mipsel-linux native builds to use the Hydra-built
bootstrap files from this PR instead:
https://github.com/NixOS/nixpkgs/pull/183487
|
|\ |
|
| |\ |
|
| | | |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This PR provides the Hydra-generated bootstrap tarballs for mips64el-linux-gnuabin32.
With this PR we now have the bootstrap-file for all three little-endian ABIs on mips: n64, n32, and o32. I do not currently plan to do big-endian mips unless some motivation arises; all mips chips are bi-endian and Debian has dropped big-endian support due to lack of interest.
I'll be following the script used in #151399, #168199, and #183487.
Files came from [this](https://hydra.nixos.org/build/188389586#tabs-summary) Hydra build, which used nixpkgs revision 97d9c84e1df4397b43ecb39359f1bd003cd44585 to instantiate:
```
/nix/store/hakn8s85s9011v61r6svp5qy8x1y64fv-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabin32.drv
```
and then built:
```
/nix/store/rjgybpnf3yiqyhvl2n2lx31jf800fii2-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabin32
```
I downloaded these files from Hydra and prefetched them into the nix store with the following commands:
```
STOREPATH=rjgybpnf3yiqyhvl2n2lx31jf800fii2-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabin32
OPTIONS="--option binary-caches https://cache.nixos.org --option trusted-public-keys cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
nix store prefetch-file \
file://$(nix store add-file --name bootstrap-tools.tar.xz $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz)
nix store prefetch-file --executable \
file://$(nix store add-path --name busybox $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox)
```
These commands produced the following output:
```
Downloaded 'file:///nix/store/w6zzd2fx2vhmjfcf5h5zc01m0swldpbw-bootstrap-tools.tar.xz' to '/nix/store/6w0f0mqblrghvh6yjwcb4xdqq9x50lbl-w6zzd2fx2vhmjfcf5h5zc01m0swldpbw-bootstrap-tools.tar.xz' (hash 'sha256-LWrpN6su2yNVurUyhZP34OiZyzgh7MfN13fIIbou8KI=').
Downloaded 'file:///nix/store/nqagw1kgdz1zlmqi00qfjrmwqk3g3bgd-busybox' to '/nix/store/i361xhbdhhnvg7zd637xpm63vbl80s0s-nqagw1kgdz1zlmqi00qfjrmwqk3g3bgd-busybox' (hash 'sha256-4N3G1qYA7vitjhsIW17pR6UixIuzrq4vZXa8F0/X4iI=').
```
I used the hashes from the output above to create the `fetchurl` invocation which is part of this commit.
I then started the bootstrap with the following command:
```
nix build -L -f . --arg localSystem '(import ./lib).systems.examples.mips64el-linux-gnuabin32' hello
```
As @lovesegfault requested, here are the the `sha256sum`s of all the `on-server` components for extra verification:
```
sha256sum /nix/store/${STOREPATH}/on-server/*
```
which produced the following output:
```
2d6ae937ab2edb2355bab5328593f7e0e899cb3821ecc7cdd777c821ba2ef0a2 /nix/store/rjgybpnf3yiqyhvl2n2lx31jf800fii2-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabin32/on-server/bootstrap-tools.tar.xz
01633f71135cb9ab1b5ce3ebb67e80cbf288739729bffc1350c1552f6f8df34b /nix/store/rjgybpnf3yiqyhvl2n2lx31jf800fii2-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabin32/on-server/busybox
```
|
| | | | |
| \ \ | |
|\ \ \ \
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
...into staging
Also merge the commit that was referenced as the base for the build
of the new bootstrap tools (although others would give the same).
|
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Hydra job building them: https://hydra.nixos.org/build/208909151
The bootstrap files can be reproduced on the parent commit, e.g. by:
cat $(nix-build pkgs/top-level/release.nix -QA stdenvBootstrapTools.aarch64-linux.dist)/nix-support/hydra-build-products
file tarball /nix/store/kdpbw0plmjqlafjnpbz31ja51m4bd2dk-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz
file busybox /nix/store/kdpbw0plmjqlafjnpbz31ja51m4bd2dk-stdenv-bootstrap-tools/on-server/busybox
and the hashes as well:
nix hash file /nix/store/kdpbw0plmjqlafjnpbz31ja51m4bd2dk-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz
sha256-aJvtsWeuQHbb14BGZ2EiOKzjQn46h3x3duuPEawG0eE=
nix hash path /nix/store/kdpbw0plmjqlafjnpbz31ja51m4bd2dk-stdenv-bootstrap-tools/on-server/busybox
sha256-0MuIeQlBUaeisqoFSu8y+8oB6K4ZG5Lhq8RcS9JqkFQ=
You can check this on any machine, as the builds are on cache.nixos.org
but also you can reproduce the hashes when rebuilt on aarch64-linux HW.
|
| |/ / /
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Hydra job building them: https://hydra.nixos.org/build/208909151
The bootstrap files can be reproduced on the commit 21ec906463ea8f11abf3f9091ddd4c3276516e58, e.g. by:
cat $(nix-build pkgs/top-level/release.nix -QA stdenvBootstrapTools.aarch64-linux.dist)/nix-support/hydra-build-products
file tarball /nix/store/kdpbw0plmjqlafjnpbz31ja51m4bd2dk-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz
file busybox /nix/store/kdpbw0plmjqlafjnpbz31ja51m4bd2dk-stdenv-bootstrap-tools/on-server/busybox
and the hashes as well:
nix hash file /nix/store/kdpbw0plmjqlafjnpbz31ja51m4bd2dk-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz
sha256-aJvtsWeuQHbb14BGZ2EiOKzjQn46h3x3duuPEawG0eE=
nix hash path /nix/store/kdpbw0plmjqlafjnpbz31ja51m4bd2dk-stdenv-bootstrap-tools/on-server/busybox
sha256-0MuIeQlBUaeisqoFSu8y+8oB6K4ZG5Lhq8RcS9JqkFQ=
You can check this on any machine, as the builds are on cache.nixos.org
but also you can reproduce the hashes when rebuilt on aarch64-linux HW.
|
|\ \ \ \
| | | | |
| | | | | |
express #208478 as assertions
|
| |/ / /
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
PR #208478 added a lot of documentation about which packages were
rebuilt in each stage of the stdenv bootstrap. However nothing
checks that these comments agree with reality; they can bitrot over
time. This PR rewrites those comments as assertions, so they cannot
bitrot.
This conversion did expose some ambiguity in our scheme for naming
the stages. Suppose that `pkgs.stdenv.name=="stdenv-stage4", then
which of these is "the stage4 coreutils"?
```
pkgs.coreutils
pkgs.stdenv.__bootPackages.coreutils
```
The choice is arbitrary, and both choices have confusing corner
cases. We should revisit this at some point.
|
|\ \ \ \
| |_|/ /
|/| | | |
hardening flags: add `FORTIFY_SOURCE=3` support
|
| |/ / |
|