summary refs log tree commit diff
path: root/pkgs/build-support/docker
Commit message (Collapse)AuthorAge
...
* dockerTools.buildImageWithNixDb: simplifications and switch to closureInfoAntoine Eiche2018-11-12
| | | | | | Since Nix 2 is now the stable Nix version, we can use closureInfo which simplifies the Nix database initialisation (size and hash are included in the "dump").
* dockerTools: Use nix instead of nixUnstableSarah Brofeldt2018-10-01
|
* dockerTools: test buildLayeredImageGraham Christensen2018-09-27
|
* dockerTools.buildLayeredImage: initGraham Christensen2018-09-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Create a many-layered Docker Image. Implements much less than buildImage: - Doesn't support specific uids/gids - Doesn't support runninng commands after building - Doesn't require qemu - Doesn't create mutable copies of the files in the path - Doesn't support parent images If you want those feature, I recommend using buildLayeredImage as an input to buildImage. Notably, it does support: - Caching low level, common paths based on a graph traversial algorithm, see referencesByPopularity in 0a80233487993256e811f566b1c80a40394c03d6 - Configurable number of layers. If you're not using AUFS or not extending the image, you can specify a larger number of layers at build time: pkgs.dockerTools.buildLayeredImage { name = "hello"; maxLayers = 128; config.Cmd = [ "${pkgs.gitFull}/bin/git" ]; }; - Parallelized creation of the layers, improving build speed. - The contents of the image includes the closure of the configuration, so you don't have to specify paths in contents and config. With buildImage, paths referred to by the config were not included automatically in the image. Thus, if you wanted to call Git, you had to specify it twice: pkgs.dockerTools.buildImage { name = "hello"; contents = [ pkgs.gitFull ]; config.Cmd = [ "${pkgs.gitFull}/bin/git" ]; }; buildLayeredImage on the other hand includes the runtime closure of the config when calculating the contents of the image: pkgs.dockerTools.buildImage { name = "hello"; config.Cmd = [ "${pkgs.gitFull}/bin/git" ]; }; Minor Problems - If any of the store paths change, every layer will be rebuilt in the nix-build. However, beacuse the layers are bit-for-bit reproducable, when these images are loaded in to Docker they will match existing layers and not be imported or uploaded twice. Common Questions - Aren't Docker layers ordered? No. People who have used a Dockerfile before assume Docker's Layers are inherently ordered. However, this is not true -- Docker layers are content-addressable and are not explicitly layered until they are composed in to an Image. - What happens if I have more than maxLayers of store paths? The first (maxLayers-2) most "popular" paths will have their own individual layers, then layer #(maxLayers-1) will contain all the remaining "unpopular" paths, and finally layer #(maxLayers) will contain the Image configuration.
* dockertools: tarsum: turn in to a buildInputGraham Christensen2018-09-26
|
* dockerTools.buildImage: test that created=now makes an unstable dateGraham Christensen2018-09-20
|
* dockerTools.buildImage: support impure datesGraham Christensen2018-09-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Because dates are an impurity, by default buildImage will use a static date of one second past the UNIX Epoch. This can be a bit frustrating when listing docker images in the CLI: $ docker image list REPOSITORY TAG IMAGE ID CREATED SIZE hello latest 08c791c7846e 48 years ago 25.2MB If you want to trade the purity for a better user experience, you can set created to now. pkgs.dockerTools.buildImage { name = "hello"; tag = "latest"; created = "now"; contents = pkgs.hello; config.Cmd = [ "/bin/hello" ]; } and now the Docker CLI will display a reasonable date and sort the images as expected: $ docker image list REPOSITORY TAG IMAGE ID CREATED SIZE hello latest de2bf4786de6 About a minute ago 25.2MB
* dockerTools.pullImage: correct default archJack Kelly2018-09-19
|
* dockerTools.pullImage: control OS and architectureNick Novitski2018-07-27
|
* pkgs/*: remove unreferenced function argumentsvolth2018-07-21
|
* dockerTools.examples: explicitly set image tag to fix docker-tools testsAntoine Eiche2018-07-06
| | | | | | docker-tools tests load images without specifying any tag value. Docker then uses the image with tag "latest" which doesn't exist anymore since commit 39e678e24e38f1f374eaf5463b424ebdf75df9af.
* dockerTools.buildImage: add option to use nix output hash as tagMathias Schreck2018-07-06
|
* dockerTools.pullImage: expose image* attributes (#41366)lewo2018-06-03
| | | | | Attributes `imageName` and `imageTag` are exposed if the image is built by our Nix tools but not if the image is pulled. So, we expose these attributes for convenience and homogeneity.
* Merge pull request #40947 from samueldr/fix/34779lewo2018-05-24
|\ | | | | dockerTools: fixes extraCommands for mkRootLayer.
| * tests/docker-tools: Adds regression test for #34779Samuel Dionne-Riel2018-05-24
| |
| * dockerTools: fixes extraCommands for mkRootLayer.Samuel Dionne-Riel2018-05-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The extraCommands was, previously, simply put in the body of the script using nix expansion `${extraCommands}` (which looks exactly like bash expansion!). This causes issues like in #34779 where scripts will eventually create invalid bash. The solution is to use a script like `run-as-root`. * * * Fixes #34779
* | skopeo: 0.1.29 -> 0.1.30Antoine Eiche2018-05-24
|/ | | | | | | Skopeo used by our docker tools was patched to work in the build sandbox (it used /var/tmp which is not available in the sandbox). Since this temporary directory can now be set at build time, we remove the patch from our docker tools.
* dockerTools.pullImage: Fix build with sandboxingaszlig2018-05-06
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Regression introduced in 736848723e5aefa5d24396c58dc6de603399efde. This commit most certainly hasn't been tested with sandboxing enabled and breaks not only pullImage but also the docker-tools NixOS VM test because it doesn't find it's certificate path and also relies on /var/tmp being there. Fixing the certificate path is the easiest one because it can be done via environment variable. I've used overrideAttrs for changing the hardcoded path to /tmp (which is available in sandboxed builds and even hardcoded in Nix), so that whenever someone uses Skopeo from all-packages.nix the path is still /var/tmp. The reason why this is hardcoded to /var/tmp can be seen in a comment in vendor/github.com/containers/image/storage/storage_image.go: Do not use the system default of os.TempDir(), usually /tmp, because with systemd it could be a tmpfs. With sandboxed builds this isn't the case, however for using Nix without NixOS this could turn into a problem if this indeed is the case. So in the long term this needs to have a proper solution. In addition to that, I cleaned up the expression a bit. Tested by building dockerTools.examples.nixFromDockerHub and the docker-tools NixOS VM test. Signed-off-by: aszlig <aszlig@nix.build> Cc: @nlewo, @Mic92, @Profpatsch, @globin, @LnL7
* dockerTools.pullImage: Skopeo pulls images by digestAntoine Eiche2018-05-02
| | | | | | | | | | | Skopeo is used to pull images from a Docker registry (instead of a Docker deamon in a VM). An image reference is specified with its name and its digest which is an immutable image identifier (unlike image name and tag). Skopeo can be used to get the digest of an image, for instance: $ skopeo inspect docker://docker.io/nixos/nix:1.11 | jq -r '.Digest'
* dockerTools.buildImage: add /nix/store with correct permissionsJean-Philippe Braun2018-04-16
| | | | Fixes #38835.
* Merge pull request #25148 from obsidiansystems/docker-dirlinksRyan Trinkle2018-04-09
|\ | | | | dockerTools: optionally preserve directory symlinks
| * Merge branch 'master' into docker-dirlinksRyan Trinkle2017-11-03
| |\
| * | dockerTools: optionally preserve directory symlinksRyan Trinkle2017-04-23
| | | | | | | | | | | | In some cases, this seems to save a lot (>40%) of space.
* | | dockerTools.tarsum: Fix upstream importSarah Brofeldt2018-03-26
| | |
* | | dockerTools: add --sort=name options on all tar callsAntoine Eiche2018-03-13
| | | | | | | | | | | | | | | | | | This is to go to a reproducible image build. Note without this options image are identical from the Docker point of view but generated docker archives could have different hashes.
* | | dockerTools: dereference hard links in tar archivesAntoine Eiche2018-03-13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is to improve image creation reproducibility. Since the nar format doesn't support hard link, the tar stream of a layer can be different if a dependency of a layer has been built locally or if it has been fetched from a binary cache. If the dependency has been build locally, it can contain hard links which are encoded in the tar stream. If the dependency has been fetched from a binary cache, the tar stream doesn't contain any hard link. So even if the content is the same, tar streams are different.
* | | dockerTools: add an onTopOfPulledImage exampleAntoine Eiche2018-03-13
| | | | | | | | | | | | | | | This allows to test if a pulled image can be updated by using our Docker tools.
* | | dockerTools.buildImage: do not add /nix/store in the tar streamAntoine Eiche2018-02-14
| | | | | | | | | | | | | | | | | | | | | Since the /nix/store directory is not immutable, tar can fails if it has to push it into the layer archive. Fixes #34137.
* | | treewide: Fix deps in a few other fixed output derivationsJohn Ericson2018-01-10
| | |
* | | dockerTools.examples: correct a typo in commentsWei-Ming Yang2018-01-01
| |/ |/| | | This commit is for correcting a typo in comments.
* | dockerTools: fix hash to accomodate the pullImage revertRobin Gloster2017-09-28
| |
* | Revert "dockerTools.pullImage: use skopeo to pull the image"Robin Gloster2017-09-28
| | | | | | | | | | | | | | | | This reverts commit 01174c5f4d7df0fd0928fbf8a2a8e633a9cf54aa. See https://github.com/NixOS/nixpkgs/pull/29302#issuecomment-332809092 for more information. This broke image format compatibility and therefore amongst others mesos.
* | Revert "dockerTools.buildImage: Switch to the format image generated by Skopeo"Robin Gloster2017-09-28
| | | | | | | | | | | | This reverts commit 35f205a4b624bceca7c53b9c19ddac5f37a5ae4c. This does not use a standard format and by that breaks mesos
* | dockerTools.examples.nix: set NIX_PAGER=cat environment variableAntoine Eiche2017-09-25
| |
* | dockerTools.buildImage: Switch to the format image generated by SkopeoAntoine Eiche2017-09-23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We were using 'Combined Image JSON + Filesystem Changeset Format' [1] to unpack and pack image and this patch switches to the format used by the registry. We used the 'repository' file which is not generated by Skopeo when it pulls an image. Moreover, all information of this file are also in the manifest.json file. We then use the manifest.json file instead of 'repository' file. Note also the manifest.json file is required to push an image with Skopeo. Fix #29636 [1] https://github.com/moby/moby/blob/749d90e10f989802638ae542daf54257f3bf71f2/image/spec/v1.1.md#combined-image-json--filesystem-changeset-format
* | dockerTools.buildImageWithNixDb: Make output paths valid and add gcrootsAntoine Eiche2017-09-20
| | | | | | | | | | | | | | | | | | | | | | The database dump doesn't contain sha and size. This leads to invalid path in the container. We have to fix the database by using nix-store. Note a better way to do this is available in Nix 1.12 (since the database dump contains all required information). We also add content output paths in the gcroots since they ca be used by the container.
* | dockerTools.buildImageWithNixDb: populate the Nix Db of the image Nix storeAntoine Eiche2017-09-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, the contents closure is copied to the layer but there is no nix database initialization. If pkgs.nix is added in the contents, nix-store doesn't work because there is no nix database. From the contents of the layer, this commit generates and loads the database in the nix store of the container. This only works if there is no parent layer that already have a nix store (to support several nix layers, we would have to merge nix databases of parent layers). We also add an example to play with the nix store inside the container. Note it seems `more` is a missing dependency of the nix package!
* | dockerTools.pullImage: use skopeo to pull the imageAntoine Eiche2017-09-17
| | | | | | | | | | | | | | | | | | | | | | | | | | Before this patch, a VM was used to spawn docker that pulled the VM. Now, the tool Skopeo does this job well so we can simplify our dockerTools since we doesn't need Docker anymore:) This also fixe the regression described in https://github.com/NixOS/nixpkgs/issues/29271 : cntlm proxy doesn't work in 17.09 while it worked in 17.03. Note Skopeo doesn't produce the same output than docker pull so, we have to update sha.
* | dockerTools.pullImage: change the docker deamon readiness mechanismAntoine Eiche2017-09-04
| | | | | | | | | | | | To wait for the docker deamon, curl requests are sent. However, if a http proxy is set, it will respond instead of the docker daemon. To avoid this, we send docker ps command instead of curl command.
* | dockerTools: fix image json and manifestMathias Schreck2017-08-03
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The image json is not exactly the same as the layer json, therefore I changed the implementation to use the `baseJson` which doesn’t include layer specific details like `id`, `size` or the checksum of the layer. Also the `history` entry was missing in the image json. I’m not totally sure if this field is required, but a I got an error from a docker registry when I’ve tried to receive the distribution manifest of an image without those `history` entry: GET: `http://<registry-host>/v2/<imageName>/manifests/<imageTag>` ```json { "errors": [ { "code": "MANIFEST_INVALID", "message": "manifest invalid", "detail": {} } ] } ``` I’ve also used a while loop to iterate over all layers which should make sure that the order of the layers is correct. Previously `find` was used and I’m not sure if the order was always correct.
* | dockerTools: fix permissions on base imageAntoine Eiche2017-07-31
| | | | | | | | | | | | | | | | If the base image has been built with nixpkgs.dockerTools, the image configuration and manifest are readonly so we first need to change their permissions before removing them. Fix #27632.
* | docker: generate the image configuration and manifestAntoine Eiche2017-07-26
| | | | | | | | This is required to push images to the Docker registry v2.
* | docker: Remove ./ pattern when packing an imageAntoine Eiche2017-07-26
| | | | | | | | Elements in images tar.gz generated by docker don't start by './'.
* | docker: do not import configuration and manifest from the base imageAntoine Eiche2017-07-26
| | | | | | | | Fix #27632.
* | docker: lowercase image name and tagAntoine Eiche2017-07-25
| | | | | | | | | | | | The docker loading (docker 1.12.6) of an image with uppercase in the name fails with the following message: invalid reference format: repository name must be lowercase
* | Merge pull request #27017 from LnL7/docker-pure-layerDaiderd Jordan2017-07-15
|\ \ | | | | | | docker-tools: set user/group when creating a pure layer
| * | docker-tools: fixup permssions for extraCommandsDaiderd Jordan2017-07-08
| | |
| * | docker-tools: set group/owner when creating a pure layerDaiderd Jordan2017-07-01
| | |
* | | tarsum: Fix source path in docker.srcSarah Brofeldt2017-07-13
|/ /
* | using inherited insteadMichael Fellinger2017-06-27
| |
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-05-10 10:44:42 +0000