path: root/doc/erlang-users-guide.xml

<chapter xmlns="http://docbook.org/ns/docbook"
         xmlns:xlink="http://www.w3.org/1999/xlink"
         xml:id="users-guide-to-the-erlang-infrastructure">

<title>User's Guide to the Erlang Infrastructure</title>
<section xml:id="build-tools">
  <title>Build Tools</title>
  <para>
    By default Rebar3 wants to manage it's own dependencies. In the
    normal non-Nix, this is perfectly acceptable. In the Nix world it
    is not. To support this we have created two versions of rebar3,
    <literal>rebar3</literal> and <literal>rebar3-open</literal>. The
    <literal>rebar3</literal> version has been patched to remove the
    ability to download anything from it. If you are not running it a
    nix-shell or a nix-build then its probably not going to work for
    you. <literal>rebar3-open</literal> is the normal, un-modified
    rebar3. It should work exactly as would any other version of
    rebar3. Any Erlang package should rely on
    <literal>rebar3</literal> and thats really what you should be
    using too.
  </para>
</section>

<section xml:id="how-to-install-erlang-packages">
  <title>How to install Erlang packages</title>
  <para>
    Erlang packages are not registered in the top level simply because
    they are not relevant to the vast majority of Nix users. They are
    installable using the <literal>erlangPackages</literal> attribute set.

    You can list the avialable packages in the
    <literal>erlangPackages</literal> with the following command:
  </para>

  <programlisting>
$ nix-env -f &quot;&lt;nixpkgs&gt;&quot; -qaP -A erlangPackages
erlangPackages.esqlite    esqlite-0.2.1
erlangPackages.goldrush   goldrush-0.1.7
erlangPackages.ibrowse    ibrowse-4.2.2
erlangPackages.jiffy      jiffy-0.14.5
erlangPackages.lager      lager-3.0.2
erlangPackages.meck       meck-0.8.3
erlangPackages.rebar3-pc  pc-1.1.0
  </programlisting>
  <para>
    To install any of those packages into your profile, refer to them by
    their attribute path (first column):
  </para>
  <programlisting>
$ nix-env -f &quot;&lt;nixpkgs&gt;&quot; -iA erlangPackages.ibrowse
  </programlisting>
  <para>
    The attribute path of any Erlang packages corresponds to the name
    of that particular package in Hex or its OTP Application/Release name.
  </para>
</section>
<section xml:id="packaging-erlang-applications">
  <title>Packaging Erlang Applications</title>
  <section xml:id="rebar3-packages">
    <title>Rebar3 Packages</title>
    <para>
      There is a Nix functional called
      <literal>buildRebar3</literal>. We use this function to make a
      derivation that understands how to build the rebar3 project. For
      example, the epression we use to build the <link
      xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link>
      project follows.
    </para>
    <programlisting>
{stdenv, fetchFromGitHub, buildRebar3, ibrowse, jsx, erlware_commons }:

buildRebar3 rec {
    name = "hex2nix";
    version = "0.0.1";

    src = fetchFromGitHub {
        owner = "ericbmerritt";
        repo = "hex2nix";
        rev = "${version}";
        sha256 = "1w7xjidz1l5yjmhlplfx7kphmnpvqm67w99hd2m7kdixwdxq0zqg";
    };

    erlangDeps = [ ibrowse jsx erlware_commons ];
}
    </programlisting>
    <para>
      The only visible difference between this derivation and
      something like <literal>stdenv.mkDerivation</literal> is that we
      have added <literal>erlangDeps</literal> to the derivation. If
      you add your Erlang dependencies here they will be correctly
      handled by the system.
    </para>
    <para>
      If your package needs to compile native code via Rebar's port
      compilation mechenism. You should add <literal>compilePort =
      true;</literal> to the derivation.
    </para>
  </section>

  <section xml:id="hex-packages">
    <title>Hex Packages</title>
    <para>
      Hex packages are based on Rebar packages. In fact, at the moment
      we can only compile Hex packages that are buildable with
      Rebar3. Packages that use Mix and other build systems are not
      supported. That being said, we know a lot more about Hex and can
      do more for you.
    </para>
    <programlisting>
{ buildHex }:
  buildHex {
    name = "esqlite";
    version = "0.2.1";
    sha256 = "1296fn1lz4lz4zqzn4dwc3flgkh0i6n4sydg501faabfbv8d3wkr";
    compilePort = true;
}
    </programlisting>
    <para>
      For Hex packages you need to provide the name, the version, and
      the Sha 256 digest of the package and use
      <literal>buildHex</literal> to build it. Obviously, the package
      needs to have already been published to Hex.
    </para>
  </section>
</section>
<section xml:id="how-to-develop">
  <title>How to develop</title>
  <section xml:id="accessing-an-environment">
    <title>Accessing an Environment</title>
    <para>
      Often, all you want to do is be able to access a valid
      environment that contains a specific package and its
      dependencies. we can do that with the <literal>env</literal>
      part of a derivation. For example, lets say we want to access an
      erlang repl with ibrowse loaded up. We could do the following.
    </para>
    <programlisting>
      ~/w/nixpkgs ❯❯❯ nix-shell -A erlangPackages.ibrowse.env --run "erl"
      Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

      Eshell V7.0  (abort with ^G)
      1> m(ibrowse).
      Module: ibrowse
      MD5: 3b3e0137d0cbb28070146978a3392945
      Compiled: January 10 2016, 23:34
      Object file: /nix/store/g1rlf65rdgjs4abbyj4grp37ry7ywivj-ibrowse-4.2.2/lib/erlang/lib/ibrowse-4.2.2/ebin/ibrowse.beam
      Compiler options:  [{outdir,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/ebin"},
      debug_info,debug_info,nowarn_shadow_vars,
      warn_unused_import,warn_unused_vars,warnings_as_errors,
      {i,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/include"}]
      Exports:
      add_config/1                  send_req_direct/7
      all_trace_off/0               set_dest/3
      code_change/3                 set_max_attempts/3
      get_config_value/1            set_max_pipeline_size/3
      get_config_value/2            set_max_sessions/3
      get_metrics/0                 show_dest_status/0
      get_metrics/2                 show_dest_status/1
      handle_call/3                 show_dest_status/2
      handle_cast/2                 spawn_link_worker_process/1
      handle_info/2                 spawn_link_worker_process/2
      init/1                        spawn_worker_process/1
      module_info/0                 spawn_worker_process/2
      module_info/1                 start/0
      rescan_config/0               start_link/0
      rescan_config/1               stop/0
      send_req/3                    stop_worker_process/1
      send_req/4                    stream_close/1
      send_req/5                    stream_next/1
      send_req/6                    terminate/2
      send_req_direct/4             trace_off/0
      send_req_direct/5             trace_off/2
      send_req_direct/6             trace_on/0
      trace_on/2
      ok
      2>
    </programlisting>
    <para>
      Notice the <literal>-A erlangPackages.ibrowse.env</literal>.That
      is the key to this functionality.
    </para>
  </section>
  <section xml:id="creating-a-shell">
    <title>Creating a Shell</title>
    <para>
      Getting access to an environment often isn't enough to do real
      development. Many times we need to create a
      <literal>shell.nix</literal> file and do our development inside
      of the environment specified by that file. This file looks a lot
      like the packageing described above. The main difference is that
      <literal>src</literal> points to project root and we call the
      package directly.
    </para>
    <programlisting>
{ pkgs ? import &quot;&lt;nixpkgs&quot;&gt; {} }:

with pkgs;

let

  f = { buildHex, ibrowse, jsx, erlware_commons }:
      buildHex {
        name = "hex2nix";
        version = "0.1.0";
        src = ./.;
        erlangDeps = [ ibrowse jsx erlware_commons ];
      };
  drv = erlangPackages.callPackage f {};

in
 drv
    </programlisting>
    <section xml:id="building-in-a-shell">
    <title>Building in a shell</title>
    <para>
      Unfortunatly for us users of Nix, Rebar isn't very cooperative
      with us from the standpoint of building a hermetic
      environment. When building the rebar3 support we had to do some
      sneaky things to get it not to go out and pull packages on its
      own. Also unfortunately, you have to do some of the same things
      when building a project inside of a Nix shell.

      <orderedlist numeration="arabic">
        <listitem>
          <para>Run <literal>rebar3-nix-bootstrap</literal> every time
          dependencies change</para>
        </listitem>
        <listitem>
          <para>Set Home to the current directory.</para>
        </listitem>
      </orderedlist>

      If you do these two things then Rebar will be happy with you. I
      codify these into a makefile. Forunately, rebar3-nix-bootstrap
      is idempotent and fairly quick. so you can run it as often as
      you like.
    </para>
    <programlisting>
# =============================================================================
# Rules
# =============================================================================
.PHONY= all test clean repl shell build test analyze bootstrap

all: test

clean:
        rm -rf _build
        rm -rf .cache

repl:
        nix-shell --run "erl"

shell:
        nix-shell --run "bash"

bootstrap:
        nix-shell --pure --run "rebar3-nix-bootstrap"

build: bootstrap
        nix-shell --pure --run "HOME=$(CURDIR) rebar3 compile"

analyze: bootstrap
        nix-shell --pure --run "HOME=$(CURDIR) rebar3 do compile,dialyzer"

test: bootstrap
        nix-shell --pure --run "HOME=$(CURDIR) rebar3 do compile,dialyzer,eunit"

    </programlisting>
    <para>
      If you add the <literal>shell.nix</literal> as described and
      user rebar as follows things should simply work.
    </para>
  </section>
</section>
</section>
<section xml:id="generating-packages-from-hex-with-hex2nix">
  <title>Generating Packages from Hex with Hex2Nix</title>
  <para>
    Updating the Hex packages requires the use of the
    <literal>hex2nix</literal> tool. Given the path to the Erlang
    modules (usually
    <literal>pkgs/development/erlang-modules</literal>). It will
    happily dump a file called
    <literal>hex-packages.nix</literal>. That file will contain all
    the packages that use a recognized build system in Hex. However,
    it can't know whether or not all those packages are buildable.
    </para>
    <para>
      To make life easier for our users, it makes good sense to go
      ahead and attempt to build all those packages and remove the
      ones that don't build. To do that, simply run the command (in
      the root of your <literal>nixpkgs</literal> repository). that follows.
    </para>
    <programlisting>
$ nix-build -A erlangPackages
    </programlisting>
    <para>
      That will build every package in
      <literal>erlangPackages</literal>. Then you can go through and
      manually remove the ones that fail. Hopefully, someone will
      improve <literal>hex2nix</literal> in the future to automate
      that.
    </para>
</section>
</chapter>