path: root/nixos/doc/manual/configuration/gpu-accel.xml

<chapter xmlns="http://docbook.org/ns/docbook"
         xmlns:xlink="http://www.w3.org/1999/xlink"
         xmlns:xi="http://www.w3.org/2001/XInclude"
         version="5.0"
         xml:id="sec-gpu-accel">
  <title>GPU acceleration</title>

  <para>
    NixOS provides various APIs that benefit from GPU hardware
    acceleration, such as VA-API and VDPAU for video playback; OpenGL and
    Vulkan for 3D graphics; and OpenCL for general-purpose computing.
    This chapter describes how to set up GPU hardware acceleration (as far
    as this is not done automatically) and how to verify that hardware
    acceleration is indeed used.
  </para>

  <para>
    Most of the aforementioned APIs are agnostic with regards to which
    display server is used. Consequently, these instructions should apply
    both to the X Window System and Wayland compositors.
  </para>

  <section xml:id="sec-gpu-accel-opencl">
    <title>OpenCL</title>

    <para>
      <link xlink:href="https://en.wikipedia.org/wiki/OpenCL">OpenCL</link> is a
      general compute API. It is used by various applications such as
      Blender and Darktable to accelerate certain operations.
    </para>

    <para>
      OpenCL applications load drivers through the <emphasis>Installable Client
      Driver</emphasis> (ICD) mechanism. In this mechanism, an ICD file
      specifies the path to the OpenCL driver for a particular GPU family.
      In NixOS, there are two ways to make ICD files visible to the ICD
      loader. The first is through the <varname>OCL_ICD_VENDORS</varname>
      environment variable. This variable can contain a directory which
      is scanned by the ICL loader for ICD files. For example:

      <screen><prompt>$</prompt> export \
  OCL_ICD_VENDORS=`nix-build '&lt;nixpkgs&gt;' --no-out-link -A rocm-opencl-icd`/etc/OpenCL/vendors/</screen>
    </para>

    <para>
      The second mechanism is to add the OpenCL driver package to
      <xref linkend="opt-hardware.opengl.extraPackages"/>. This links the
      ICD file under <filename>/run/opengl-driver</filename>, where it will
      be visible to the ICD loader.
    </para>

    <para>
      The proper installation of OpenCL drivers can be verified through
      the <command>clinfo</command> command of the <package>clinfo</package>
      package. This command will report the number of hardware devices
      that is found and give detailed information for each device:
    </para>

    <screen><prompt>$</prompt> clinfo | head -n3
Number of platforms  1
Platform Name        AMD Accelerated Parallel Processing
Platform Vendor      Advanced Micro Devices, Inc.</screen>

    <section xml:id="sec-gpu-accel-opencl-amd">
      <title>AMD</title>

      <para>
       Modern AMD <link
       xlink:href="https://en.wikipedia.org/wiki/Graphics_Core_Next">Graphics
       Core Next</link> (GCN) GPUs are supported through the
       <package>rocm-opencl-icd</package> package. Adding this package to
       <xref linkend="opt-hardware.opengl.extraPackages"/> enables OpenCL
       support:

       <programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [
         rocm-opencl-icd
       ];</programlisting>
      </para>
    </section>

    <section xml:id="sec-gpu-accel-opencl-intel">
      <title>Intel</title>

      <para>
       <link
         xlink:href="https://en.wikipedia.org/wiki/List_of_Intel_graphics_processing_units#Gen8">Intel
       Gen8 and later GPUs</link> are supported by the Intel NEO OpenCL
       runtime that is provided by the
       <package>intel-compute-runtime</package> package. For Gen7 GPUs,
       the deprecated Beignet runtime can be used, which is provided
       by the <package>beignet</package> package. The proprietary Intel
       OpenCL runtime, in the <package>intel-ocl</package> package, is
       an alternative for Gen7 GPUs.
      </para>

      <para>
       The <package>intel-compute-runtime</package>, <package>beignet</package>,
       or <package>intel-ocl</package> package can be added to
       <xref linkend="opt-hardware.opengl.extraPackages"/> to enable OpenCL
       support. For example, for Gen8 and later GPUs, the following
       configuration can be used:

      <programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [
        intel-compute-runtime
      ];</programlisting>

      </para>
    </section>
  </section>

  <section xml:id="sec-gpu-accel-vulkan">
    <title>Vulkan</title>

    <para>
      <link xlink:href="https://en.wikipedia.org/wiki/Vulkan_(API)">Vulkan</link> is a
      graphics and compute API for GPUs. It is used directly by games or indirectly though
      compatibility layers like <link xlink:href="https://github.com/doitsujin/dxvk/wiki">DXVK</link>.
    </para>

    <para>
     By default, if <xref linkend="opt-hardware.opengl.driSupport"/> is enabled,
     <package>mesa</package> is installed and provides Vulkan for supported hardware.
    </para>

    <para>
      Similar to OpenCL, Vulkan drivers are loaded through the <emphasis>Installable Client
      Driver</emphasis> (ICD) mechanism. ICD files for Vulkan are JSON files that specify
      the path to the driver library and the supported Vulkan version. All successfully
      loaded drivers are exposed to the application as different GPUs.
      In NixOS, there are two ways to make ICD files visible to Vulkan applications: an
      environment variable and a module option.
    </para>

    <para>
      The first option is through the <varname>VK_ICD_FILENAMES</varname>
      environment variable. This variable can contain multiple JSON files, separated by
      <literal>:</literal>. For example:

      <screen><prompt>$</prompt> export \
  VK_ICD_FILENAMES=`nix-build '&lt;nixpkgs&gt;' --no-out-link -A amdvlk`/share/vulkan/icd.d/amd_icd64.json</screen>
    </para>

    <para>
      The second mechanism is to add the Vulkan driver package to
      <xref linkend="opt-hardware.opengl.extraPackages"/>. This links the
      ICD file under <filename>/run/opengl-driver</filename>, where it will
      be visible to the ICD loader.
    </para>

    <para>
      The proper installation of Vulkan drivers can be verified through
      the <command>vulkaninfo</command> command of the <package>vulkan-tools</package>
      package. This command will report the hardware devices and drivers found,
      in this example output amdvlk and radv:
    </para>

    <screen><prompt>$</prompt> vulkaninfo | grep GPU
                GPU id  : 0 (Unknown AMD GPU)
                GPU id  : 1 (AMD RADV NAVI10 (LLVM 9.0.1))
     ...
GPU0:
        deviceType     = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
        deviceName     = Unknown AMD GPU
GPU1:
        deviceType     = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU</screen>

    <para>
      A simple graphical application that uses Vulkan is <command>vkcube</command>
      from the <package>vulkan-tools</package> package.
    </para>

    <section xml:id="sec-gpu-accel-vulkan-amd">
      <title>AMD</title>

      <para>
       Modern AMD <link
       xlink:href="https://en.wikipedia.org/wiki/Graphics_Core_Next">Graphics
       Core Next</link> (GCN) GPUs are supported through either radv, which is
       part of <package>mesa</package>, or the <package>amdvlk</package> package.
       Adding the <package>amdvlk</package> package to
       <xref linkend="opt-hardware.opengl.extraPackages"/> makes amdvlk the
       default driver and hides radv and lavapipe from the device list. A
       specific driver can be forced as follows:

       <programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [
         pkgs.<package>amdvlk</package>
       ];

       # To enable Vulkan support for 32-bit applications, also add:
       <xref linkend="opt-hardware.opengl.extraPackages32"/> = [
         pkgs.driversi686Linux.<package>amdvlk</package>
       ];

       # Force radv
       <xref linkend="opt-environment.variables"/>.AMD_VULKAN_ICD = "RADV";
       # Or
       <xref linkend="opt-environment.variables"/>.VK_ICD_FILENAMES =
         "/run/opengl-driver/share/vulkan/icd.d/radeon_icd.x86_64.json";
       </programlisting>
      </para>
    </section>
  </section>

  <section xml:id="sec-gpu-accel-common-issues">
   <title>Common issues</title>

   <section xml:id="sec-gpu-accel-common-issues-permissions">
    <title>User permissions</title>

    <para>
     Except where noted explicitly, it should not be necessary to
     adjust user permissions to use these acceleration APIs. In the default
     configuration, GPU devices have world-read/write permissions
     (<filename>/dev/dri/renderD*</filename>) or are tagged as
     <code>uaccess</code> (<filename>/dev/dri/card*</filename>).  The
     access control lists of devices with the <varname>uaccess</varname>
     tag will be updated automatically when a user logs in through
     <command>systemd-logind</command>. For example, if the user
     <emphasis>jane</emphasis> is logged in, the access control list
     should look as follows:

     <screen><prompt>$</prompt> getfacl /dev/dri/card0
# file: dev/dri/card0
# owner: root
# group: video
user::rw-
user:jane:rw-
group::rw-
mask::rw-
other::---</screen>

     If you disabled (this functionality of) <command>systemd-logind</command>,
     you may need to add the user to the <code>video</code> group and
     log in again.
    </para>
   </section>

   <section xml:id="sec-gpu-accel-common-issues-mixing-nixpkgs">
    <title>Mixing different versions of nixpkgs</title>

    <para>
     The <emphasis>Installable Client Driver</emphasis> (ICD)
     mechanism used by OpenCL and Vulkan loads runtimes into its address
     space using <code>dlopen</code>. Mixing an ICD loader mechanism and
     runtimes from different version of nixpkgs may not work. For example,
     if the ICD loader uses an older version of <package>glibc</package>
     than the runtime, the runtime may not be loadable due to
     missing symbols. Unfortunately, the loader will generally be quiet
     about such issues.
    </para>

    <para>
     If you suspect that you are running into library version mismatches
     between an ICL loader and a runtime, you could run an application with
     the <code>LD_DEBUG</code> variable set to get more diagnostic
     information. For example, OpenCL can be tested with
     <code>LD_DEBUG=files clinfo</code>, which should report missing
     symbols.
    </para>
   </section>
  </section>
</chapter>