1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
|
<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-gpu-accel">
<title>GPU acceleration</title>
<para>
NixOS provides various APIs that benefit from GPU hardware
acceleration, such as VA-API and VDPAU for video playback; OpenGL
and Vulkan for 3D graphics; and OpenCL for general-purpose
computing. This chapter describes how to set up GPU hardware
acceleration (as far as this is not done automatically) and how to
verify that hardware acceleration is indeed used.
</para>
<para>
Most of the aforementioned APIs are agnostic with regards to which
display server is used. Consequently, these instructions should
apply both to the X Window System and Wayland compositors.
</para>
<section xml:id="sec-gpu-accel-opencl">
<title>OpenCL</title>
<para>
<link xlink:href="https://en.wikipedia.org/wiki/OpenCL">OpenCL</link>
is a general compute API. It is used by various applications such
as Blender and Darktable to accelerate certain operations.
</para>
<para>
OpenCL applications load drivers through the <emphasis>Installable
Client Driver</emphasis> (ICD) mechanism. In this mechanism, an
ICD file specifies the path to the OpenCL driver for a particular
GPU family. In NixOS, there are two ways to make ICD files visible
to the ICD loader. The first is through the
<literal>OCL_ICD_VENDORS</literal> environment variable. This
variable can contain a directory which is scanned by the ICL
loader for ICD files. For example:
</para>
<programlisting>
$ export \
OCL_ICD_VENDORS=`nix-build '<nixpkgs>' --no-out-link -A rocm-opencl-icd`/etc/OpenCL/vendors/
</programlisting>
<para>
The second mechanism is to add the OpenCL driver package to
<xref linkend="opt-hardware.opengl.extraPackages" />. This links
the ICD file under <literal>/run/opengl-driver</literal>, where it
will be visible to the ICD loader.
</para>
<para>
The proper installation of OpenCL drivers can be verified through
the <literal>clinfo</literal> command of the clinfo package. This
command will report the number of hardware devices that is found
and give detailed information for each device:
</para>
<programlisting>
$ clinfo | head -n3
Number of platforms 1
Platform Name AMD Accelerated Parallel Processing
Platform Vendor Advanced Micro Devices, Inc.
</programlisting>
<section xml:id="sec-gpu-accel-opencl-amd">
<title>AMD</title>
<para>
Modern AMD
<link xlink:href="https://en.wikipedia.org/wiki/Graphics_Core_Next">Graphics
Core Next</link> (GCN) GPUs are supported through the
rocm-opencl-icd package. Adding this package to
<xref linkend="opt-hardware.opengl.extraPackages" /> enables
OpenCL support:
</para>
<programlisting language="bash">
hardware.opengl.extraPackages = [
rocm-opencl-icd
];
</programlisting>
</section>
<section xml:id="sec-gpu-accel-opencl-intel">
<title>Intel</title>
<para>
<link xlink:href="https://en.wikipedia.org/wiki/List_of_Intel_graphics_processing_units#Gen8">Intel
Gen8 and later GPUs</link> are supported by the Intel NEO OpenCL
runtime that is provided by the intel-compute-runtime package.
For Gen7 GPUs, the deprecated Beignet runtime can be used, which
is provided by the beignet package. The proprietary Intel OpenCL
runtime, in the intel-ocl package, is an alternative for Gen7
GPUs.
</para>
<para>
The intel-compute-runtime, beignet, or intel-ocl package can be
added to <xref linkend="opt-hardware.opengl.extraPackages" /> to
enable OpenCL support. For example, for Gen8 and later GPUs, the
following configuration can be used:
</para>
<programlisting language="bash">
hardware.opengl.extraPackages = [
intel-compute-runtime
];
</programlisting>
</section>
</section>
<section xml:id="sec-gpu-accel-vulkan">
<title>Vulkan</title>
<para>
<link xlink:href="https://en.wikipedia.org/wiki/Vulkan_(API)">Vulkan</link>
is a graphics and compute API for GPUs. It is used directly by
games or indirectly though compatibility layers like
<link xlink:href="https://github.com/doitsujin/dxvk/wiki">DXVK</link>.
</para>
<para>
By default, if <xref linkend="opt-hardware.opengl.driSupport" />
is enabled, mesa is installed and provides Vulkan for supported
hardware.
</para>
<para>
Similar to OpenCL, Vulkan drivers are loaded through the
<emphasis>Installable Client Driver</emphasis> (ICD) mechanism.
ICD files for Vulkan are JSON files that specify the path to the
driver library and the supported Vulkan version. All successfully
loaded drivers are exposed to the application as different GPUs.
In NixOS, there are two ways to make ICD files visible to Vulkan
applications: an environment variable and a module option.
</para>
<para>
The first option is through the
<literal>VK_ICD_FILENAMES</literal> environment variable. This
variable can contain multiple JSON files, separated by
<literal>:</literal>. For example:
</para>
<programlisting>
$ export \
VK_ICD_FILENAMES=`nix-build '<nixpkgs>' --no-out-link -A amdvlk`/share/vulkan/icd.d/amd_icd64.json
</programlisting>
<para>
The second mechanism is to add the Vulkan driver package to
<xref linkend="opt-hardware.opengl.extraPackages" />. This links
the ICD file under <literal>/run/opengl-driver</literal>, where it
will be visible to the ICD loader.
</para>
<para>
The proper installation of Vulkan drivers can be verified through
the <literal>vulkaninfo</literal> command of the vulkan-tools
package. This command will report the hardware devices and drivers
found, in this example output amdvlk and radv:
</para>
<programlisting>
$ vulkaninfo | grep GPU
GPU id : 0 (Unknown AMD GPU)
GPU id : 1 (AMD RADV NAVI10 (LLVM 9.0.1))
...
GPU0:
deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
deviceName = Unknown AMD GPU
GPU1:
deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
</programlisting>
<para>
A simple graphical application that uses Vulkan is
<literal>vkcube</literal> from the vulkan-tools package.
</para>
<section xml:id="sec-gpu-accel-vulkan-amd">
<title>AMD</title>
<para>
Modern AMD
<link xlink:href="https://en.wikipedia.org/wiki/Graphics_Core_Next">Graphics
Core Next</link> (GCN) GPUs are supported through either radv,
which is part of mesa, or the amdvlk package. Adding the amdvlk
package to <xref linkend="opt-hardware.opengl.extraPackages" />
makes amdvlk the default driver and hides radv and lavapipe from
the device list. A specific driver can be forced as follows:
</para>
<programlisting language="bash">
hardware.opengl.extraPackages = [
pkgs.amdvlk
];
# To enable Vulkan support for 32-bit applications, also add:
hardware.opengl.extraPackages32 = [
pkgs.driversi686Linux.amdvlk
];
# Force radv
environment.variables.AMD_VULKAN_ICD = "RADV";
# Or
environment.variables.VK_ICD_FILENAMES =
"/run/opengl-driver/share/vulkan/icd.d/radeon_icd.x86_64.json";
</programlisting>
</section>
</section>
<section xml:id="sec-gpu-accel-common-issues">
<title>Common issues</title>
<section xml:id="sec-gpu-accel-common-issues-permissions">
<title>User permissions</title>
<para>
Except where noted explicitly, it should not be necessary to
adjust user permissions to use these acceleration APIs. In the
default configuration, GPU devices have world-read/write
permissions (<literal>/dev/dri/renderD*</literal>) or are tagged
as <literal>uaccess</literal>
(<literal>/dev/dri/card*</literal>). The access control lists of
devices with the <literal>uaccess</literal> tag will be updated
automatically when a user logs in through
<literal>systemd-logind</literal>. For example, if the user
<emphasis>jane</emphasis> is logged in, the access control list
should look as follows:
</para>
<programlisting>
$ getfacl /dev/dri/card0
# file: dev/dri/card0
# owner: root
# group: video
user::rw-
user:jane:rw-
group::rw-
mask::rw-
other::---
</programlisting>
<para>
If you disabled (this functionality of)
<literal>systemd-logind</literal>, you may need to add the user
to the <literal>video</literal> group and log in again.
</para>
</section>
<section xml:id="sec-gpu-accel-common-issues-mixing-nixpkgs">
<title>Mixing different versions of nixpkgs</title>
<para>
The <emphasis>Installable Client Driver</emphasis> (ICD)
mechanism used by OpenCL and Vulkan loads runtimes into its
address space using <literal>dlopen</literal>. Mixing an ICD
loader mechanism and runtimes from different version of nixpkgs
may not work. For example, if the ICD loader uses an older
version of glibc than the runtime, the runtime may not be
loadable due to missing symbols. Unfortunately, the loader will
generally be quiet about such issues.
</para>
<para>
If you suspect that you are running into library version
mismatches between an ICL loader and a runtime, you could run an
application with the <literal>LD_DEBUG</literal> variable set to
get more diagnostic information. For example, OpenCL can be
tested with <literal>LD_DEBUG=files clinfo</literal>, which
should report missing symbols.
</para>
</section>
</section>
</chapter>
|