1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
|
# GPU acceleration {#sec-gpu-accel}
NixOS provides various APIs that benefit from GPU hardware acceleration,
such as VA-API and VDPAU for video playback; OpenGL and Vulkan for 3D
graphics; and OpenCL for general-purpose computing. This chapter
describes how to set up GPU hardware acceleration (as far as this is not
done automatically) and how to verify that hardware acceleration is
indeed used.
Most of the aforementioned APIs are agnostic with regards to which
display server is used. Consequently, these instructions should apply
both to the X Window System and Wayland compositors.
## OpenCL {#sec-gpu-accel-opencl}
[OpenCL](https://en.wikipedia.org/wiki/OpenCL) is a general compute API.
It is used by various applications such as Blender and Darktable to
accelerate certain operations.
OpenCL applications load drivers through the *Installable Client Driver*
(ICD) mechanism. In this mechanism, an ICD file specifies the path to
the OpenCL driver for a particular GPU family. In NixOS, there are two
ways to make ICD files visible to the ICD loader. The first is through
the `OCL_ICD_VENDORS` environment variable. This variable can contain a
directory which is scanned by the ICL loader for ICD files. For example:
```ShellSession
$ export \
OCL_ICD_VENDORS=`nix-build '<nixpkgs>' --no-out-link -A rocm-opencl-icd`/etc/OpenCL/vendors/
```
The second mechanism is to add the OpenCL driver package to
[](#opt-hardware.opengl.extraPackages).
This links the ICD file under `/run/opengl-driver`, where it will be visible
to the ICD loader.
The proper installation of OpenCL drivers can be verified through the
`clinfo` command of the clinfo package. This command will report the
number of hardware devices that is found and give detailed information
for each device:
```ShellSession
$ clinfo | head -n3
Number of platforms 1
Platform Name AMD Accelerated Parallel Processing
Platform Vendor Advanced Micro Devices, Inc.
```
### AMD {#sec-gpu-accel-opencl-amd}
Modern AMD [Graphics Core
Next](https://en.wikipedia.org/wiki/Graphics_Core_Next) (GCN) GPUs are
supported through the rocm-opencl-icd package. Adding this package to
[](#opt-hardware.opengl.extraPackages)
enables OpenCL support:
```nix
hardware.opengl.extraPackages = [
rocm-opencl-icd
];
```
### Intel {#sec-gpu-accel-opencl-intel}
[Intel Gen8 and later
GPUs](https://en.wikipedia.org/wiki/List_of_Intel_graphics_processing_units#Gen8)
are supported by the Intel NEO OpenCL runtime that is provided by the
intel-compute-runtime package. For Gen7 GPUs, the deprecated Beignet
runtime can be used, which is provided by the beignet package. The
proprietary Intel OpenCL runtime, in the intel-ocl package, is an
alternative for Gen7 GPUs.
The intel-compute-runtime, beignet, or intel-ocl package can be added to
[](#opt-hardware.opengl.extraPackages)
to enable OpenCL support. For example, for Gen8 and later GPUs, the following
configuration can be used:
```nix
hardware.opengl.extraPackages = [
intel-compute-runtime
];
```
## Vulkan {#sec-gpu-accel-vulkan}
[Vulkan](https://en.wikipedia.org/wiki/Vulkan_(API)) is a graphics and
compute API for GPUs. It is used directly by games or indirectly though
compatibility layers like
[DXVK](https://github.com/doitsujin/dxvk/wiki).
By default, if [](#opt-hardware.opengl.driSupport)
is enabled, mesa is installed and provides Vulkan for supported hardware.
Similar to OpenCL, Vulkan drivers are loaded through the *Installable
Client Driver* (ICD) mechanism. ICD files for Vulkan are JSON files that
specify the path to the driver library and the supported Vulkan version.
All successfully loaded drivers are exposed to the application as
different GPUs. In NixOS, there are two ways to make ICD files visible
to Vulkan applications: an environment variable and a module option.
The first option is through the `VK_ICD_FILENAMES` environment variable.
This variable can contain multiple JSON files, separated by `:`. For
example:
```ShellSession
$ export \
VK_ICD_FILENAMES=`nix-build '<nixpkgs>' --no-out-link -A amdvlk`/share/vulkan/icd.d/amd_icd64.json
```
The second mechanism is to add the Vulkan driver package to
[](#opt-hardware.opengl.extraPackages).
This links the ICD file under `/run/opengl-driver`, where it will be
visible to the ICD loader.
The proper installation of Vulkan drivers can be verified through the
`vulkaninfo` command of the vulkan-tools package. This command will
report the hardware devices and drivers found, in this example output
amdvlk and radv:
```ShellSession
$ vulkaninfo | grep GPU
GPU id : 0 (Unknown AMD GPU)
GPU id : 1 (AMD RADV NAVI10 (LLVM 9.0.1))
...
GPU0:
deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
deviceName = Unknown AMD GPU
GPU1:
deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
```
A simple graphical application that uses Vulkan is `vkcube` from the
vulkan-tools package.
### AMD {#sec-gpu-accel-vulkan-amd}
Modern AMD [Graphics Core
Next](https://en.wikipedia.org/wiki/Graphics_Core_Next) (GCN) GPUs are
supported through either radv, which is part of mesa, or the amdvlk
package. Adding the amdvlk package to
[](#opt-hardware.opengl.extraPackages)
makes amdvlk the default driver and hides radv and lavapipe from the device list.
A specific driver can be forced as follows:
```nix
hardware.opengl.extraPackages = [
pkgs.amdvlk
];
# To enable Vulkan support for 32-bit applications, also add:
hardware.opengl.extraPackages32 = [
pkgs.driversi686Linux.amdvlk
];
# Force radv
environment.variables.AMD_VULKAN_ICD = "RADV";
# Or
environment.variables.VK_ICD_FILENAMES =
"/run/opengl-driver/share/vulkan/icd.d/radeon_icd.x86_64.json";
```
## Common issues {#sec-gpu-accel-common-issues}
### User permissions {#sec-gpu-accel-common-issues-permissions}
Except where noted explicitly, it should not be necessary to adjust user
permissions to use these acceleration APIs. In the default
configuration, GPU devices have world-read/write permissions
(`/dev/dri/renderD*`) or are tagged as `uaccess` (`/dev/dri/card*`). The
access control lists of devices with the `uaccess` tag will be updated
automatically when a user logs in through `systemd-logind`. For example,
if the user *jane* is logged in, the access control list should look as
follows:
```ShellSession
$ getfacl /dev/dri/card0
# file: dev/dri/card0
# owner: root
# group: video
user::rw-
user:jane:rw-
group::rw-
mask::rw-
other::---
```
If you disabled (this functionality of) `systemd-logind`, you may need
to add the user to the `video` group and log in again.
### Mixing different versions of nixpkgs {#sec-gpu-accel-common-issues-mixing-nixpkgs}
The *Installable Client Driver* (ICD) mechanism used by OpenCL and
Vulkan loads runtimes into its address space using `dlopen`. Mixing an
ICD loader mechanism and runtimes from different version of nixpkgs may
not work. For example, if the ICD loader uses an older version of glibc
than the runtime, the runtime may not be loadable due to missing
symbols. Unfortunately, the loader will generally be quiet about such
issues.
If you suspect that you are running into library version mismatches
between an ICL loader and a runtime, you could run an application with
the `LD_DEBUG` variable set to get more diagnostic information. For
example, OpenCL can be tested with `LD_DEBUG=files clinfo`, which should
report missing symbols.
|