epoxy: explicitly search libGL path as fallback

Don't rely on questionable impact of DT_RPATH on dlopen().

This is a bit of a messy subject, but probably the clearest
reference to motivate *not* relying on how dlopen() behaves
in the presence of RPATH or RUNPATH is the following:

https://sourceware.org/ml/libc-hacker/2002-11/msg00011.html

FWIW the dlopen() manpage only mentions the the RPATH
and RUNPATH in the "executable file for the calling program";
no mention of the executable files for libraries--
this has been brought to the attention of the relevant
parties and AFAICT nothing has been done.

The best reference for glibc behavior is
apparently to ... "try it and see".
Luckily a generous soul did exactly that
and reported the findings:

https://www.spinics.net/lists/linux-man/msg02291.html

Qt wrote on the subject a bit when they were bit by this,
linking to the above articles (directly or indirectly).

See:
http://blog.qt.io/blog/2011/10/28/rpath-and-runpath/

--------

Since we know the path of libGL at build-time for libepoxy,
there's a simple solution we can use to avoid all of this:
simply teach libepoxy to explicitly look in the libGL path.

This commit patches libepoxy to accomplish this,
looking to "LIBGL_PATH" as a fallback if it cannot find
the libraries otherwise.

---------

This fixes use of libepoxy w/musl on NixOS!

2 files changed, 38 insertions, 4 deletions

diff --git a/pkgs/development/libraries/epoxy/default.nix b/pkgs/development/libraries/epoxy/default.nix
index 81966f14a56..e0f8e9ff8c7 100644
--- a/pkgs/development/libraries/epoxy/default.nix
+++ b/pkgs/development/libraries/epoxy/default.nix
@@ -25,10 +25,9 @@ stdenv.mkDerivation rec {
     substituteInPlace src/dispatch_common.h --replace "PLATFORM_HAS_GLX 0" "PLATFORM_HAS_GLX 1"
   '';
 
-  # add libGL to rpath because libepoxy dlopen()s libEGL
-  postFixup = optionalString stdenv.isLinux ''
-    patchelf --set-rpath "${stdenv.lib.makeLibraryPath [ libGL ]}:$(patchelf --print-rpath $out/lib/libepoxy.so.0.0.0)" $out/lib/libepoxy.so.0.0.0
-  '';
+  patches = [ ./libgl-path.patch ];
+
+  NIX_CFLAGS_COMPILE = ''-DLIBGL_PATH="${getLib libGL}/lib"'';
 
   meta = {
     description = "A library for handling OpenGL function pointer management";
diff --git a/pkgs/development/libraries/epoxy/libgl-path.patch b/pkgs/development/libraries/epoxy/libgl-path.patch
new file mode 100644
index 00000000000..6f50b9d262b
--- /dev/null
+++ b/pkgs/development/libraries/epoxy/libgl-path.patch
@@ -0,0 +1,35 @@
+From 4046e0ac8ed93354c01de5f3b5cae790cce70404 Mon Sep 17 00:00:00 2001
+From: Will Dietz <w@wdtz.org>
+Date: Thu, 29 Mar 2018 07:21:02 -0500
+Subject: [PATCH] Explicitly search LIBGL_PATH as fallback, if defined.
+
+---
+ src/dispatch_common.c | 12 ++++++++++++
+ 1 file changed, 12 insertions(+)
+
+diff --git a/src/dispatch_common.c b/src/dispatch_common.c
+index bc2fb94..776237b 100644
+--- a/src/dispatch_common.c
++++ b/src/dispatch_common.c
+@@ -306,6 +306,18 @@ get_dlopen_handle(void **handle, const char *lib_name, bool exit_on_fail)
+     pthread_mutex_lock(&api.mutex);
+     if (!*handle) {
+         *handle = dlopen(lib_name, RTLD_LAZY | RTLD_LOCAL);
++#ifdef LIBGL_PATH
++        if (!*handle) {
++          char pathbuf[sizeof(LIBGL_PATH) + 1 + 1024 + 1];
++          int l = snprintf(pathbuf, sizeof(pathbuf), "%s/%s", LIBGL_PATH, lib_name);
++          if (l < 0 || l >= sizeof(pathbuf)) {
++            // This really shouldn't happen
++            fprintf(stderr, "Error prefixing library pathname\n");
++            exit(1);
++          }
++          *handle = dlopen(pathbuf, RTLD_LAZY | RTLD_LOCAL);
++        }
++#endif
+         if (!*handle) {
+             if (exit_on_fail) {
+                 fprintf(stderr, "Couldn't open %s: %s\n", lib_name, dlerror());
+-- 
+2.16.3
+