.%*. .-.
.%@@@+. .--=%@@@-
=@@@@@@- :--+@@@@@@@@@*
*@%@@@@@%: :--=#@@@@@@@@@@@@#@@+
@@::%@@@@@@-: :::-=%@@@@@@@@@@@@@@%#*- :@*
.@@ =@@@@@@@@@+-:::::::-=*@@@@@@@@@@@@@@@@@@@%##- @@:
#@# +%@@@@@@@@@@@@@@@@@@@@@@@@@@@@@%%#*: -@@
#@- *%@@@@@@@@@@@@@@@@@%%%#= %@=
@@: -=+++== .. @@:
@@ . -@@@ +@@
#@% =@@@@- +@@@@# *+ %@#
#@- -@@@@@@@. +@@@@@% .%@@@@= @@:
@@: +@@@@@@@@@ +@+ +@@@ %@@@@@@@@@ -@@
@@ *@%@* :@@@ *@+ -@@@ %@@@@@@@@@@@# @@@
#@% %@ @% %@@ *@* .@@@ =@@@*@@ :@@@@ @@:
%@:## *@+ :@@ :@@ @@@ %@@: @# .@@@ .@@
@@:. @@: .@@ @@ @@@ %@: @@ @@% @@%
@@ -@@. =@@ @@. @@@ %@. #@- @@# @@.
%@% +@@. @@. %@- @@@ #@ @@. :@@ -@@
%@: +@@+ .@% #@= @@@.@ *@@ +@+ @@*
@@: +@@%. .%% *@+ @@@ %@@ -@# @@
@@ :@@@@@@% :@@ @@% %@@ @# *@@
%@@ #@@@@* @@ @@% %@@. %@. @@:
@@: ===: @@. @@% %@@: =@%. .@@
@@. #@= @@% %@@%+%@#. @@%
@@ +@* @@# -@@@@@* .@@
%@@ :@@ @@# %@@#: =@@
@@: .@@ @@# : @@=
@@. @@. @@# :@@
@@ #@= @@% %@%
#@@ +@+ #@% .@@
@@: =@# +@% =@%
@@ :@% -@@ @@+
@@ .@@ -@: -@@
-@@ @% .: *@#
@@* # @@.
@@ =@@
.@@ @@*
.@@ .@@
*@@ =@#
@@* @@*
@@ -@@
.@@ +@*
.@@ @@-
-@@ =@@
%@% #@*
@@ @@:
@@ +@@
.@@ %@+
:@% @@.
:@% *@%
-@. %@=
%@ @@.
@@ *@%
@@ %@=
@% @@:
:@% +@@
:@* =+- #@+
:@: .%@@@. @@-
-@ =@@@@@# :@@
=@ *@#.=@@% #@%
*@ #@- -@@% %@-
@@ #@: :@@% @@:
@@ #% .@@% .#@@* -@@
@% #% .@@% +@@@@@- %@%
@# %@ :@@% .@@@@@@@@ %@:
@# %@ :@@% +@@% .@@@@ @@:
@# %@ :@@% #@@: :@@@ -@@
@# @@. .@@@ .@@% .@@@= %@%
@# @@. .@@@: %@@- @@@% %@:
@# @@ @@@@ %@@@ *@@@ @@:
@# .@@ @@@@@@@@@@= :@@@ @@
* @@@ :@@@@@@@@. :@@@ *@@
.@@@@@% -@@@@@. .@@@ @@=
@@@@@: .. @@@ @@.
-@%: @@@ @@.
@@@ @@
-@@ =@@
.@@ @@+
.@@:@@.
@@#@-
@@@.
+@-
-
mediocregopher's lil web corner
- Harder than I thought it would be!
When I last left off with the nebula project I wanted to nix-ify the build process for Cryptic's mobile_nebula fork. While I've made progress on the overall build, one particular bit of it really held me up, so I'm writing about that part here. I'll finish the full build at a later time.
gomobile is a toolkit for the go programming language to allow for
running go code on Android and iOS devices. mobile_nebula uses gomobile to
build a simple wrapper around the nebula client that the mobile app can then
hook into.
This means that in order to nix-ify the entire mobile_nebula project I first
need to nix-ify gomobile, and since there isn't (at time of writing) an
existing package for gomobile in the nixpkgs repo, I had to roll my own.
I started with a simple buildGoModule nix expression:
pkgs.buildGoModule {
pname = "gomobile";
version = "unstable-2020-12-17";
src = pkgs.fetchFromGitHub {
owner = "golang";
repo = "mobile";
rev = "e6ae53a27f4fd7cfa2943f2ae47b96cba8eb01c9";
sha256 = "03dzis3xkj0abcm4k95w2zd4l9ygn0rhkj56bzxbcpwa7idqhd62";
};
vendorSha256 = "1n1338vqkc1n8cy94501n7jn3qbr28q9d9zxnq2b4rxsqjfc9l94";
}
The basic idea here is that buildGoModule will acquire a specific revision of
the gomobile source code from github, then attempt to build it. However,
gomobile is a special beast in that it requires a number of C/C++ libraries in
order to be built. I discovered this upon running this expression, when I
received this error:
./work.h:12:10: fatal error: GLES3/gl3.h: No such file or directory
12 | #include <GLES3/gl3.h> // install on Ubuntu with: sudo apt-get install libegl1-mesa-dev libgles2-mesa-dev libx11-dev
This stumped me for a bit, as I couldn't figure out a) the "right" place to
source the GLES3 header file from, and b) how to properly hook that into the
buildGoModule expression. My initial attempts involved trying to include
versions of the header file from my androidsdk nix package which I had already
gotten (mostly) working, but the version which ships there appears to expect to
be using clang. cgo (go's compiler which is used for C/C++ interop) only
supports gcc, so that strategy failed.
I didn't like having to import the header file from androidsdk anyway, as it
meant that my gomobile would only work within the context of the
mobile_nebula project, rather than being a standalone utility.
At this point I flailed around some more trying to figure out where to get this
header file from. Eventually I stumbled on the nix-index project,
which implements something similar to the locate utility on linux: you give it
a file pattern, and it searches your active nix channels for any packages which
provide a file matching that pattern.
Since nix is amazing it's not actually necessary to install nix-index, I
simply start up a shell with the package available using nix-shell -p
nix-index. On first run I needed to populate the index by running the
nix-index command, which took some time, but after that finding packages which
provide the file I need is as easy as:
> nix-shell -p nix-index
[nix-shell:/tmp]$ nix-locate GLES3/gl3.h
(zulip.out) 82,674 r /nix/store/wbfw7w2ixdp317wip77d4ji834v1k1b9-libglvnd-1.3.2-dev/include/GLES3/gl3.h
libglvnd.dev 82,674 r /nix/store/pghxzmnmxdcarg5bj3js9csz0h85g08m-libglvnd-1.3.2-dev/include/GLES3/gl3.h
emscripten.out 82,666 r /nix/store/x3c4y2h5rn1jawybk48r6glzs1jl029s-emscripten-2.0.1/share/emscripten/system/include/GLES3/gl3.h
So my mystery file is provided by a few packages, but libglvnd.dev stood out
to me as it's also the pacman package which provides the same file in my real
operating system:
> yay -Qo /usr/include/GLES3/gl3.h
/usr/include/GLES3/gl3.h is owned by libglvnd 1.3.2-1
This gave me some confidence that this was the right track.
My next fight was with cgo itself. Go's build process provides a few different
entry points for C/C++ compiler/linker flags, including both environment
variables and command-line arguments. But I wasn't using go build directly,
instead I was working through nix's buildGoModule wrapper. This added a huge
layer of confusion as all of nixpkgs is pretty terribly documented, so you
really have to just divine behavior from the source
(good luck).
After lots of debugging (hint: NIX_DEBUG=1) I determined that all which is
actually needed is to set the CGO_CFLAGS variable within the buildGoModule
arguments. This would translate to the CGO_CFLAGS environment variable being
set during all internal commands, and whatever go build commands get used
would pick up my compiler flags from that.
My new nix expression looked like this:
pkgs.buildGoModule {
pname = "gomobile";
version = "unstable-2020-12-17";
src = pkgs.fetchFromGitHub {
owner = "golang";
repo = "mobile";
rev = "e6ae53a27f4fd7cfa2943f2ae47b96cba8eb01c9";
sha256 = "03dzis3xkj0abcm4k95w2zd4l9ygn0rhkj56bzxbcpwa7idqhd62";
};
vendorSha256 = "1n1338vqkc1n8cy94501n7jn3qbr28q9d9zxnq2b4rxsqjfc9l94";
CGO_CFLAGS = [
"-I ${pkgs.libglvnd.dev}/include"
];
}
Running this produced a new error. Progress! The new error was:
/nix/store/p792j5f44l3f0xi7ai5jllwnxqwnka88-binutils-2.31.1/bin/ld: cannot find -lGLESv2
collect2: error: ld returned 1 exit status
So pretty similar to the previous issue, but this time the linker wasn't finding
a library file rather than the compiler not finding a header file. Once again I
used nix-index's nix-locate command to find that this library file is
provided by the libglvnd package (as opposed to libglvnd.dev, which provided
the header file).
Adding libglvnd to the CGO_CFLAGS did not work, as it turns out that flags
for the linker cgo uses get passed in via CGO_LDFLAGS (makes sense). After
adding this new variable I got yet another error; this time X11/Xlib.h was not
able to be found. I repeated the process of nix-locate/add to CGO_*FLAGS a
few more times until all dependencies were accounted for. The new nix expression
looked like this:
pkgs.buildGoModule {
pname = "gomobile";
version = "unstable-2020-12-17";
src = pkgs.fetchFromGitHub {
owner = "golang";
repo = "mobile";
rev = "e6ae53a27f4fd7cfa2943f2ae47b96cba8eb01c9";
sha256 = "03dzis3xkj0abcm4k95w2zd4l9ygn0rhkj56bzxbcpwa7idqhd62";
};
vendorSha256 = "1n1338vqkc1n8cy94501n7jn3qbr28q9d9zxnq2b4rxsqjfc9l94";
CGO_CFLAGS = [
"-I ${pkgs.libglvnd.dev}/include"
"-I ${pkgs.xlibs.libX11.dev}/include"
"-I ${pkgs.xlibs.xorgproto}/include"
"-I ${pkgs.openal}/include"
];
CGO_LDFLAGS = [
"-L ${pkgs.libglvnd}/lib"
"-L ${pkgs.xlibs.libX11}/lib"
"-L ${pkgs.openal}/lib"
];
}
The CGO_*FLAGS variables took care of all compiler/linker errors, but there
was one issue left: buildGoModule apparently runs the project's tests after
the build phase. gomobile's tests were actually mostly passing, but some
failed due to trying to copy files around, which nix was having none of. After
some more buildGoModule source divination I found that
if I passed an empty checkPhase argument it would skip the check phase, and
therefore skip running these tests.
The final nix expression looks like so:
pkgs.buildGoModule {
pname = "gomobile";
version = "unstable-2020-12-17";
src = pkgs.fetchFromGitHub {
owner = "golang";
repo = "mobile";
rev = "e6ae53a27f4fd7cfa2943f2ae47b96cba8eb01c9";
sha256 = "03dzis3xkj0abcm4k95w2zd4l9ygn0rhkj56bzxbcpwa7idqhd62";
};
vendorSha256 = "1n1338vqkc1n8cy94501n7jn3qbr28q9d9zxnq2b4rxsqjfc9l94";
CGO_CFLAGS = [
"-I ${pkgs.libglvnd.dev}/include"
"-I ${pkgs.xlibs.libX11.dev}/include"
"-I ${pkgs.xlibs.xorgproto}/include"
"-I ${pkgs.openal}/include"
];
CGO_LDFLAGS = [
"-L ${pkgs.libglvnd}/lib"
"-L ${pkgs.xlibs.libX11}/lib"
"-L ${pkgs.openal}/lib"
];
checkPhase = "";
}
Once I complete the nix-ification of mobile_nebula I'll submit a PR to the
nixpkgs upstream with this, so that others can have gomobile available as
well!
Published 2021-02-13
This post is part of a series.
Previously: Building Mobile Nebula
Next: Evaluation of Network Filesystems
This site can also be accessed via the gemini protocol: gemini://mediocregopher.com/