Build errors with golang bindings about foundationdb

[brk0v]

What works for me:

1. Install client binary dep from: https://apple.github.io/foundationdb/downloads.html

2. Install mono:

sudo apt-get install mono-complete

3. Get last bindings:

go get github.com/apple/foundationdb
cd $GOPATH/src/github.com/apple/foundationdb/bindings/go/

4. Use local install:

./fdb-go-install.sh localinstall

5. Rewrite version to 510:

find . -name '*.go' | xargs -I{} sed -i  's/520/510/g' {}

6. Export build flags:

export   CGO_CFLAGS="-g -O2 -I/home/vagrant/gopath//src/github.com/apple/foundationdb/bindings/c"
export   CGO_LDFLAGS="-g -O2 -L/usr/lib"

7. Build your go code:

cd %YOUR_REPO_HERE%
go build

[alecgrieser]

Hm, well, I'm not sure exactly what's going on with that linker error. This appears to be a regression between release-5.2 and master, so I'm trying to track that down. As a preliminary thing, building off of the release-5.2 or even release-5.1 branches should fix the problem, I would think.

As the to the API version, this came up a little with #211, but I can go a little more into the API version and its usage in the go bindings. So, each version of libfdb_c has a maximum API version that it supports. For release-5.1, that version is 510. For release-5.2, that version is 520. For master, that version is currently 520, though it will probably be bumped to 600 at some point. Each binding also sets a header version specifying the minimum API version of the C bindings they expect which matches the "default" API version for that release. So, the go bindings on the master branch require a version of the C bindings that can support a version of the FDB API that is version 520 or newer, i.e., version 5.2 or newer, i.e., not the version we ship in our packages.

My recommendation would thus be that for those who are building the go bindings just because they want to write a go application, then my recommendation would be to follow @brk0v's instructions, but with one of two modifications. (1) Instead of changing all of the API versions through sed and find, run git checkout 5.1.5 to go to the right tag and then run localinstall, or (2) download the fdb-go-install script on its own and run fdb-go-install.sh install instead of go get and localinstall (and then skip the re-write of the API versions). The script was designed to work outside the context of the repo, but perhaps it has some weird dependency I wasn't aware of.

If you are trying to build the go bindings because you wish to do work on the go bindings, you probably want to build them against the appropriate (newer) version of libfdb_c (and fdbserver), in which case you should be using the versions of those that you have built from your repo rather than directly from the client packages.

It's possible that the real issue was the friends we made along the way here is that we need to play nicer with go's package manager and dependency versioning system. We are somewhat encumbered by the fact that we have some generated files that our build process has to generate (hence the mono dependency), and its possible that there is a more idiomatic way to handle semantic versioning within packages that we aren't playing nicely with.

[GavinYang-LR]

I have same problem when build in docker vm.

/usr/local/foundationdb/lib/libfdb_c.so: undefined reference to `std::istream::ignore(long)'
/usr/local/foundationdb/lib/libfdb_c.so: undefined reference to `std::basic_istream<wchar_t, std::char_traits<wchar_t> >::ignore(long)'
collect2: error: ld returned 1 exit status
bindings/go/include.mk:98: recipe for target '/usr/local/foundationdb/bindings/go/build/pkg/linux_amd64/github.com/apple/foundationdb/bindings/go/src/fdb.a' failed

[alexmiller-apple]

$ ar x /usr/lib/gcc/x86_64-linux-gnu/4.9/libstdc++.a
$ for file in *.o; do nm $file | c++filt | grep basic_istream ; done | grep ignore
0000000000000000 T std::basic_istream<wchar_t, std::char_traits<wchar_t> >::ignore(long)

$ ar x /usr/lib/gcc/x86_64-linux-gnu/4.9/libstdc++_pic.a
$ for file in *.o; do nm $file | c++filt | grep basic_istream ; done | grep ignore
                 U std::basic_istream<wchar_t, std::char_traits<wchar_t> >::ignore(long)

The PIC and non-PIC versions of ubuntu's gcc 4.9 libstdc++ provide different symbols. Good lord.

I think I can work around specifically this, so please excuse me while I go write a truly disgusting patch.

[GavinYang-LR]

@brk0v Good job，it's work for me. And why we need find . -name '*.go' | xargs -I{} sed -i 's/520/510/g' {} If i don't rewrite version, go build will work for my demo project,but can't execute demo:
panic: This version of the FoundationDB Go binding is not supported by the installed FoundationDB C library. The binding requires a library that supports API version 200, but the installed library supports a maximum version of 510.

[GavinYang-LR]

@alecgrieser
I try modification (1):

[wade.xu@ntgtstapp44 go]$ ./fdb-go-install.sh localinstall
Building generated files.
make -C /home/wade.xu/go/src/github.com/apple/foundation bindings/c/foundationdb/fdb_c_options.g.h
make: *** /home/wade.xu/go/src/github.com/apple/foundation: No such file or directory.  Stop.
Could not generate required c header

==============================================================
Update:

1. checkout branch release-5.1
2. merage #253 into release-5.1

file:fdb-go-install.sh

-            fdbdir="${destdir}/foundation"
+            fdbdir="${destdir}/foundationdb"

[hgray1]

Closing as it looks like it's fixed. Please re-open if needed. Thanks!

[mhindery]

I'm still running into go errors when building a container for a Go client program. Using this Dockerfile

FROM ubuntu:18.04

RUN apt-get update && apt-get install -y git golang-go wget

# Command line arguments for the go build
ENV GOOS=linux GOARCH=amd64 CGO_ENABLED=0 GOPATH=/go/ GOBIN=/go/bin

# Install FoundationDB client libraries
RUN wget https://www.foundationdb.org/downloads/5.2.5/ubuntu/installers/foundationdb-clients_5.2.5-1_amd64.deb && dpkg -i foundationdb-clients_5.2.5-1_amd64.deb && rm foundationdb-clients_5.2.5-1_amd64.deb

# Fetch dependencies and compile
CMD go get -a -ldflags '-s' {program_name}

Building and running this (mount my src folder at runtim):

docker build -t test .
docker run -it -v $PWD:/go/ -w /go/src/ test /bin/bash

I get the following error:

# github.com/apple/foundationdb/bindings/go/src/fdb
github.com/apple/foundationdb/bindings/go/src/fdb/generated.go:48:9: undefined: NetworkOptions

I have tried to do a git checkout release-5.2 in the source code folder, that did not do anything. Any advice?

[alecgrieser]

I think we've seen errors of this form when people have tried cross-compiling (e.g., Linux on macOS). The NetworkOptions struct is defined in fdb.go:

foundationdb/bindings/go/src/fdb/fdb.go

Lines 82 to 86 in f4b3299

	// NetworkOptions is a handle with which to set options that affect the entire
	// FoundationDB client. A NetworkOptions instance should be obtained with the
	// fdb.Options function.
	type NetworkOptions struct {
	}

So, I'm not sure what this is. It's possible the cross-compiling thing was a red herring given this bug report. Hm.

[alecgrieser]

Okay, I figured it out.

tl;dr Set the CGO_ENABLED environment variable to 1

The problem is related to cgo, which we need to bridge our go bindings with the C client. Apparently, cgo compilation is determined by the CGO_ENABLED environment variable. From the cgo docs:

The cgo tool is enabled by default for native builds on systems where it is expected to work. It is disabled by default when cross-compiling. You can control this by setting the CGO_ENABLED environment variable when running the go tool: set it to 1 to enable the use of cgo, and to 0 to disable it. The go tool will set the build constraint "cgo" if cgo is enabled. The special import "C" implies the "cgo" build constraint, as though the file also said "// +build cgo". Therefore, if cgo is disabled, files that import "C" will not be built by the go tool. (For more about build constraints see https://golang.org/pkg/go/build/#hdr-Build_Constraints).

When I ran your command in the docker file you provided, I noticed that, for whatever reason, the CGO_ENABLED environment variable was set to 0. I guess this means that go doesn't expect the native build to "work", which is...a little concerning for single platform builds. (Perhaps reasonable for cross-platform builds.) Looking at build commands, it appears that go skips over those files that require cgo silently without any kind of visible warning. That being said, I set it, and then the build succeeded.

I suspect you will still need to set the repo to the release-5.2 branch (or, like, the 5.2.7 tag--take your pick of 5.2 tags). The fdb-go-install.sh script is supposed to encapsulate much of the weirdness of dealing with our go bindings. It is a standalone script that can be run without the rest of the repository. It doesn't set the CGO_ENABLED flag to 1 (yet), but maybe it should.