Comments (16)
@mewwts The master branch of the Go client requires the master branch of librdkafka (the development is kept in synch).
So you have two options:
- use the latest release of the Go client (v0.9.4) with a recent release of librdkafka (>=v0.9.4)
- remove any installed versions of librdkafka, clone librdkafka master and build and install it.
from confluent-kafka-go.
pkg-config will need to find the rdkafka.pc file that is installed when you do 'make install' in librdkafka, the location depends on ./configure --prefix and defaults to /usr/local.
It might be the case that /usr/local is not in your PKG_CONFIG_PATH, the easiest fix is to install librdkafka into /usr, like this:
$ ./configure --prefix /usr
$ make
$ sudo make install
from confluent-kafka-go.
from confluent-kafka-go.
Thanks, i got working 👍
from confluent-kafka-go.
@liaoyizhe Make sure to install librdkafka v0.11.0 as instructed here:
https://github.com/confluentinc/confluent-kafka-go/#installing-librdkafka
from confluent-kafka-go.
That looks like it is finding librdkafka >0.11.0 when compiling (rdkafka.h header file) but not when linking (library file).
I suggest removing all installed versions of librdkafka from your system, using both your package manager and by searching for rdkafka.h and librdkafka.so*.
Then build and install librdkafka 0.11.3 (latest version) and then rebuild the Go client (go clean -i ./...
first).
from confluent-kafka-go.
I have librdkafka
installed, but still see this error. Could you be a bit more specific on what causes this?
from confluent-kafka-go.
Thanks a bunch @edenhill - that was clarifying!
from confluent-kafka-go.
@edenhill Hi edenhill,
- I have installed librdkafka from source code from git clone https://github.com/edenhill/librdkafka.git and doing make install.
After then cloning "go get -u github.com/confluentinc/confluent-kafka-go/kafka" its giving following error:
pkg-config --cflags rdkafka
Package rdkafka was not found in the pkg-config search path.
Perhaps you should add the directory containing `rdkafka.pc'
to the PKG_CONFIG_PATH environment variable
No package 'rdkafka' found
pkg-config: exit status 1
- If i insalled librdkafka from binary (latest version is 0.9.5) and if i clone confluent-kafka-go its giving following error:
$ go get gopkg.in/confluentinc/confluent-kafka-go.v0/examples/go-kafkacat
github.com/confluentinc/confluent-kafka-go/kafka
could not determine kind of name for C.RD_KAFKA_EVENT_STATS
Do i need to install librdkafka 0.9.4 for making it sync with confluent-kafka-go v0.9.4-RC2 ?
Any clue on this issue ?
from confluent-kafka-go.
@edenhill thank you very much for your reply. I did the same things previously that you have described in your comment. But still the same error after cleaning up every thing and re installing librdkafka from source then doing
$ go get -u github.com/confluentinc/confluent-kafka-go/kafka
Its showing following error again:
pkg-config --cflags rdkafka Package rdkafka was not found in the pkg-config search path. Perhaps you should add the directory containing "rdkafka.pc" to the PKG_CONFIG_PATH environment variable No package "rdkafka" found pkg-config: exit status 1
If i install librdkafka from rpm then importing kafka using
$ go get -u github.com/confluentinc/confluent-kafka-go/kafka
Its showing following error:
02:10 PM Tue Jun 06$ go get -u github.com/confluentinc/confluent-kafka-go/kafka github.com/confluentinc/confluent-kafka-go/kafka /tmp/go-build990109538/github.com/confluentinc/confluent-kafka-go/kafka/_obj/event.cgo2.o: In function _cgo_2117b3ae5bb7_Cfunc_rd_kafka_event_stats: confluentinc/confluent-kafka-go/kafka/event.go:127: undefined reference to rd_kafka_event_stats collect2: error: ld returned 1 exit status
I cleaned up everything and started again both way its showing the same error.
from confluent-kafka-go.
Sorry for the delay.
The first problem seems to be an installation path problem, librdkafka is installed into a location where pkg-config will not look. You can probably set up PKG_CONFIG_PATH to point to the directory containing the installed rdkafka.pc file.
The latter problem is that the RPM package is too old, you should use the Confluent YUM repositories to make sure you have the latest version of librdkafka installed:
http://docs.confluent.io/current/installation.html#rpm-packages-via-yum
I would go for this second approach.
from confluent-kafka-go.
@edenhill
I'm sorry to bother you
The program reported an error
/tmp/go-build129217119/command-line-arguments/_obj/exe/producer_example: error while loading shared libraries: librdkafka.so.1: cannot open shared object file: No such file or directory
how can i fix it?
thank u.
from confluent-kafka-go.
hello,
kind of having the same problem, but on windows with visual studio code
# pkg-config --cflags rdkafka
pkg-config: exec: "pkg-config": executable file not found in %PATH%
Process exiting with code: 2
i dont have pkg-config on windows
@edenhill you mention two options
So you have two options:
use the latest release of the Go client (v0.9.4) with a recent release of librdkafka (>=v0.9.4)
remove any installed versions of librdkafka, clone librdkafka master and build and install it.
how can i use the first one ? if i want the latest librdkafka release from
https://github.com/edenhill/librdkafka/releases/tag/v0.11.0 they only provide sources which i have to compile right ?
can i use microsoft windows nuget ?! with go ? ..
thanks
from confluent-kafka-go.
Hi @edenhill ,
Sorry to bother you.
Actually, couple of years ago, you helped me figure out an issue of kafka php client. :)
Recently I had a try on kafka golang client, after installing librdkafka by apt-get install librdkafka, I came across the error:
2017/12/11 07:51:34 ERROR ▶ 0005 Failed to build the application: # github.com/confluentinc/confluent-kafka-go/kafka
../github.com/confluentinc/confluent-kafka-go/kafka/00version.go:45:2: error: #error "confluent-kafka-go requires librdkafka v0.11.0 or later. Install the latest version of librdkafka from the Confluent repositories, see http://docs.confluent.io/current/installation.html"
Then I build it from source, another error occurred:
2017/12/11 08:07:06 ERROR ▶ 0005 Failed to build the application: # github.com/confluentinc/confluent-kafka-go/kafka
/tmp/go-build337827339/github.com/confluentinc/confluent-kafka-go/kafka/_obj/consumer.cgo2.o: In function _cgo_dbdabba0f90b_Cfunc_rd_kafka_offsets_store': /tmp/go-build/github.com/confluentinc/confluent-kafka-go/kafka/_obj/cgo-gcc-prolog:306: undefined reference to
rd_kafka_offsets_store'
/tmp/go-build337827339/github.com/confluentinc/confluent-kafka-go/kafka/_obj/event.cgo2.o: In function _cgo_dbdabba0f90b_Cfunc_rd_kafka_event_stats': /tmp/go-build/github.com/confluentinc/confluent-kafka-go/kafka/_obj/cgo-gcc-prolog:98: undefined reference to
rd_kafka_event_stats'
/tmp/go-build337827339/github.com/confluentinc/confluent-kafka-go/kafka/_obj/offset.cgo2.o: In function _cgo_dbdabba0f90b_Cfunc_rd_kafka_offsets_for_times': /tmp/go-build/github.com/confluentinc/confluent-kafka-go/kafka/_obj/cgo-gcc-prolog:61: undefined reference to
rd_kafka_offsets_for_times'
/tmp/go-build337827339/github.com/confluentinc/confluent-kafka-go/kafka/_obj/producer.cgo2.o: In function do_produce': ../github.com/confluentinc/confluent-kafka-go/kafka/producer.go:41: undefined reference to
rd_kafka_producev'
collect2: error: ld returned 1 exit status
File rdkafka++.pc and rdkafka.pc exist under folder /usr/lib/pkgconfig, and export PKG_CONFIG_PATH=/usr/lib/pkgconfig was executed.
Could you help to give me some advice? Not sure what went wrong.
Thanks a lot!
from confluent-kafka-go.
@edenhill
Thank you so much for your help.
It works!
from confluent-kafka-go.
@lisadeng Glad to hear!
from confluent-kafka-go.
Related Issues (20)
- Feature Request: Support to specify `RecordNameStrategy`
- Subscribed topic not available even though it's already created HOT 8
- Failed to build confluent-kafka-go on arm64 machine. HOT 2
- golang confluent kafka producer client running into error - Failed to create thread: Resource temporarily unavailable (11) HOT 1
- Automatic offset commit led to the loss of some messages HOT 2
- delivery.report.only.error check does not work in case type of the value is not boolean
- Async Producer Panic HOT 1
- Is the producer a tcp keepalive connection
- Signal: bus error when running on macos sonoma 14.4.1 HOT 1
- allow us to unsubscribe from a topic more easily HOT 4
- Producer messages stuck flushing even though there weren't any writes. HOT 4
- Consumers unable to join group until group leader is restarted HOT 3
- Newly added consumers with different version of library in group are not consuming from subscribed topics.
- Schema reference for JSON schemas HOT 1
- consumer recived repeat message after crash
- Does confluent-kafka-go producer PartitionAny is round robin produce message to kafka? HOT 1
- Can we set krb5.conf path for kinit?
- Duplicate messages consumed during rebalancing HOT 2
- Schema Registry misleading error message when unauthorized
- Segmentation violation Error with confluent kafka GO sdk v2 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from confluent-kafka-go.