Comments (4)
Connection errors are treated as temporary errors and librdkafka will retry connecting to its known brokers indefinately.
Given that the application is configured with a correct broker list it is advised for an application not to take any action when a broker goes down since it will most likely come back up sooner or later, or another broker will assume leadership of the given partitions, so the library and application will eventually pick up where it left off.
Instead you should try to define your application-specific constraints in time:
- how long is it okay for a consumer application not to see any new messages?
- how long may the producer try to deliver a message for the application? (
message.timeout.ms
)
And let the underlying client operate without interruption within those bounds, abstracting your application from Kafka specific know-how.
Anyway, if you're not buying into this, or maybe just curious, you can look for the kafka.Error event.
You will be interested in the ErrTransport and ErrAllBrokersDown events.
from confluent-kafka-go.
@mcandre What if the broker is not down but the client is unable to connect due to a misconfiguration of the broker address. Unless we get a failure event it's not possible for us to know about the problem.
from confluent-kafka-go.
Thanks for the detailed explanation, that's a good way to treat these errors.
from confluent-kafka-go.
librdkafka will retry connecting to its known brokers indefinately.
Hi @edenhill , wondering is there a way to config it? Like how often to retry connecting to broker?
from confluent-kafka-go.
Related Issues (20)
- Extremely slow protobuf deserialization and serialization for schema registry HOT 5
- malformed DWARF TagVariable entry HOT 5
- [Knowledge gathering question] How does librdkafka creates and sends batches over the network to the broker? HOT 2
- Do we really need the `default case` in the consumer for loop
- Can't statically build binary with vendored library on arm64 musl HOT 2
- Multiple schema registry URLs does not seem to be supported
- Serialisation poor performance due to a lack of schema caching HOT 1
- Work with go gin error HOT 5
- Build and run binary return code 137 HOT 3
- What's the difference between `producer.Events()` chan and `deliverChan` HOT 2
- Add error handling for SubscribeTopics function call in consumer_example.go
- High memory usage with the `kafka.consumer`
- Increased memory usage with GetMetadata call
- Build fails on arm64 architecture HOT 1
- Feature Request: Support to specify `RecordNameStrategy`
- Subscribed topic not available even though it's already created HOT 7
- Failed to build confluent-kafka-go on arm64 machine. HOT 2
- golang confluent kafka producer client running into error - Failed to create thread: Resource temporarily unavailable (11) HOT 1
- Automatic offset commit led to the loss of some messages HOT 2
- delivery.report.only.error check does not work in case type of the value is not boolean
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from confluent-kafka-go.