Coder Social home page Coder Social logo

jeygeethan / docker-kafka-cluster Goto Github PK

View Code? Open in Web Editor NEW
11.0 11.0 12.0 7 KB

A clusterized docker setup for the kafka. It requires a zookeeper cluster and I have provided the same in another repo. Please read readme.md

License: MIT License

Shell 100.00%

docker-kafka-cluster's People

Contributors

jeygeethan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

docker-kafka-cluster's Issues

BROKER_ID parameter not recognized?

I can't run the second node, seems the BROKER_ID is not correctly recognized.

On the command line I set it to 1, within Docker it's suddenly 0.

#docker run \
>   --name=kafka2 \
>   -p 9092:9092 \
>   -e KAFKA_HOST=server2.server.de \
>   -e KAFKA_PORT=9092 \
>   -e ZOOKEEPER_CONNECT=server1.server.de:2181,server2.server.de:2181 \
>   -e BROKER_ID=1 \
>   -d jeygeethan/kafka-cluster
9f7137bbf5269c1810308d0f28bfdf33a83673e250b2947877daf5f4e05d486c

# docker logs -f kafka2
advertised host: server2.server.de
advertised port: 9092
zookeeper connect: server1.server.de.de:2181,server2.server.de.de:2181
[2017-04-05 21:53:55,272] INFO KafkaConfig values:
	advertised.host.name = server2.server.de
	...
	broker.id = 0
	...
 (kafka.server.KafkaConfig)
[2017-04-05 21:53:55,309] INFO starting (kafka.server.KafkaServer)
[2017-04-05 21:53:55,311] INFO Connecting to zookeeper on server1.server.de:2181,server2.server.de:2181 (kafka.server.KafkaServer)
[2017-04-05 21:53:55,318] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2017-04-05 21:53:55,321] INFO Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT (org.apache.zookeeper.ZooKeeper)
[2017-04-05 21:53:55,321] INFO Client environment:host.name=9f7137bbf526 (org.apache.zookeeper.ZooKeeper)
[2017-04-05 21:53:55,321] INFO Client environment:java.version=1.8.0_102 (org.apache.zookeeper.ZooKeeper)
[2017-04-05 21:53:55,321] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2017-04-05 21:53:55,321] INFO Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre (org.apache.zookeeper.ZooKeeper)
...
[2017-04-05 21:53:55,686] INFO Creating /brokers/ids/0 (is it secure? false) (kafka.utils.ZKCheckedEphemeral)
[2017-04-05 21:53:55,694] INFO Result of znode creation is: NODEEXISTS (kafka.utils.ZKCheckedEphemeral)
[2017-04-05 21:53:55,695] FATAL [Kafka Server 0], Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.lang.RuntimeException: A broker is already registered on the path /brokers/ids/0. This probably indicates that you either have configured a brokerid that is already in use, or else you have shutdown this broker and restarted it faster than the zookeeper timeout so it appears to be re-registering.
	at kafka.utils.ZkUtils.registerBrokerInZk(ZkUtils.scala:295)
	at kafka.utils.ZkUtils.registerBrokerInZk(ZkUtils.scala:281)
	at kafka.server.KafkaHealthcheck.register(KafkaHealthcheck.scala:64)
	at kafka.server.KafkaHealthcheck.startup(KafkaHealthcheck.scala:45)
	at kafka.server.KafkaServer.startup(KafkaServer.scala:231)
	at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
	at kafka.Kafka$.main(Kafka.scala:67)
	at kafka.Kafka.main(Kafka.scala)
[2017-04-05 21:53:55,696] INFO [Kafka Server 0], shutting down (kafka.server.KafkaServer)
[2017-04-05 21:53:55,697] INFO [Socket Server on Broker 0], Shutting down (kafka.network.SocketServer)
[2017-04-05 21:53:55,704] INFO [Socket Server on Broker 0], Shutdown completed (kafka.network.SocketServer)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.