Changing ports about kafka-stack-docker-compose

[jeffijoe]

Running kafkacat to list the broker meta, I get:

Metadata for all topics (from broker -1: localhost:20031/bootstrap):
 3 brokers:
  broker 2 at kafka2:9093
  broker 1 at kafka1:9092
  broker 3 at kafka3:9094

Which should have been 10.0.1.9:20031 and so on. If I use the same host and container port, then the IP is displayed correctly, and everything works.

Is it possible to change the host ports without breaking things? 😄

[simplesteph]

yes. Updated README

Q: Can I change the zookeeper ports?

A: yes. Say you want to change zoo1 port to 12181 (only relevant lines are shown):

  zoo1:
    ports:
      - "12181:12181"
    environment:
        ZOO_PORT: 12181
        
  kafka1:
    environment:
      KAFKA_ZOOKEEPER_CONNECT: "zoo1:12181"

Q: Can I change the Kafka ports?

A: yes. Say you want to change kafka1 port to 12345 (only relevant lines are shown). Note only LISTENER_DOCKER_EXTERNAL changes:

  kafka1:
    image: confluentinc/cp-kafka:5.0.0
    hostname: kafka1
    ports:
      - "12345:12345"
    environment:
      KAFKA_ADVERTISED_LISTENERS: LISTENER_DOCKER_INTERNAL://kafka1:19092,LISTENER_DOCKER_EXTERNAL://${DOCKER_HOST_IP:-127.0.0.1}:12345

[jeffijoe]

Thanks!

Curious, why does host and container port have to be the same?

[simplesteph]

Have a look through the blog at the top of the readme

[jeffijoe]

I read the entire thing 😄 I still don't understand why the ports need to be the same in order for it to advertise the correct host and port when connecting externally?

[simplesteph]

1. client connects to Kafka on port A
2. Kafka responds and says btw I run on port B and you should use B
3. client tries to connect to Kafka on port B

If A!=B then things will be problematic
This is due to how Kafka implements broker discovery. Each broker knows best how to be connected to it and clients have to use what brokers advertise

[jeffijoe]

I can't find any documentation regarding all listeners requiring the use of the same port though? Do you think it's a Kafka bug?