IncompatibleBrokerVersion: Kafka broker does not support the 'CreateTopicsRequest_v0' Kafka protocol. about aiokafka

[trentbitterman]

@davidhuser I had the same problem as you, but I realized I never called the admin_client.start method. Once I added that to my context manager function it started working. Hopefully that helps you too.

@asynccontextmanager
async def get_admin_client():
    admin_client = AIOKafkaAdminClient(
        bootstrap_servers=settings.KAFKA_BOOTSTRAP_SERVERS
    )
    try:
        await admin_client.start()
        yield admin_client
    finally:
        await admin_client.close()

[davidhuser]

works now, thanks @trentbitterman !

[ods]

AIOKafkaAdminClient must be bootstrapped before use, the preferred way is

async with AIOKafkaAdminClient(...) as client:
    ...

[davidhuser]

I have the same issue as OP. If I use the suggested context manager I get

TypeError: 'AIOKafkaAdminClient' object does not support the asynchronous context manager protocol

so I wrap it into a asynccontextmanager:

from contextlib import asynccontextmanager
from aiokafka.admin import AIOKafkaAdminClient, NewTopic

@asynccontextmanager
async def get_admin_client():
    admin_client = AIOKafkaAdminClient(
        bootstrap_servers=settings.KAFKA_BOOTSTRAP_SERVERS
    )
    try:
        yield admin_client
    finally:
        await admin_client.close()


async def create_topic():
    async with get_admin_client() as admin_client:
        try:
            await admin_client.create_topics(
                [
                    NewTopic(
                        name='mytopic',
                        num_partitions=1,
                        replication_factor=1,
                    )
                ]
            )
        except Exception as e:
            print("oh no")

Still get the error as OP:

2024-07-16 15:34:23.329 | ERROR    | kafka_utils:create_topic:70 - Failed to create topic 'costs': IncompatibleBrokerVersion: Kafka broker does not support the 'CreateTopicsRequest_v0' Kafka protocol.

How can we create topics with aiokafka?

---

Version 0.11.0