How to send binary data by cppkafka library? about cppkafka

[mfontanini]

Could you come up with a short reproducible code I can use to test this? I've been using cppkafka to send protobuf messages (they serialize as binary data) so this definitely works.

Are the messages successfully sent? Are you waiting for the messages to be sent or are you closing your producer right after sending? You may want to wait for your produced messages to be acked.

[feifeidadi]

Hi Matias Fontanini,

Thanks a lot for your reply.
You are right. It sends the binary buffer successfully after waiting for a moment after producer.produce(builder);.

Best regards,
Mike

[mfontanini]

That's great. Note that you can use the BufferedProducer class which, besides allowing to buffer and produce message, it also lets you wait until they're acked. It's almost a drop-in replacement for Producer, see this test where it's used. Then just call produer.flush() and that'll wait until messages are acked by the kafka broker(s).

[feifeidadi]

Hi Matias Fontanini,

Thank you for your quick response :).

Does it mean that the BufferedProducer class works in asynchronous mode by default?
In case of producer.flush() is not called, the delivery status needs to be checked in the delivery report callback, right?

Best regards,
Mike

[mfontanini]

It can actually work both ways: if you call BufferedProducer::produce, the message is produced as usual (asynchronously). If you call BufferedProducer::add_message, then the message is queued internally and only actually sent to kafka when calling flush. That call will block until all messages are actually acked.

[feifeidadi]

OK, it's clear enough. Thanks a lot for your explanation Matias Fontanini.

Best regards & happy weekend,
Mike