extension to read Kafka Avro Serializer messages using Kafka Avro Decoder about jruby-kafka

[talevy]

After looking further into this... It would be possible to utilize the existing java implementation of the Serializer/Deserializer for KafkaAvro described here after a slight modification to jruby-kafka's create_consumer_config function in group.rb. Where it would be possible to add custom properties like schema.registry.url for configuring the serializer classes appropriately. Once this is set, Logstash can update to expose this as well.

https://github.com/joekiller/jruby-kafka/blob/master/lib/jruby-kafka/group.rb#L221

looks like Logstash can already start exposing this for the producer:
https://github.com/joekiller/jruby-kafka/blob/master/lib/jruby-kafka/producer.rb#L98

[Kargreat]

@tal: Exactly! we need to make few configuration changes to the logstash-Kafka-input plugin

[Kargreat]

Hey, wanted to quickly check if you are working at enhancing it?

[talevy]

@Kargreat unfortunately, I do not have immediate bandwidth to do so. Just wanted to follow up so that we have possible action plans once time is made available. maybe @joekiller has some suggestions.

[joekiller]

No bandwidth here either but a passing PR would really help me push a new
version out :)
On Jul 8, 2015 3:36 PM, "Tal Levy" [email protected] wrote:

@Kargreat https://github.com/Kargreat unfortunately, I do not have
immediate bandwidth to do so. Just wanted to follow up so that we have
possible action plans once time is made available. maybe @joekiller
https://github.com/joekiller has some suggestions.

—
Reply to this email directly or view it on GitHub
#29 (comment)
.

[Kargreat]

Sure!

Sent from my iPhone

On Jul 9, 2015, at 6:24 PM, Joseph Lawson [email protected] wrote:

No bandwidth here either but a passing PR would really help me push a new
version out :)
On Jul 8, 2015 3:36 PM, "Tal Levy" [email protected] wrote:

@Kargreat https://github.com/Kargreat unfortunately, I do not have
immediate bandwidth to do so. Just wanted to follow up so that we have
possible action plans once time is made available. maybe @joekiller
https://github.com/joekiller has some suggestions.

—
Reply to this email directly or view it on GitHub
#29 (comment)
.

—
Reply to this email directly or view it on GitHub.

[joekiller]

fyi @Kargreat a new version of jruby-kafka is out. It doesn't implement this request but if you have a fork, please check out the changes.

[joekiller]

Finally looked at this some. I think it'll be easiest to just include Avro by default. Related to logstash-plugins/logstash-input-kafka#37 (comment)