3.0 - Cannot get past iterator.hasNext about jruby-kafka

[joekiller]

hasNext is a blocking call so it would appear locked without messages until
the consumer.timeout.ms hits (default is -1 so blocking). Did this just
happen when you upgraded? Have you changed the number of topic partitions?
Got a test to replicate the issue?

What Java are you running? Jruby version, etc. Also check Kafka 0.9.0
issues:
https://issues.apache.org/jira/browse/KAFKA/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel

Maybe this?
https://issues.apache.org/jira/plugins/servlet/mobile#issue/KAFKA-3172
On Feb 16, 2016 10:40 AM, "geojr" [email protected] wrote:

In 3.0, it appears the process gets locked when calling iterator.hasNext
and I can no longer consume messages.

—
Reply to this email directly or view it on GitHub
#46.

Yeah, I definitely did not give you a lot of info to go on. Sorry about that.

So I can confirm that on other kafka clients, messages are flowing so that's not the problem. When I downgrade the gem to 2.0 with the exact same configuration, I get messages and everything works as normal.

Java = OpenJDK 1.8.0_66 / Ubuntu 15.10
Kafka 0.8.20
Is 3.0 supposed to be Kafka 0.9 compatible only?

[joekiller]

3.0 is a 0.9.0 build but it's supposed to be backwards compatible. If you
want it to work now, stick with the 2.X release as it is 0.8.2.2 as well.

If you can write a test that creates the problem I could take it from there.

What's your broker running for Java/Kafka? and same questions for the
producer.
On Feb 16, 2016 1:36 PM, "geojr" [email protected] wrote:

Yeah, I definitely did not give you a lot of info to go on. Sorry about
that.

So I can confirm that on other kafka clients, messages are flowing so
that's not the problem. When I downgrade the gem to 2.0 with the exact same
configuration, I get messages and everything works as normal.

Java = OpenJDK 1.8.0_66 / Ubuntu 15.10
Kafka 0.8.20
Is 3.0 supposed to be Kafka 0.9 compatible only?

—
Reply to this email directly or view it on GitHub
#46 (comment)
.

Honestly, it's the sample consumer code in your readme.md that's not
working (with the exception of me using a topic that is local to us)

A coworker advised me his consumer was working which is how I figured out
that under 2.0 things were working fine but 3.0 it was not.

The broker is running 0.8.1 but with Java 7.
Edit: Producer is running the same as the broker as it turns out.

[joekiller]

I'll see if I can replicate but if you can make a fork/PR with an example test that blows up it'll help the most :). I'm guessing it has something to do with Java 7/0.8.1 and the 0.9.0 Java 8 client.

[joekiller]

I tested this and got the same problem. I observed some zookeeper client
errors. Will dog deeper.
On Feb 16, 2016 11:54 AM, "geojr" [email protected] wrote:

Honestly, it's the sample consumer code in your readme.md that's not
working (with the exception of me using a topic that is local to us)

A coworker advised me his consumer was working which is how I figured out
that under 2.0 things were working fine but 3.0 it was not.

The broker is running 0.8.1 but with Java 7.

On Tue, Feb 16, 2016 at 1:49 PM, Joseph Lawson [email protected]
wrote:

3.0 is a 0.9.0 build but it's supposed to be backwards compatible. If you
want it to work now, stick with the 2.X release as it is 0.8.2.2 as well.

If you can write a test that creates the problem I could take it from
there.

What's your broker running for Java/Kafka? and same questions for the
producer.
On Feb 16, 2016 1:36 PM, "geojr" [email protected] wrote:

Yeah, I definitely did not give you a lot of info to go on. Sorry about
that.

So I can confirm that on other kafka clients, messages are flowing so
that's not the problem. When I downgrade the gem to 2.0 with the exact
same
configuration, I get messages and everything works as normal.

Java = OpenJDK 1.8.0_66 / Ubuntu 15.10
Kafka 0.8.20
Is 3.0 supposed to be Kafka 0.9 compatible only?

—
Reply to this email directly or view it on GitHub
<

https://github.com/joekiller/jruby-kafka/issues/46#issuecomment-184818441>

.

—
Reply to this email directly or view it on GitHub
<
https://github.com/joekiller/jruby-kafka/issues/46#issuecomment-184850259>
.

J.R. Smith
Software Developer

Main 972.546.3878
https://www-staging.geoforce.net/uploads/downloads/esignature.html
Fax [phone]
https://www-staging.geoforce.net/uploads/downloads/esignature.html
Toll Free 888.574.3878
https://www-staging.geoforce.net/uploads/downloads/esignature.html
Geoforce.com http://www.geoforce.com/

*Geoforce *750 Canyon Dr. Suite 140
Coppell, Texas 75019

https://www.facebook.com/geoforceinc https://twitter.com/geoforce
https://www.linkedin.com/company/geoforce-inc
https://instagram.com/geoforce_tracking/
https://oilpro.com/company/152135/geoforce

CONFIDENTIAL - If you are not the intended recipient, please delete this
message without disclosing it.

—
Reply to this email directly or view it on GitHub
#46 (comment)
.

[joekiller]

So I was confused. The 0.9.X consumer and producers cannot talk to 0.8 brokers. A 0.9.X broker can be configured to talk 0.8 bit not vice versa. So use the 2.X stuff.

Ahh I was not aware of that either. I was under the impression that it was
all backwards compatible. Good to know. Thank you so much for your help and
response time. 2.x has been working well in my tests.
On Feb 18, 2016 9:50 PM, "Joseph Lawson" [email protected] wrote:

So I was confused. The 0.9.X consumer and producers cannot talk to 0.8
brokers. A 0.9.X broker can be configured to talk 0.8 bit not vice versa.
So use the 2.X stuff.

—
Reply to this email directly or view it on GitHub
#46 (comment)
.

[joekiller]

Good deal. I just lol'd at most of my responses. They all seem to be victim
of autocorrect.
On Feb 18, 2016 7:59 PM, "geojr" [email protected] wrote:

Ahh I was not aware of that either. I was under the impression that it was
all backwards compatible. Good to know. Thank you so much for your help and
response time. 2.x has been working well in my tests.
On Feb 18, 2016 9:50 PM, "Joseph Lawson" [email protected] wrote:

So I was confused. The 0.9.X consumer and producers cannot talk to 0.8
brokers. A 0.9.X broker can be configured to talk 0.8 bit not vice versa.
So use the 2.X stuff.

—
Reply to this email directly or view it on GitHub
<
https://github.com/joekiller/jruby-kafka/issues/46#issuecomment-186040140>
.

—
Reply to this email directly or view it on GitHub
#46 (comment)
.