Release status of ElephantBird 2.x + LzoPigStorage ? about elephant-bird

[rangadi]

Hi Miguno,

kevinweil's master is the "official" release. This is what we are using at Twitter right now. 2.0.2 is considered stable and is the official release. There will be 2.0.3 sometime today or tomorrow.

Our process so far has been: A all the development patches into kevinweil/eb-dev. The we merge eb-dev into master when we are ready to make a release.

I will take a look at LzoPigStorage.java. I how it is different from LzoTextLoader in functionality (both use LzoTextInputformat for reading).

[dvryaboy]

Master on Kevin's repo is the "official" branch to which we push
releases. A release is basically whatever we are running on Twitter's
internal clusters. The eb-dev branch on Kevin's repo is the
integration branch, into which changes are pulled in from different
contributors, mostly myself and Raghu these days, but there are
occasional others (like the recent work to add "deprecated" api
support for reading Thrift).

As we are finding some corner cases right now, having just upgraded on
Monday to Pig 8 and EB 2.0+, minor version bumps are going to be a bit
more frequent than before.. we will likely push 2.0.3 today. I expect
this will stabilize shortly.

We can certainly pull in Gerrit's LzoPigStorage, he just needs to send
a pull request :).

Happy to hear you are using the stuff!

D

On Wed, Jun 15, 2011 at 2:24 AM, miguno
[email protected]
wrote:

Hi Kevin & all,

I have two quick question:

First, I have seen that Dmitriy merged (kevinweil/elephant-bird@ebaa220c5c728) his Pig-0.8 compatibility branch of EB into Kevin's eb-dev. Kevin's master branch of kevinweil/elephant-bird now contains the "2.0.2 bug fix release" and is just one version-bump commit behind eb-dev. So my question would be: Is EB 2.0.2 considered stable or an official release? Is Kevin's repo the recommended code base or rather Dmitriy's? (I'm a bit confused what development happens in each repo, sorry :-D)

Second, gerritjvv forked Dmitriy's EB repo some time ago (gerritjvv/elephant-bird). He added the class LzoPigStorage.java. This class appears to be a very simple, straight-forward LZO-enabled wrapper for standard PigStorage -- less than 10 lines of effective code. If the implementation is sound, do you think it would make sense to bring LzoPigStorage into your repository?

Many thanks for your work on ElephantBird, it's appreciated!

Best,
Michael

Reply to this email directly or view it on GitHub:
https://github.com/kevinweil/elephant-bird/issues/56

[miguno]

Many thanks for your replies and all the infos, Raghu and Dmitriy!

I have asked Gerrit about said pull request.

Happy to hear you are using the stuff!

Happy to be using what you are sharing! :-)

[gerritjvv]

Hi, I'll make the pull request.
There will be other commits also, just merge in the ones of interest.

[gerritjvv]

OK, I've made the pull request to dvryaboy:master as this is the repo I branched.

[dvryaboy]

We're looking at Gerrit's patch, adding a few things. Closing this ticket in the meantime. Btw 2.0.3 is out as of today.

[rangadi]

sent https://github.com/kevinweil/elephant-bird/pull/62 against keviweil/eb-dev. Thanks gerritjvv.