0-length argument error at keyval about rmr2

[piccolbo]

I'd look into the first error first "syntax error near unexpected token". I'd love to know how this file name was created

/mnt/data/mapred/local/taskTracker/musigma/jobcache/job_201303201047_0032/attempt_201303201047_0032_m_000000_0/work/tmp/RtmpF1qXQp/file708455830e3d c(" new/output/alltags-v33_1/AtRisk/part-00001", " new/output/alltags-v33_1/AtRisk/part-00002")ʼ

After that rmr is probably processing an empty input and the recycling error is more than understandable.
Since it's a hadoop dfs -put call that's failing, unless you deliberately called this external program, what is failing is a to.dfs. Look at its first argument, kv.

[prateekkulkarni4105]

I had a to.dfs in the code and I removed it and I got the same 0-length argument error, without the dfs -put error. I understand what you're talking about when you say that rmr might be processing an empty input, but the 2 if conditions make sure that we don't reach the keyval if the input to it is empty, and we are confused as to why this would happen.

[piccolbo]

That is progress towards a simple, self contained example that would allow
me to reproduce the problem and fix it. It would be nice if we could fix
programs looking at the error logs without access to the source code, but
that's not how it works. I need to have the source code and I need to run
it. If it is long and proprietary you need it to narrow it down to
something I can understand within time and legal bounds. Selected snippets
are seldom helpful. The two conditions guarantee absolutely nothing as
#some processing is omitted. You need to move the conditions right before
the keyval call. You can also add an rmr.str statement as follows

kv = keyval("constant_key",processed_input_data")
rmr.str(kv)
kv

You can also apply other steps described here
https://github.com/RevolutionAnalytics/RHadoop/wiki/Debugging-rmr-programs

The ability to debug your own programs even if they run on top of Hadoop is
invaluable -- even if the bug is in the library, you'll be able to narrow
it down and report with a much higher degree of confidence.

Antonio

On Thu, Mar 21, 2013 at 1:23 PM, Prateek Kulkarni
[email protected]:

I had a to.dfs in the code and I removed it and I got the same 0-length
argument error, without the dfs -put error. I understand what you're
talking about when you say that rmr might be processing an empty input, but
the 2 if conditions make sure that we don't reach the keyval if the input
to it is empty, and we are confused as to why this would happen.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/28#issuecomment-15263076
.

[mayankbansal89]

Hi,

So we tried adding the rmr.str(kv) as you had suggested,

The output if it was as follows

Length of Keys = 1
Length of Values = 3
Error in rmr.recycle(k, v) : Canʼt recycle 0-length argument
Calls: ... keyval.writer -> format -> recycle.keyval -> keyval -> rmr.recycle
Execution halted

I also checked the code for rmr.recycle and it did not make much sense for that error to crop up. Could you please help us out with this.

[piccolbo]

That is not the output of rmr.str. It prints the stack where the call
happened and then the expression passed to it, then its value.

Antonio

On Fri, Mar 22, 2013 at 1:39 AM, mayankbansal89 [email protected]:

Hi,

So we tried adding the rmr.str(kv) as you had suggested,

The output if it was as follows

Length of Keys = 1
Length of Values = 3
Error in rmr.recycle(k, v) : Canʼt recycle 0-length argument
Calls: ... keyval.writer -> format -> recycle.keyval -> keyval ->
rmr.recycle
Execution halted

I also checked the code for rmr.recycle and it did not make much sense for
that error to crop up. Could you help us out with this.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/28#issuecomment-15286129
.

[mayankbansal89]

It should be in the stderr logs ?
If yes, I can't find it. Please let me know if I am looking at the right place.

[piccolbo]

You are correct. This is puzzling.

[mayankbansal89]

Found it, sorry my bad and I had written rmr.str after the return(kv) statement.
It is actually zero, the earlier lengths were part of my cat statement and I was doing a cat of nrow of the object before placing them inside a keyval object.
The output that I get now is
Length of Keys = 1
Length of Values = 3
Dotted pair list of 6
$ : language (function() { load("./rmr-local-env5fe53103ae8e") ...
$ : language rmr2:::map.loop(map = map, keyval.reader = input.reader(), keyval.writer = if (is.null(reduce)) { output.writer() ...
$ : language as.keyval(map(keys(kv), values(kv)))
$ : language is.keyval(x)
$ : language map(keys(kv), values(kv))
$ : language rmr.str(kv)
kv
List of 2
$ key: chr "AR"
$ val:ʼdata.frameʼ: 0 obs. of 3 variables:
..$ i_dmid : num(0)
..$ prob : num(0)
..$ prob_ds: num(0)
Error in rmr.recycle(k, v) : Canʼt recycle 0-length argument
Calls: ... split.keyval -> recycle.keyval -> keyval -> rmr.recycle

The length of keys and length of values is part of my cat statements to the stderr log.
and below it is the rmr.str call.

[piccolbo]

So you see that the value part of the key-value pair has 0 rows and the key
has 1 element. So that is currently not allowed. I can extend the semantics
to accept this case if we find a reasonably general definition, but right
now the acceptable cases are described
herehttps://github.com/RevolutionAnalytics/RHadoop/wiki/Keyval-types-and-combinations.
For now you can either change your program so that this doesn't happen,
thinking it's a bug, or just special case it like if(! nrow(values(kv)) == 0) kv. Does this get you going?

On Fri, Mar 22, 2013 at 9:13 AM, mayankbansal89 [email protected]:

Found it, sorry my bad and I had written rmr.str after the return(kv)
statement.
It is actually zero, the earlier lengths were part of my cat statement and
I was doing a cat of nrow of the object before placing them inside a keyval
object.
The output that I get now is
Length of Keys = 1
Length of Values = 3
Dotted pair list of 6
$ : language (function() { load("./rmr-local-env5fe53103ae8e") ...
$ : language rmr2:::map.loop(map = map, keyval.reader = input.reader(),
keyval.writer = if (is.null(reduce)) { output.writer() ...
$ : language as.keyval(map(keys(kv), values(kv)))
$ : language is.keyval(x)
$ : language map(keys(kv), values(kv))
$ : language rmr.str(kv)
kv
List of 2
$ key: chr "AR"
$ val:ʼdata.frameʼ: 0 obs. of 3 variables:
..$ i_dmid : num(0)
..$ prob : num(0)
..$ prob_ds: num(0)

Error in rmr.recycle(k, v) : Canʼt recycle 0-length argument
Calls: ... split.keyval -> recycle.keyval -> keyval -> rmr.recycle

The length of keys and length of values is part of my cat statements to
the stderr log.
and below it is the rmr.str call.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/28#issuecomment-15305784
.

[mayankbansal89]

It looks like a coding issue only, which I was missing out on.
Sorry for using up your time on this.
Adding your if(!nrow(values(kv))) statement gets me going properly.
Thank you !! :)

[piccolbo]

No problem!