Error Schema are not the same about json-2-csv

[mrodrig]

Thanks for posting this, Herve. I agree that the ordering of the document keys should not cause the schema error that you are receiving. I'm working on fixing this and should have it fixed in the near future (hopefully tomorrow or Friday). Sorry for the inconvenience this may have caused.

[mrodrig]

Thanks again for reporting this. It should be fixed in NPM version 1.1.0 which I just published. If you experience any other problems with the module, please let me know.

[hepr]

Thanks for your fix. All is good and my tests are ok :-)

2015-04-10 14:38 GMT+02:00 Mike Rodrigues [email protected]:

Thanks again for reporting this. It should be fixed in NPM version 1.1.0
which I just published. If you experience any other problems with the
module, please let me know.

—
Reply to this email directly or view it on GitHub
#12 (comment).

[hepr]

Hi,

I reproduce this problem :-(
[ '"_id";"address";"name";"Tag"',
'"_id";"name";"address";"Tag"' ]

I can give you a db if you want.

Thanks for your help

Herve

[mrodrig]

Hi Herve,

Thanks for letting me know. Which version are you using? Could you send me the version number from node_modules/json-2-csv/package.json? I just tried out a simple example using version 1.3.0 and it appears to be working as expected. Here's my test example for reference:

var json2csv = require('json-2-csv');

var docs = [
        {
                '_id': '235235626324342',
                'name': 'Mike',
                'address': 'Test1123',
                'Tag': 'mytag'
        },
        {
                '_id': '456365834905679',
                'address': 'Var123',
                'name': 'Joe Shmoe',
                'Tag': 'notag'
        }
];

json2csv.json2csv(docs, function (err, csv) {
        if (err) { return console.log(err); }
        console.log('success', csv);
});

Thanks,
Mike

[hepr]

Hi Mike,

I'm in 1.3.0 version

With your example it's ok.
I've joined my extract and the error at the last line. cf output.log

Thanks

Herve

2015-06-08 16:50 GMT+02:00 Mike [email protected]:

Hi Herve,

Thanks for letting me know. Which version are you using? Could you send me
the version number from node_modules/json-2-csv/package.json? I just tried
out a simple example using version 1.3.0 and it appears to be working as
expected. Here's my test example for reference:

var json2csv = require('json-2-csv');

var docs = [
{
'_id': '235235626324342',
'name': 'Mike',
'address': 'Test1123',
'Tag': 'mytag'
},
{
'_id': '456365834905679',
'address': 'Var123',
'name': 'Joe Shmoe',
'Tag': 'notag'
}
];

json2csv.json2csv(docs, function (err, csv) {
if (err) { return console.log(err); }
console.log('success', csv);
});

Thanks,
Mike

—
Reply to this email directly or view it on GitHub
#12 (comment).

[mrodrig]

Hmm. Is there any way that you could provide an example so I can take a look? Seems like it would be an issue with lines 37-40 of json-2-csv.js:

var diff = _.difference(firstDocSchema, _.flatten(keyList));
if (!_.isEqual(diff, [])) {
    schemaDifferences++;
}

[hepr]

Sorry this is the link to download an example included in this file :

<removed by @mrodrig>

Thanks

Herve

2015-06-09 1:10 GMT+02:00 Mike [email protected]:

Hmm. Is there any way that you could provide an example so I can take a
look? Seems like it would be an issue with lines 37-40 of json-2-csv.js:

var diff = _.difference(firstDocSchema, .flatten(keyList));
if (!.isEqual(diff, [])) {
schemaDifferences++;
}

—
Reply to this email directly or view it on GitHub
#12 (comment).

[mrodrig]

Hi Herve,

Thanks for the log file. It looks like the reason why it's throwing the 'Schema not the same' error is because a few of the documents towards the top have a 'name' attribute, and others do not. One option to fix this would be to run a loop like this over the data before calling json2csv:
(example using underscore)

// Fetch documents from the database
var documents = ... ;

// Loop over the documents to ensure the schemas are the same
documents = _.map(documents, function (document) {
    // Set the document's name attribute to an empty string if it does not exist
    document.name = document.name ? document.name : '';
    return document;
});

In the future I might look into removing that error and just having the module use null or '' as a default value if the key doesn't exist.

Hope this helps!

Thanks,
Mike

[hepr]

Hi Mike,

thank you for this help.
I'll implement it.

Thanks :-)

Herve

2015-06-09 14:34 GMT+02:00 Mike [email protected]:

Hi Herve,

Thanks for the log file. It looks like the reason why it's throwing the
'Schema not the same' error is because a few of the documents towards the
top have a 'name' attribute, and others do not. One option to fix this
would be to run a loop like this over the data before calling json2csv:
(example using underscore)

// Fetch documents from the database
var documents = ... ;

// Loop over the documents to ensure the schemas are the same
documents = _.map(documents, function (document) {
// Set the document's name attribute to an empty string if it does not exist
document.name = document.name ? document.name : '';
return document;
});

In the future I might look into removing that error and just having the
module use null or '' as a default value if the key doesn't exist.

Hope this helps!

Thanks,
Mike

—
Reply to this email directly or view it on GitHub
#12 (comment).

[hepr]

This is my fix in my code :

var result = docs.map(function (doc) {

                    for (var i = 0, len = fields.length; i < len; i++) {
                        if (typeof newdoc[fields[i]] == 'undefined') {
                            newdoc[fields[i]] = "";
                        }
                    }

                    return newdoc;
                });

It works fine :-)

Thanks

Herve

2015-06-09 14:34 GMT+02:00 Mike [email protected]:

Hi Herve,

Thanks for the log file. It looks like the reason why it's throwing the
'Schema not the same' error is because a few of the documents towards the
top have a 'name' attribute, and others do not. One option to fix this
would be to run a loop like this over the data before calling json2csv:
(example using underscore)

// Fetch documents from the database
var documents = ... ;

// Loop over the documents to ensure the schemas are the same
documents = _.map(documents, function (document) {
// Set the document's name attribute to an empty string if it does not exist
document.name = document.name ? document.name : '';
return document;
});

In the future I might look into removing that error and just having the
module use null or '' as a default value if the key doesn't exist.

Hope this helps!

Thanks,
Mike

—
Reply to this email directly or view it on GitHub
#12 (comment).

[mrodrig]

Great - Glad we were able to solve it!

Thanks,
Mike