cyrusofeden / csv.js Goto Github PK

Is there a demo page like http://papaparse.com/demo ?

1. I defined the path for require in the config.
2. require(['csv'], function (csv) { /* csv = undefined */ })
3. Added csv to the shim object, made it export 'CSV' as it is in the source code..
4. Require still not recognising csv for some reason

Its really strange, has CSV.js been tried with require.js?

Thanks!

I find this little lib very useful, but I've stumbled upon a strange failure, happening sometimes (not always) when null values are involved.
A sample reproduction on Chrome DevTools console:

> a=data[12]
["2015-07-15 00:00:00.000", 711, "ihstelekom", null, null]
> b=data[13]
[null, null, "sifcepsa", 724, "2015-07-16 12:00:00.000"]
> new CSV([a]).encode()
""2015-07-15 00:00:00.000",711,"ihstelekom",,"
> new CSV([b]).encode()
",,"sifcepsa",724,"2015-07-16 12:00:00.000""
> new CSV([a,b]).encode()
""2015-07-15 00:00:00.000",711,"ihstelekom",,<new-line>"null",,"sifcepsa",,"

As it can be seen, encoding a or b separately works ok. But encoding both of them fails (there's a spurious "null" value in b's encoded value, instead of the expected "nothing".
I did take a quick glance but didn't find the culprit. Could you help, please?

Hello,

There is no trim() function when code parse fields. It's possible to add it, or to add it via options ?

4180 says in section 2 claim 6 that if there's a quote or a newline in the value, the field needs to be quoted

   6.  Fields containing line breaks (CRLF), double quotes, and commas
       should be enclosed in double-quotes.  For example:

       "aaa","b CRLF
       bb","ccc" CRLF

Here are four failed cases: embedded \n, embedded \r, embedded \r\n, and embedded "

Johns-MacBook-Pro:scratch john$ node
> var csv = require('comma-separated-values');
undefined
> csv.encode([ [1,2,3,4], ["A\rB","C\nD","E\r\nF","G\"H"] ]);
'1,2,3,4\r\nA\rB,C\nD,E\r\nF,G"H'

An example is Twitter's snowflake IDs. They are 64 bit integers, I believe, which causes truncation in Excel and Calc when loaded in as numbers.

An example CSV:

"Network","ID"
"twitter",479386071891505153

You'll notice that the number loses the precision and becomes 479386071891505000 or similar.

A fix is to quote the numbers so they become strings and then pull them in as strings in Excel/Calc.

Is there any way that we can get an option for this when encoding?

Maybe the heading has to be escaped.

new CSV([{a:1, b:2}, {a:1, b:2, c:3}], {
  header: ['a','b','c'],
}).encode();

will return:

"a","b","c"
1,2
1,2

value "c" will not render

same apply if first value is null:

new CSV([{a:1, b:2, c: null}, {a:1, b:2, c:3}], {
  header: ['a','b','c'],
}).encode();

‘var Phonebook = new CSV(listStr,
{
header : true
}
).forEach(function (options)
{

                Arraytmp.push(options["name"]);
				console.log("~~~~~~~~" + options["name"]);
                Arraytmp.push(options["office_number"]);
				console.log("%%%%%%%%" + options["office_number"]);
        }
        );’例如原数组为：da,34\r\n
                                    23,rf\r\n
                            rf不会被解析

I don't know why this commit is a bug should be fix, so v3.6.4 is a breaking change and break my code, and let me busy this morning 😢

If ignore empty lines, we can’t one-to-one align the row index between parsing code and csv editor. For example, I generate a csv let use to edit, and use a empty line separate each section for for pretty look.

"title"
"desc"
"command"
"" <- as a separator
"User ID"
"integer"
"uid"
"1234"

When parse back the csv, if ignore the empty lines, I need to shift the row index, it is hard to debug because the rows which return by parse(), the array index is different from csv editor.

So, I request a option the let me setting this behavior.

See standard here.

In file csv.js line 260, you should also check for double quotes. Better to do it in a single replace with regexp. See this answer for code example http://stackoverflow.com/a/24922761/139361

If the file contains empty cell in any row with heading, then that cell content is converted to number 0

Hi,

Currently, the parser reads empty lines and sets default values as follows:

If the first column is a number then it gets a 0, otherwise if it is a string then it gets "". For the other columns numbers get NaN and strings get undefined.

var data = '\
year,age,status,sex,population\r\n\
1850,20,0,1,1017281\r\n
\r\n\
1850,20,0,2,1003841\r\n\
';
new CSV(data, { header: true }).parse();
/*
Returns:
[
  { year: 1850, age: 20, status: 0, sex: 1, population: 1017281 },
  { year: 0, age: NaN, status: NaN, sex: NaN, population: NaN },
  { year: 1850, age: 20, status: 0, sex: 2, population: 1003841 }
]

var data2 = '\
name,age\r\n\
bob,20\r\n
\r\n\
Chris,20\r\n\
';
new CSV(data2, { header: true }).parse();
/*
Returns:
[
  { name: "bob", age: 20 },
  { name: "", age: NaN },
  { name: "Chris", age: 20 }
]

*/

My initial assumption/hope was for the empty lines to simply be ignored.

I would love to hear your thoughts on that.

Cheers,
Salim

If my string only contains newlines and no carriage returns: "ab,cd\nef,gh" i will get {ab,c},{ef,g}

I have need for reading CSV files from multiple sources, some of which have multi-row headers (only one of which has actual column headers). It would be nice to be able to tell the parser to ignore the first X rows of a file and/or look for headers at a set row number.

I'm trying to parse the following csv but i get an extra quote at the end of the first field

Date Download,Image ID,Filename,Original Filename,License,Earnings,Tax
"2016-11-15 05:01:51","62944391","160700127","","240","24.0800","0.00"
"2016-11-15 05:01:57","62944391","160700127","","1080","108.3600","0.00"

I'm using and i get the date with an extra quote at the end
var csv = new Csv(data, { header: true });

i'm doing something wrong?

FYI, I get the error:

Object doesn't support property or method 'call'

at this line:

return Array.isArray?Array.isArray(t):"[object Array]"==toString.call(t)

Maybe put somewhere in the README a minimum browser support.

Cheers

So you can do fs.createReadStream().pipe(new Csv).pipe(outStream)

Hi, i found a case that when the last column does not have a header, it can not be parsed.

The codes look like this

const data = "user_id,name\r\n1234,xxx,extra\r\n";
console.log(new CSV(data).parse());

// =>
[
    [user_id, name],
    [1234, xxx],
]

Actually i want to get a result like this

[
    [user_id, name, ''],
    [1234,xxx,extra],
]

Besides, i found that if there is a comma at the end of the header, the last column can be parsed, but there seem to be some other problems

const data = "user_id,name,\r\n1234,xxx,extra\r\n";
console.log(new CSV(data).parse());

// =>
[
    [user_id, name,0],
    [1234, xxx,NaN],
]

https://codesandbox.io/s/determined-bell-4bb6r?file=/src/App.js

Is there any way to solve it？

Hy, In the unminified version (at least) I have:
return data instanceof Array ? this.encode(stream) : this.parse(stream);
but I think it should be
return this.data instanceof Array ? this.encode(stream) : this.parse(stream);

If a cell has two lines, and the first line tail with two spaces, for example

1,"test  
abc"
2,"test  
abc"
3,"test  
abc"

There are two space after test, What expect to get is

 [["1", "test \nabc"], ["2", "test \nabc"], ["3", "test \nabc"]]

It work fine in version 3.6.1 csv.js.

But when using csv.min.js, I got

 [["1,"test", "", ""], ["abc"", undefined, undefined], ["2,"test", "", ""], ["abc"", undefined, undefined], ["3,"test", "", ""], ["abc"", undefined, undefined]]

It doesn't make sense. And I use uglify-js create a min.js myself, it didn't have this bug.

I added some auto detections for line endings and value delimiters in a chrome app I built. Something worth adding to the library? These functions can fail, but should work most of the time. Maybe behind boolean options like autoDetectLine and autoDetectDelimiter?

/**
 * Detects and returns used line ending
 * @param data The raw data string
 * @return '\r\n', '\r' or '\n'
 */
function detectLineEnding(data) {
    if (Utils.strCount('\r\n', data) > 0) {
        return '\r\n'; // Windows
    } else if (Utils.strCount('\r', data) > 0) {
        return '\r'; // Old mac
    } else {
        return '\n'; // Unix/Mac
    }
}

/**
 * Detects and returns used value delimiter
 * @param data The raw data string
 * @param line The line ending character
 * @return ';', ',' or '\t'
 */
function detectValueDelimiter(data, line) {
    var arr = data.split(line);
    var chars = [';', '\t'];
    for(var i = 0; i < chars.length; i++) {
        if(testChar(chars[i]))
            return chars[i];
    }
    return ',';

    function testChar(c) {
        var first = Utils.strCount(c, arr[0]);
        if (first === 0)
            return false;
        for (var i = 1; i < arr.length; i++) {
            var count = Utils.strCount(c, arr[i]);
            if (count !== first)
                return false;
        }
        return true;
    }
}

/**
 * Sample implementation of counting substring occurrences.
 */
Utils.strCount = function (needle, haystack) {
    return (haystack.match(new RegExp(needle, 'g')) || []).length
}

See http://tools.ietf.org/html/rfc4180 for an example of quoted fields, newlines in quotes, and escaped quotes.

Any help?

node_modules/comma-separated-values/csv.js:58
cc = function(text, i) { return text.charCodeAt(i); };

has no method 'charCodeAt'
at cc (/vagrant/node_modules/comma-separated-values/csv.js:58:46)
at new CSV (/vagrant/node_modules/comma-separated-values/csv.js:59:36)
at ReadStream. (/vagrant/util/tula_importer.coffee:13:3, :16:14)
at ReadStream.EventEmitter.emit (events.js:95:17)
at ReadStream. (stream_readable.js:745:14)
at ReadStream.EventEmitter.emit (events.js:92:17)
at emitReadable (_stream_readable.js:407:10)
at emitReadable (_stream_readable.js:403:5)
at readableAddChunk (_stream_readable.js:165:9)
at ReadStream.Readable.push (_stream_readable.js:127:10)
at onread (fs.js:1563:12)
at Object.wrapper as oncomplete

Thanks!

Hi

I think that it is not acceptable to remove tags in the repository, because there are projects which might fail to deploy/run one day with the following error:

bower comma-separated-values#~3.2.0                              ENORESTARGET No tag found that was able to satisfy ~3.2.0

Additional error details:
Available versions: 3.3.0, 2.2.0, 1.2.3
rake aborted!
Command failed with status (1): [cd /home/avsej/prj/foobar && bower install...]

Please, don't do that

I'm getting the following error:

/app/node_modules/comma-separated-values/csv.js:50
    return string.replace(/"/g,'\\"');
                 ^
TypeError: Cannot read property 'replace' of undefined
    at sanitizeString (/app/node_modules/comma-separated-values/csv.js:50:18)

When executing the following code. Everything works fine without the configuration object.

new CSV(fileData, {
    header: [
        'monthNum',
        'projectedLossRateAtMonth',
        'actualLossRateAtMonth',
        'adjustedProjectedLossRate',
        'stressedLossRate',
        'breakEvenLossRate',
        'pim',
        'rating'
    ]
}).parse();

In Interned Explorer 11, I get an Invalid Calling Object error when executing the following code:

var op_val_objects      = new CSV(op_val_string, {lineDelimiter: "\n", header: true}).parse();

Where op_val_string is the following string:

"option,value
Cap,1
Trim,2
Post Caps,4
Top Trim,8
Overlap,16
Good Neighbor,32"

Here's the error:
SCRIPT65535: Invalid calling object
File: csv.js, Line: 13, Column: 5

Here's the referenced line in context:

 function isArray(object) {
    return toString.call(object) == '[object Array]';  // This is line 13
  }

I suspect that this might come from Internet Explorer not supporting "call" on many types of object that the other browsers commonly do. It seems I've seen reports of this kind of trouble with DOM objects; but I'm not sure if the same conditions apply here.

To reproduce, run the following code example from README.md

var data = [[1850, 20, 0, 1, 1017281], [1850, 20, 0, 2, 1003841]];
new CSV(data).encode();

generates the string:

"1850,20,,1,1017281
1850,20,,2,1003841"

Rather than the expected:

"1850,20,0,1,1017281
1850,20,0,2,1003841"

For my own, limited context, I added:

if (value == 0) return "0";

to the top of the stringify function. That fixed it for me. I'm not particularly familiar with this code base, so I doubt that's the right thing to do.