This csv file crashes the parser about csv-parser

[vincentlaucsb]

Your header row has 7 commas (8 columns) while your data rows have 8 commas (9 columns).

You'll either want to strip out the extra commas before sending it through the CSVReader or create your own subclass of CSVReader which overrides bad_row_handler such as here: https://github.com/vincentlaucsb/csv-parser/blob/master/include/internal/csv_reader.cpp#L34

[bangusi]

OK. But I would have though the library should not silently fail when encountering such files.
BTW I did not create this file, it was generated by some bank so it is a legitimate case of csv files one may encounter in real life. As mentioned above office applications such as LibreOffice are able to handle it.

[vincentlaucsb]

I agree there should probably be a warning when the majority of rows are rejected.

However, I don't think it's a reasonably formatted CSV file. I don't know why you'd add an extra comma to your data rows when there's not one in your header row. I've heard of it before, but it's not present in the majority of CSV files.

This CSV parser is also capable of handling it, you just need to tell it that it's a weird file. I think the easiest way to do it is to just turn the header row off, use auto-detection, and use bad_row_handler to capture the header row. I built this parser mainly to load CSV files into databases or run data analysis on them, which is why it's a bit anal about the column count.

[bangusi]

Another suggestion would be to provide a configuration option that allows the user to ignore extra rows. I have seen this feature in other file import (into database ) utilities.
It is not practical to modify the library for every weird case.

[bangusi]

I just noticed that this issue #66 is supposed to address the same problem that I reported in #77.
Looking forward to the enhancement.
o called "Malformed rows" are actually not rare in csv files.

[vincentlaucsb]

The new version should handle variable length-rows in a first-class manner

[bangusi]

Wondering what the expected behaviour should be after the fix.
Now the library instead of crashing is simply silently fails ( stops processing the file ).
Is there a configuration change I need to make?
All I did was download version 1.3.0 the run against the same input file that was crashing it.

[vincentlaucsb]

Try passing in a CSVFormat where you set variable_columns(true)

Example: https://github.com/vincentlaucsb/csv-parser#handling-variable-numbers-of-columns

[bangusi]

Yes, now works.

[vincentlaucsb]

Good to hear.