mstraubac / fast-cpp-csv-parser Goto Github PK

What steps will reproduce the problem?
1. create an io::CSVReader<8, io::trim_chars<>, io::no_quote_escape<','>, 
io::throw_on_overflow, io::no_comment>
2. call read_header(io::ignore_missing_column, ...) on the above instance, 
passing eight strings
3. attempt to compile program

What is the expected output? What do you see instead?
The code should compile and run as expected; instead, a compile error is thrown 
with the following text:

.../csv_v2.h:942:71: error: member reference base type 'std::string const[8]' 
is not a structure or union
                                             - std::begin(column_names.begin));
                                                          ~~~~~~~~~~~~^~~~~~


What version of the product are you using? On what operating system?
csv_v2.h initial commit, XCode 4.6.2, OSX 10.8.4, default project configuration 
(Apple LLVM compiler 4.2, GNU99 C dialect, GNU++11 C++ dialect)

Please provide any additional information below.

The issue only occurred when has_column was called in the program, and was 
resolved by changing the code of has_column to

    return  col_order.end() != std::find(
                    col_order.begin(),
                    col_order.end(),
                    std::find( std::begin(column_names), std::end(column_names), name) - std::begin(column_names));

as per 
http://stackoverflow.com/questions/14595285/cannot-use-begin-or-end-on-an-array

What steps will reproduce the problem?
1. Parse a CSV with newlines in quotes (as for example mentioned on this page: 
http://creativyst.com/Doc/Articles/CSV/CSV01.htm )

What is the expected output? What do you see instead?
The string, but with newlines in the value. Instead it throws 
escaped_string_not_closed.

Is there any way to nicely fix this? I couldn't see a way very easily, since 
the input is done line by line. That would need to be revised to do it column 
by column I think.

What steps will reproduce the problem?
1. g++ main.cpp -o t.exe -std=c++0x -lpthread
2.
3.

What is the expected output? What do you see instead?


C:\Users\Hassan\Documents\NetBeansProjects\MALT-extractor>g++ main.cpp -o t.exe 
-std=c++0x -lpthread
In file included from main.cpp:9:0:
csv.h:102:19: error: field 'bytes_read' has incomplete type
csv.h: In member function 'void io::LineReader::init()':
csv.h:145:5: error: 'bytes_read' was not declared in this scope
csv.h:147:6: error: invalid use of incomplete type 'struct std::future<int>'
c:\mingw\bin\../lib/gcc/mingw32/4.6.2/include/c++/future:111:11: error: 
declaration of 'struct std::future<int>'
csv.h: In member function 'char* io::LineReader::next_line()':
csv.h:214:8: error: 'bytes_read' was not declared in this scope
csv.h:220:7: error: invalid use of incomplete type 'struct std::future<int>'
c:\mingw\bin\../lib/gcc/mingw32/4.6.2/include/c++/future:111:11: error: 
declaration of 'struct std::future<int>'
csv.h: In destructor 'io::LineReader::~LineReader()':
csv.h:256:7: error: 'bytes_read' was not declared in this scope
In file included from main.cpp:9:0:
csv.h: In constructor 'io::CSVReader<column_count, trim_policy, quote_policy, 
overflow_policy, comment_policy>::CSVReader(Args ...)':
csv.h:877:31: error: 'to_string' is not a member of 'std'
main.cpp: In function 'int main(int, char**)':
main.cpp:28:7: error: 'cout' is not a member of 'std'
c:\mingw\bin\../lib/gcc/mingw32/4.6.2/include/c++/future: At global scope:
c:\mingw\bin\../lib/gcc/mingw32/4.6.2/include/c++/future:143:5: error: 
'std::future<typename std::result_of<_Functor(_ArgTypes ...)>::type> std::async
(std::launch, _Fn&&, _Args&& ...) [with _Fn = 
io::LineReader::init()::<lambda()>, _Args = {}, typename 
std::result_of<_Functor(_ArgTypes ...)>::type =
 int]', declared using local type 'io::LineReader::init()::<lambda()>', is used but never defined [-fpermissive]
c:\mingw\bin\../lib/gcc/mingw32/4.6.2/include/c++/future:143:5: error: 
'std::future<typename std::result_of<_Functor(_ArgTypes ...)>::type> std::async
(std::launch, _Fn&&, _Args&& ...) [with _Fn = 
io::LineReader::next_line()::<lambda()>, _Args = {}, typename 
std::result_of<_Functor(_ArgTypes ...)>::t
ype = int]', declared using local type 
'io::LineReader::next_line()::<lambda()>', is used but never defined 
[-fpermissive]




What version of the product are you using? On what operating system?
I am using mingw ( gcc 4.6.2) on windows vista


Please provide any additional information below.

What steps will reproduce the problem?
1. double quote escaped columns are not trimmed when using

io::CSVReader<12, io::trim_chars<' ', '\t'>, io::double_quote_escape<',', 
'\"'>, io::throw_on_overflow, io::single_line_comment<'N'> > 
in("LoanStatsNewSmall.csv");

What is the expected output? What do you see instead?
Expect columns as such to be trimmed
" 11.89%"

instead the space is not trimmed

What version of the product are you using? On what operating system?
Latest on Windows

Please provide any additional information below.

Here is the nearly reproducible test code

            io::CSVReader<12, io::trim_chars<' ', '\t'>, io::double_quote_escape<',', '\"'>, io::throw_on_overflow, io::single_line_comment<'N'> > in("LoanStatsNewSmall.csv");
            in.read_header(io::ignore_extra_column, "acc_open_past_24mths", "funded_amnt", "loan_status", "issue_d", "term", "installment", "int_rate", "total_pymnt", "out_prncp", "out_prncp_inv", "total_rec_int", "total_rec_prncp");

            std::map<std::string, std::string> raw_loan;

            while (in.read_row(
                raw_loan["acc_open_past_24mths"],
                raw_loan["funded_amnt"],
                raw_loan["loan_status"],
                raw_loan["issue_d"],
                raw_loan["term"],
                raw_loan["installment"], 
                raw_loan["int_rate"],
                raw_loan["total_pymnt"],
                raw_loan["out_prncp"],
                raw_loan["out_prncp_inv"],
                raw_loan["total_rec_int"],
                raw_loan["total_rec_prncp"])) {
             }

Wrote a small patch for some simple things:

* add static keyword to some constants
  -> is redundant but makes things more explicit
* moved primitive member initialization from constructors to definition
  -> is possible in c++11 which is required anyway
  -> also fixes the typo in struct with_errno where instead
     of the member variable, the global errno macro was written
  -> may break msvc compatiblity even further :)

Review and apply if you like it.

The library contains an additional undocumented policy that allows to identify 
lines as comment and ignore them. The documentation does not yet mention this. 

What steps will reproduce the problem?
1. Build on windows with Visual Studio 2013

What is the expected output? What do you see instead?
Compile is successful


What version of the product are you using? On what operating system?
latest Windows

Please provide any additional information below.
1. Had to create a custom snprintf function since VS2013 still doesn't have one

2. Had to remove all the constexpr since VS2013 complains about those

3. Also VS2013 complains about multiple default constructors, this time it's 
seems right, these can either be chosen when no arguments are supplied
CSVReader() = delete;
explicit CSVReader(Args...args):in(std::forward<Args>(args)...){

I commented out the first one

Attached is the new file with all the patches

What steps will reproduce the problem?
1. unzip stop_times.txt.7z
2. try io::CSVReader<5> in("stop_times.txt");
3. this works; add an additional line to the file by copying the last line
4. breaks with a segfault

What is the expected output? What do you see instead?
The expected output is nothing, I see a segfault

What version of the product are you using? On what operating system?
Ubuntu Linux 12.4, gcc 4.6.3

Please provide any additional information below.
As far as I could debug, the error occurs while leaving scope in the init() 
function.

What steps will reproduce the problem?
1. I have a CSV file with some extra new lines (i.e. the lines with data are 
separated with an empty line), so I wanted to use the single line comment 
feature to ignore the lines starting with '\n'
The files look like this

header1,header2
2,2

3,3

4,4

instead of 

header1,header2
2,2
3,3
4,4

2. io::CSVReader<NCOL,io::trim_chars<' 
'>,io::no_quote_escape<','>,io::single_line_comment<'\r','\n'> >in(filename);
3. try reading the file in question

What is the expected output? What do you see instead?

I would expect the empty lines to be ignored, but I get "Too few columns in 
line 3 in file" as an error

What version of the product are you using? On what operating system?

Current version, on Mac OS X mountain lion

Please provide any additional information below.