drg-adaptive / mysql-insert-csv Goto Github PK

The field parsing seems to be a bit broken when parsing a CSV file. If a string field contains a " (quote followed by a space), the field is not properly terminated by a closing ' character. I ran into this issue where there are some text fields that refer to measurements like 1/2" and the SQL generated was improperly formatted. Here's an example:
14463,NULL,00536991501,INSULIN SYRINGE/NEEDLE (generic MONOJECT INSULIN SYRINGE),DOES NOT APPLY,MISC,MISC,28G X 1/2" 1 ML,97051030906370,INSULIN SYRINGE/NEEDLE,INSULIN SYRINGE/NEEDLE U-100,N,G,B,0,1,0
However, the SQL generated for this line is:
(14463,NULL,'00536991501','INSULIN SYRINGE/NEEDLE (generic MONOJECT INSULIN SYRINGE)','DOES NOT APPLY','MISC','MISC','28G X 1/2\" 1 ML,97051030906370,INSULIN SYRINGE/NEEDLE,INSULIN SYRINGE/NEEDLE U-100,N,G,B,0,1,0\n14464,NULL,00536993001,EASY TOUCH INSULIN SYRINGE,DOES NOT APPLY,MISC,MISC,31G X 5/16\" 1 ML,97051030906387,EASY TOUCH INSULIN SYRINGE,INSULIN SYRINGE/NEEDLE U-100,N,T,B,0,1,0',NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL),
Note that this is also including the following line, but the closing quotes enclose other fields. This appears to be an issue whenever quotes are escaped as part of the string, rather than enclosing a string field (that includes a comma). I suspect this is just being incorrectly parsed.

Instead of always writing progress to the console, allow the user to specify a callback method that will receive the current percentage completed.

On instantiation, allow the user to pass in an object specifying transformer methods for specific columns.

Excel seems to leave some non-displayable characters whenever it generates a new CSV file. Saving an existing CSV through excel does not seem to do this. This becomes an issue because these characters are interpreted to be part of the header and this utility fails to match (there is a stray '?' character that doesn't appear in the original CSV).