Coder Social home page Coder Social logo

p-ranav / csv Goto Github PK

View Code? Open in Web Editor NEW
235.0 14.0 32.0 522 KB

[DEPRECATED] See https://github.com/p-ranav/csv2

License: MIT License

C++ 99.32% CMake 0.68%
csv-parser header-only concurrent-queue library mit-license fast multi-threaded robin-map cpp17

csv's Issues

non-multi-threaded mode

I would prefer to default to threads off for simplicity and error propagation, but I think csv looks like a good project. Looking around at a few of the sources, it seems that this might not be too hard to make happen?

Overload write for void write_row(unordered_flat_map<std::string_view, std::string> row_map)

Is it possible to make it simpler to read data and then write that same data?

The read function reads in rows as unordered_flat_map<std::string_view, std::string>, but the write function expects unordered_flat_map<std::string, std::string>. This appears to require manual conversion between the types, making something like converting a CSV file to a TSV file more difficult than I had hoped.

Implement Dialect.doublequote Dialect.escapechar

Dialect.doublequote
Controls how instances of quotechar appearing inside a field should themselves be quoted. When True, the character is doubled. When False, the escapechar is used as a prefix to the quotechar. It defaults to True.

On output, if doublequote is False and no escapechar is set, Error is raised if a quotechar is found in a field.

Dialect.escapechar
A one-character string used by the writer to escape the delimiter if quoting is set to QUOTE_NONE and the quotechar if doublequote is False. On reading, the escapechar removes any special meaning from the following character. It defaults to None, which disables escaping.

Option to append to file when writing

This is same use case as #16 except for writing to a file. An option to have the writer append to an existing file would be very helpful. Even better might be adding a csv::Writer constructor which takes a std::ostream.

What I am doing wrong?

Hello!
What I want:

  1. I want to create CSV log file with list of exceptions in order to easy browse and analyse in Calc or Excel.
  2. I want to write unit test for saving the exception list (in my CSV file).

What I did:
I program saveAsCSV() and loadFromCSV() functions:

void Exceptions::saveAsCSV(std::wstring aPathToLogFile)
{
    csv::Writer lLogCSV(gToNarrow(aPathToLogFile));
    lLogCSV.use_dialect("excel");
    lLogCSV.configure_dialect()
        .delimiter(", ")
        .column_names("Type", "Code", "Message", "Component",
            "File", "Line", "Function");

    for(std::list<ExceptionPtr>::iterator i(mExceptions.begin())
         ; i != mExceptions.end(); ++i)
    {
        lLogCSV.write_row(std::to_string((*i)->type())
            , std::to_string((*i)->code())
            , gToNarrow((*i)->message())
            , gToNarrow((*i)->component())
            , gToNarrow((*i)->file())
            , std::to_string((*i)->line())
            , gToNarrow((*i)->function())
            );
    }
    lLogCSV.close();
}

std::list<ExceptionPtr> Exceptions::loadFromCSV(std::wstring aPathToLogFile)
{
    dNotifyInfo(fr(tr("Try to load CSV file: {0}"), aPathToLogFile), eFileSystem);
    std::list<ExceptionPtr> lResult;

    csv::Reader lCSVFile;
    lCSVFile.use_dialect("excel");
    std::string lPathToFileName(gToNarrow(aPathToLogFile));
    lCSVFile.read(lPathToFileName);

    while(lCSVFile.busy())
    {
        if(lCSVFile.ready())
        {
            auto lRow = lCSVFile.next_row();
            std::cout << lRow["Type"]
                    << ", " << lRow["Code"]
                    << ", " << lRow["Message"]
                    << ", " << lRow["Component"]
                    << ", " << lRow["File"]
                    << ", " << lRow["Line"]
                    << ", " << lRow["Funciton"]
                    << std::endl << std::flush;

            ExceptionPtr lEx(new Exception(static_cast<Exception::Type>(
                std::stoi(lRow["Type"]))
                , std::stoull(lRow["Code"])
                , gToWide(lRow["Message"])
                , gToWide(lRow["Component"])
                , gToWide(lRow["File"])
                , std::stoi(lRow["Line"])
                , gToWide(lRow["Funciton"])
                ));
            lResult.push_back(lEx);
        }
        else
            std::this_thread::sleep_for(100ms);
    }

    return lResult;
}

saveAsCSV() function works as expected and generates file:

Type, Code, Message, Component, File, Line, Function
0, 1, My exception 1, EnergoKodInstrumentyTest, /home/szyk/!-EnergoKod/!-Libs/EnergoKodInstrumenty/Tests/Src/ExceptionsTest.cpp, 54, virtual void ExceptionsTest::run()
0, 1, My exception 2, EnergoKodInstrumentyTest, /home/szyk/!-EnergoKod/!-Libs/EnergoKodInstrumenty/Tests/Src/ExceptionsTest.cpp, 60, virtual void ExceptionsTest::run()
0, 1, My exception 3, EnergoKodInstrumentyTest, /home/szyk/!-EnergoKod/!-Libs/EnergoKodInstrumenty/Tests/Src/ExceptionsTest.cpp, 66, virtual void ExceptionsTest::run()
1, 2, My warning 1, EnergoKodInstrumentyTest, /home/szyk/!-EnergoKod/!-Libs/EnergoKodInstrumenty/Tests/Src/ExceptionsTest.cpp, 70, virtual void ExceptionsTest::run()
1, 2, My warning 2, EnergoKodInstrumentyTest, /home/szyk/!-EnergoKod/!-Libs/EnergoKodInstrumenty/Tests/Src/ExceptionsTest.cpp, 71, virtual void ExceptionsTest::run()
1, 2, My warning 3, EnergoKodInstrumentyTest, /home/szyk/!-EnergoKod/!-Libs/EnergoKodInstrumenty/Tests/Src/ExceptionsTest.cpp, 72, virtual void ExceptionsTest::run()
2, 3, My info 1, EnergoKodInstrumentyTest, /home/szyk/!-EnergoKod/!-Libs/EnergoKodInstrumenty/Tests/Src/ExceptionsTest.cpp, 74, virtual void ExceptionsTest::run()
2, 3, My info 2, EnergoKodInstrumentyTest, /home/szyk/!-EnergoKod/!-Libs/EnergoKodInstrumenty/Tests/Src/ExceptionsTest.cpp, 75, virtual void ExceptionsTest::run()
2, 3, My info 3, EnergoKodInstrumentyTest, /home/szyk/!-EnergoKod/!-Libs/EnergoKodInstrumenty/Tests/Src/ExceptionsTest.cpp, 76, virtual void ExceptionsTest::run()

But I am unable to load anything from CSV file. The output is:

POCZĄTEK pod testu: Test saving exceptions to CSV.
Information! Code: 2, Message:
Try to load CSV file: /var/tmp/ExceptionTestsExceptionsTest.csv
0, , , , , , 
Wystąpił wyjątek! Wiadomość:
stoull
Naciśnij <RETURN> aby zamknąć to okno...

Note: In loadFromCSV() function I follow strictly your README.md entrance example. So I have no idea how to program it correctly.
Please help me.

ps. If this place is not suitable for such questions - please give me advice where should I ask.

Thank you and best regards.

Support reading directly from std::istream

I have a use case where I want to parse the first part of a CSV file with my own parser and the rest of it with your parser. Currently, I will have to re-parse most of the file with your parser because it only takes a filename as an argument. I have a std::istream exactly at the position where I want to pick up parsing with your parser. Would it be possible to support reading from a std::istream?

Issue with the CSV header

I'm trying to use the csv::Writer as a backend for my Persistent Manager class but for some reason I cannot get it to write the header. Basically, I have something like this:

class PM {
	class Writer;
}

class Writer{

	using namespace std;

	string filename;
	unique_ptr<csv::Writer> writer;

	Writer(string filename) filename(filename) {
		writer = make_unique<csv::Writer>(filename);
		writer->configure_dialect().delimiter(cols);
	}

	~Write(){
		writer->close();
	}

	void write(map<sting, string> row) {
               // I override the method to accept vector<string> as well.
		writer->configure_dialect().column_names(cols);

		writer->write_row(row);
	}
}

Do you see any particular issue with this? I'm not quite sure why honestly, everything looks fine.

Not wroking correctly on file with too many lines

Environment

  • Ubuntu 18.04
  • CLion 2019.1.4

Issue

I test the program on a CSV file 20k_rows_data.csv.txt with 20K lines and the program does not work correctly. (I change the filename with .txt, because GitHub issue does not support uploading .csv file.)

int main() {
  csv::Reader csv;
  csv.read("../tests/inputs/20k_rows_data.csv.txt");
  auto rows = csv.rows();
  auto cols = csv.cols();
  int row_count = 0;
  for (auto row : rows) {
    std::string s = std::to_string(++row_count);
    for (auto col : cols) {
      s += ' ' + (std::string)(row[col]);
    }
    std::cout << s << std::endl;
  }
}

Part of the output is like (copy from my console):

5332     
5333     
5334 1 1 1 1 1
5335     
5336     
5337 1 1 1 1 1
5338     
5339     
5340     
5341 1 1 1 1 1
5342     
5343

Note that the outputs are not the same each time I run it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.