parallel write possibility about fastparquet

[martindurant]

This implemented as part of dask - you could do something similar without dask, using threads or whatever, if you wished.

[areshytko]

can you please provide some reference how it can be done manually (without dask). I couldn't find the relevant info in the project documentation

[martindurant]

The easiest would probably be to write each data-frame to a separate parquet file in the same directory, parallelized in whatever manner is suitable to you, and then call merge() when done. Looking at the code of merge(), you will see that it loads in some metadata from each file, which you could in theory be avoided with a little extra code.

[areshytko]

Many thanks!