almeida-marcos / deirokay

Data parsing and validation done right

Deirokay

Deirokay (dejɾo'kaj) is a tool for data profiling and data validation.

Deirokay separates document parsing from validation logic, so that you can create your statements about your data without worrying whether or not your file has been properly parsed.

You can use Deirokay for:

• Data parsing from files (CSV, parquet, excel, or any other pandas-compatible format);
• Data validation, via Deirokay Statements;
• Data profiling, which generates Deirokay Statements automatically based on an existing file. You may use these statements later against new documents to make sure the validation still holds for new data.

Installation

To start using Deirokay, install its package by following the instructions below.

By default, only core dependencies are installed when supplying the pip command:

pip install Deirokay

Depending on your use cases, you need to include extra dependencies. Use any of the commands below:

pip install Deirokay[s3]  # Optional dependencies for use with AWS S3
pip install Deirokay[pandas]  # For Pandas backend
pip install Deirokay[dask]  # For Dask or Dask Distributed backends

You may also install any combination of the extras above by separating them with commas:

pip install Deirokay[dask,s3]  # Dask + S3 deps

To check if Deirokay is properly installed and to know its version, enter the following command in the Python prompt:

>>> import deirokay
>>> deirokay.__version__
'X.Y.Z'

If you wish to contribute for Deirokay development, you need to install a more restrict set of packages, in order to guarantee that you are always creating and testing code to work with least recent supported versions:

pip install Deirokay[dev]

Documentation

Please, read the docs.

Contributing

Check our contributing guidelines.