michaelcurrin / daylio-csv-parser

Improve the usability of the Daylio CSV export and explore reports around your data ☺️ 📆 🐍

Daylio CSV Parser ☺️ 📆 🐍

Improve the usability of the Daylio CSV export and explore reports around your data

This Python 3 CLI took will convert a Daylio CSV export into a more usable CSV and a SQLite database.

About Daylio CSV

What is Daylio? Read on this doc page.

This daylio-csv-parser application parses a CSV exported from Daylio (in premium mode) to create a more useful CSV.

In particular, the activities column with multiple activities listed in a single cell is split out into multiple columns, with appropriate names and values as 0 or 1.

Sample usage

A summary of the command-line API is covered below.

Main feature

Run the main script to read CSV exported from Daylio at a configured location and create a clean CSV.

$ make csv

Other features

Assuming you have created a clean CSV using the step above, you can run any of these commands if you wish to. The order does not matter.

Database

Create a database file from the cleaned CSV. You can then access data in the database.

$ make db

Aggregate mood report

View a report around mood score aggregate stats.

$ ./mood_report.py

Regression report

View a report on a stats model which was fitted your data.

$ ./fit_model.py

Documentation

License

Released under MIT by @MichaelCurrin.

daylio-csv-parser's Issues

Add doc badges

Improve CSV handling

Based on README usage.

Streamline - a new or fixed CSV mean steps need to be redone and then the next ones.

Delete file? Repeat import didn't clear entry.

Actually not used, just CLEANED CSV is reread.

Remove leading zeroes which are NA

For fields which are infrequent or added later, do not consider absence of the field use as zero.
This will affect prediction scores.

When field appears used for the first time, all previous rows must get NA for that field. And NA must be treated properly in regression - research this first to see that it works.

In the clean_csv.py script, process command-line args

For now only use default values.

Add makefile

Add docs site

Centralize CSV date parsing

Improve fit_model.py

et coeff and print that and not other fields.

Can I apply to 5 point scale?
Years and months have over 1 - is the range not -1 to 1 correlation?

Add command-line args to mood_report.py

Use argparse to read in a custom value rather than use the default.

Add command-line args to fit_model.py

Use argparse to read in a custom value rather than use the default.

I'm getting this error: original_activities_str = row["activities"] KeyError: 'activities'

Not sure what this is about. I had another error shown to me in a .csv that I edited with Excel. This one, however, is a .csv that I edited with Excel without transforming the data with a PowerQuery, just used Ctrl + F to replace some of the strings.

---

(venv) Tet@DESKTOP: make csv

python -m dayliopy.clean_csv
Reading CSV: C:\Users\Tet\OneDrive\Documentos\github-gitlab\daylio-csv-parser\dayliopy/var/data_in/daylio_export.csv
Traceback (most recent call last):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2800.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2800.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\Tet\OneDrive\Documentos\github-gitlab\daylio-csv-parser\dayliopy\clean_csv.py", line 227, in <module>
    main()
  File "C:\Users\Tet\OneDrive\Documentos\github-gitlab\daylio-csv-parser\dayliopy\clean_csv.py", line 223, in main
    process(csv_in, csv_out)
  File "C:\Users\Tet\OneDrive\Documentos\github-gitlab\daylio-csv-parser\dayliopy\clean_csv.py", line 207, in process
    available_activities, in_data = read_csv(csv_in)
  File "C:\Users\Tet\OneDrive\Documentos\github-gitlab\daylio-csv-parser\dayliopy\clean_csv.py", line 151, in read_csv
    activities_list = process_activities(original_activities_str)
  File "C:\Users\Tet\OneDrive\Documentos\github-gitlab\daylio-csv-parser\dayliopy\clean_csv.py", line 124, in process_activities
    activities_split = activities_str.split(" | ")
AttributeError: 'NoneType' object has no attribute 'split'
make: *** [Makefile:52: csv] Error 1

Here's my file for debugging:

• daylio_export.csv

Additional information:

Windows 10
Python 3.9.10
pip 21.2.4
GNU make 4.3

Update regression

Find why month_1 is missing

Remove rows with insignificant p values from the output (or sort).
Rerun model on only significant. Or use a different approach for entering variables.

Use black

Support for multiple moods

I can understand why it doesn't support it, it's not straight forward. I have an idea how to make it work, but I'd need to experiment a while longer. I have multiple moods added so I'd like it to work for myself, happy to write it. I think it'd need to be a multistage process - restructure the conf moods so they can store a list, then first process those lists into a big dictionary as {str: int} . Like when the script loads, process the CONF moods into a list for each mood number, then into a dictionary and finally call that dictionary to do the normal translation.

Use plan notes

From plans.txt

This was before git so maybe not relevant