wiebket / delprocess Goto Github PK

• Add SANEDI and ERC logos to README

• Instead of loadprofiles.saveReducedProfiles writing to profiles_dir/interval, write to usr_dir/resampled_profiles.
• Instead of surveys.genS writing to usr_dir/usr/features write to usr_dir/survey_features

Hi @wiebket,
I hope you're well!
I just wanted to find out if the missing values in the meter readings for specific hours of the day for households represent power disruptions or outage of some sort?
By missing values, I mean excluding readings with 0's.

I would greatly appreciate your feedback!
Thank you!

Currently reads a year's data into memory and then uses a resample operation to reduce it. Crashes in year 1999 on laptop RAM as dataset is too large. Try:

1. try dask and groupby ProfileID only --> may chuck out RecorderID, which will need to be joined afterwards
2. chunk resample operation with a for loop
3. figure out dask properly and try again

Fix install instructions:

• install no longer prompts for data directory, that only happens when running the processing from the command line for the first time

• surveys.py
• loadprofiles.py
• plotprofiles.py

• saveReducedProfiles
• genS
• searchQuestions
• automate generation of spec files for genS

pyqt may be a great option

Create visualisations of key descriptive elements for a processed dataset. Can be put in plotprofiles.py, which should be renamed to plotdlr.py.

rename command line tools to dlrpro_profiles and dlrpro_surveys. Shorter than existing.