qcri / sleep_awake_benchmark

This code is part of the paper ''A Large Scale Benchmark to Validate Sleep-Wake Scoring Algorithms'' currently under review.

Sleep Awake Benchmark

This code is part of the paper ''A Large Scale Benchmark to Validate Sleep-Wake Scoring Algorithms''. See the full publication at: https://www.nature.com/articles/s41746-019-0126-9

@article{palotti2019benchmark,
  title={Benchmark on a large cohort for sleep-wake classification with machine learning techniques},
  author={Palotti, Joao and Mall, Raghvendra and Aupetit, Michael and Rueschman, Michael and Singh, Meghna and Sathyanarayana, Aarti and Taheri, Shahrad and Fernandez-Luque, Luis},
  journal={npj Digital Medicine},
  volume={2},
  number={1},
  pages={50},
  year={2019},
  publisher={Nature Publishing Group}
}

How To Use it

1. Download MESA datasets from sleepdata.org (https://sleepdata.org/datasets/mesa)
2. Process dataset with notebook -- generate_mesa_dataset_task.ipynb
3. In general, all scripts will need to preprocess the data (i.e., get the mean activity value in a win of size X). To save time, we use implemented a preprocessing script (sleep_processdataset.py) which the only parameter is the task number. It outputs a file called hdf_task. Run with:

$ python sleep_processdataset.py <taskid>

4. Run formulas (automatically reads dataset from hdf_task) and process all formulas:

$ python sleep_formulas.py <taskid>

5. Run machine learning models:

$ python sleep_ml.py <taskid>

6. Run LSTM/CNN:

$ python sleep_nn.py <taskid> <seq_len:20,50,100> <kind:LSTM,CNN>

7. Geting the results. 7.1. After running all scripts, a bunch of intermediary result files was created (e.g., task1_formulas.csv). The script ship.sh collect them all and send them to their expected diretory (e.g., result or summary). 7.2. sleep_summary.py generates the summaries files, which are the input for the ipython/jupyter notebooks (result_analysis.ipynb, sleep_auc_plots.ipynb and sleep_plot345.ipynb). Notebooks provide an easy way to visualize and analyze the results, as well as generate the plots used in the manuscript.

sleep_awake_benchmark's Issues

hdf_task file too large for memory

Hi!

Thanks so much for making this available! I'm interested in rerunning this code to get a benchmark for performance and compute time.

I hit a wall in the pipeline though in sleep_formulas.py, where I need to load the hdf_task file into memory. For me, this file is about 6Gb and my computer crashes when I try to load it.

Is it supposed to be that big? In contrast, the dftest_task1.csv file is only 20Mb or so. Would it be possible to make a lighter version of that file? Do you have any advice on getting through this obstacle? I'm not sure how tractable it will be for me to run this pipeline versus rewriting my own.

Thanks so much!
-Aaron