htwangtw / adie_ongoingthoughts

ADIE ongoing thought related analysis plan

ADIE data curation tools

Authors: Hao-Ting Wang, Will Strawson, Joel Patchitt

Last updated: 15-01-2021

As ADIE is a big dataset, it's the best to use Bash command to interact with the data. We host it on the CISC storage voulme on Sussex HPC.

All tools were written based on BIDSv1.4.1.

Set-up your environment

Environment was tested under Ubuntu20.04 with Python >= 3.7.

We recommend running the project in a virtual environment.

Linux

cd /path/to/this/repo/
virtualenv env -p /path/to/bin/python3.8
source env/bin/activate
pip install -r requirements.txt
# install ADIE data analysis aid
pip install -e .

After setup for the first time, you need to activate the enviroment to use this project.

cd /path/to/this/repo/
source env/bin/activate

To deactivate:

deactivate

For developers:

virtualenv env -p /path/to/bin/python3.8
source env/bin/activate
pip install -r dev-requirements.txt
pip install -e .

Populate BIDS dataset

All scripts are under bin. Packaged analysis module adie is in directory of the same name. If you follow the setup instruction, you should be able to use the adie module as a normal python module in the project environment after sourcing the virtual environment.

cd /path/to/this/repo/
source env/bin/activate

I am considering to publish it as a standard library for the ease of use. If you want to work on it, let me know!

Add a new participant

We created command line tool beh2bids to convert experiment log and physiology smr file. To use this tool, place the new data under the relevant directory in sourcedata. We only support conversion of CSV data log for now.

See more detail of this tool:

$ beh2bids --help

Usage: beh2bids [OPTIONS] BIDS_ROOT

  bids_root: path to the root direcotry of BIDS dataset

Options:
  -t, --task TEXT     Task directory name unders sourcedata.
  -s, --subject TEXT  ADIE subject identifier (ADIE??? / CONADIE??? or
                      suffix with session such as "ADIE???BL")

  -f, --behfile TEXT  Behavioural data log file name pattern
                      [default: *.csv]

  -p, --physio TEXT   physiology recording type, such as ecg, pulseox.
                      Set to None if no physiology data is present.

  --help              Show this message and exit.

Batch process new experiment data

Please find cluster job script bin/populate_nback.sh and use it as a template. bin/populate_nback.sh is compatible with Sussex HPC scheduler SGE.

fMRI data

WIP

Questions, bugs related to this tool

Please let me know by opening an issue!

Contribution guidelines

Please have a look at nibable's contribution guidelines. I found them really useful so nor bothered to write my own. These guidelines are designed to make it as easy as possible to get involved. If you have any questions that aren't discussed in the documentation, or it's difficult to find what you're looking for, please let me know by opening an issue!

adie_ongoingthoughts's Issues

Who has T1s?

Need to know exactly how many people do and don't have structural scans. Will figure this next week :)

Reminder to myself to make a report

Just had a thought - it would be good to make a report with tables and graphs 📊 etc that describe the data so far. I'd like to know:

• how many participants (ps) and age distributions
• distribution of runs across participants e.g. 50% of ps completed 2 runs
• how many of these ps have questionnaires/ psychometric stuff

You get the idea.
📉📊📈

CISC ID to behavioural data ID conversion

Need a converter and encrypt the dictionary for conversion.

[FEAT] Convert task fMRI event logs

The event logs related to the task fMRI recording is now stored under:
/path/to/projects/critchley_adie/BIDS_data/sourcedata/fmri_event_logs
We need to convert the logs to BIDS standard event files, the Spike2 file to BIDS physio format, and add the spike2 file to derivatives/physio_smr/ with BIDS format file name and mirror the directory structure of the main BIDS data.

After the fMRI files being converted by @willstrawson using heudiconv, there should be place holder event files.
Create new functions in adie/events.py to populate event logs to the BIDS event files (see BIDS specification).
Source content for the event logs are wri and/or log can be found in /path/to/projects/critchley_adie/BIDS_data/sourcedata/fmri_event_logs.
Relevant smr files can be converted with existing functions created by @htwangtw, with new naming rules.

Alternatively, a script stored under bin/ that does the job would be fine too.

@htwangtw is happy to review your solutions.

Task fMRI event logs

The event logs are in a very stripped-down format.
We need to convert it to BIDS.
For now, I will focus on tasks that are relevant to our analysis.

Annotate phenotype data

Lisa has provided the full assessment data and a reference of all the tasks used.
Have documentation on what things means per column will be useful.
Look up information and write the thing.

Need to copy newly created niftis into existing BIDS subject directories

Subject's Nifti files are not currently stored in their sub- directory in BIDS_data/. Instead, they're in wills_data/. Also, the session level directory names are currently the scan date, rather than the global session label (e.g. 'ses-baseline').

I need to a) copy the Niftis into the correct subjects directory and b) Change the session label name so it matches the labels currently used in BIDS_data/

Don't necessarily need help with this, just thought it was a good idea to log this as an issue so I don't forget.

Overview of the plan

The analysis is going to be focusing on the ongoing thought aspect of the ADIE dataset.
This involves the fMRI data, York mind wandering task with ECG measures.
Following the initial meeting, we agreed to work on the following goals:

• Curate dataset and summarise the available (fMRI) task and limitations
• Written participant information that can be used for future publication
• Document analysis plans and rationales

Potential information we would like to include in this repository:

• Funding source and related disclaimer
• Information on authors of the dataset (we two clearly did not collect it)

Suggestion: since we know there are subjects with no T1. We probably need to look into strategy (ideally in FSL) on preprocessing. There were discussion on fMRIPrep related to a direct EPI to MNI space registration. Do you think it would be a good idea to work on a extra feature in fMRIPrep for the analysis? We can work on a Python based prototype and approach the fMRIPrep team for their help to make it fMRIPrep ready?

Please add things I missed out in this thread.

[fMRIPrep] Preprocessing

fMRI data is not preprocessed. I want to run the data through fMIRprep with minimal preprocessing so the resulting data can be used for any subsequent analysis. I say we deal with gradient-based preprocessing steps (e.g. parcellations) in a different issue.

Generate top-level `participants.tsv` and subject level scan log

Currently, there's no top-level participants.tsv. This file should log some basic information about each subject (see BIDS). In sourcedata there's a very extensive version that's basically a combination of participants.tsv and scan logs.

This dataset involves multiple sessions for the patient group. A session can involve multiple days. It would be useful to have a scan log to record the time of the sessions and visits.

BIDS description of scan log and participants file
Example of a multi-session multi-run study
Example of BIDS directory with scan log