lts4 / seizure_eeg Goto Github PK

Is your feature request related to a problem? Please describe.

Much information is unnecessarily duplicated in the segments dataframe (e.g. date and sampling rate). We could move all that to a separate dataframe indexed by file name.
In there we could also include

• Available channels
• Path to original file

Describe the solution you'd like

A separate file would carry all that information.

Problems

We would have to provide another dataframe to EEGDataset, unless the segments df point to the metadata instead of the actual file (this requires reading two files at each call).

The epilepsy benchmarks provide preprocessing for more datasets. Align to that.

Describe the bug
When using segments_by_labels(df, targetlabel=[...], relabel= True), an error occur that the return value is a Series, rather than a DataFrame[ClipsDF].

Possible Reason
def _relabel(df: DataFrame[ClipsDF], target_labels: List[int]) -> DataFrame[ClipsDF]:
lmap = {label: i for i, label in enumerate(target_labels)}
return df["label"].map(lmap)

df["label"].map(Imap) return the column mapping result, which is a series. The correct return should be:

df["label"] = df["label"].map(Imap)
return df

The README file is out of date.

Describe the bug
In config labels with null mapping shall be discarded. Segments are correctly dropped, but other segments from same session are kept

To Reproduce

Map mysz to None and notice that v1.5.2/edf/train/02_tcp_le/065/00006544/s001_2010_06_16/00006544_s001_t000 is kept.

Expected behavior
Session is completely dropped unless other labels are present.

Is your feature request related to a problem? Please describe.

Parquet is not human readable

Describe the solution you'd like

Moving to csv or some other "excel" compatible format would allow for quicker human inspection.

Describe alternatives you've considered

Adding one intermediate file from which we compute the parquet would allow to have human-readable format and no zero lines (cfr other issue), while keeping the same implementation for the pipeline

Is your feature request related to a problem? Please describe.

Naming is inconsistent with literature/consortium and some fields could be improved

Describe the solution you'd like

• Add one level of hierarchy in indices: split session in session/file
• Extract the date from EDF files, to include time
• Rename segments to events
• Change label from numerical to string -- ⚠️ TBD

Additional context
Example:

test.csv

Is your feature request related to a problem? Please describe.
Short ictal clips are discarded, along with corresponding background if they are too short.

Describe the solution you'd like
Would be useful to use the overlap argument to specify whether to keep short clips

Additional context
Shall carefully consider what happens if ictal segments overlap the pre-ictal backroung of following seizure.

Is your feature request related to a problem? Please describe.

Currently much of the preprocessing is duplicated (e.g. channel validation and resampling). Furthermore, there is no option to add custom steps (e.g. bandpass filtering or Fourier transforms)

Describe the solution you'd like

• Keep a module that loads the data from EDF or dataset-specific format into a uniform dataframe.
• Centralize the (optional) resampling
• Allow for user-defined processing (matrix-to-matrix)

Describe alternatives you've considered

We could apply user-defined transform in a second moment by iterating over all files, but it might be problematic if the process is not completed in one pass.