mackelab / epiphyte

Python toolkit for working with high-dimensional neural data recorded during naturalistic, continuous stimuli @a-darcher @rachrapp

Epiphyte

What is Epiphyte?

Epiphyte is a worked data management solution that enables flexible analysis and parallel collaboration on complex datasets. This solution is applied specifically to data consisting of high-dimensional neural signals and a naturalistic, continuous stimulus, although it could be adapted to any paradigm.

Usage

Epiphyte is a worked example, and contains a series of tutorial notebooks meant to bring users through the process of configuring, designing, and deploying a database environment that enables remote collaboration across multiple cities on a complex dataset.

Going through the tutorials will yield a fully functional database, populated with generated mock neural activity and stimulus annotations. While users can in principle modify the import structure and table definitions, the goal of this project is:

1. to motivate a specific database structure for complex datasets, and
2. to provide a worked example for building and embedding such a database into an ecosystem that facilitates analysis and remote collaboration.

How to install Epiphyte?

Epiphyte can be configured and deployed for three main use cases, as found in the above flowchart.

Option A: Install a local instance of Epiphyte.

Use cases:

• You want to test out the database infrastructure before deploying to a remote server.
• You will be the only user and do not work with large data files.

Follow the installation instructions here:

and continue to Tutorial 4: Configure and connect to the database.

Option B: Install a remote instance of Epiphyte, without MinIO.

Use cases:

• Multiple people, accessing from separate locations, will use the database.
• You do not need to support large data files.

Complete Tutorial 1: Launch the MySQL Database

and skip to Tutorial 3: Install and set up Epiphyte.

Option C: Install a remote instance of Epiphyte, with MinIO.

Use cases:

• Multiple people, accessing from separate locations, will use the database.
• You use large data files (e.g., LFP, movie data, multi-hour calcium imaging).

Start at Tutorial 1: Launch the MySQL database and continue through the remaining tutorials.

epiphyte's Issues

datajoint tables sometimes stall/hang

example: UnitLevelDataCleaning.populate() gets hung up -- but still actually populates the table.

Running requirements.txt

Attempted to off-the-shelf use the repo, and ran into two cases with the requirements.txt dependencies.

Removed:
datajoint @ file:///home/conda/feedstock_root/build_artifacts/datajoint_1589638625183/work
tqdm @ file:///home/conda/feedstock_root/build_artifacts/tqdm_1588513834538/work

and pip installed datajoint/tqdm

[Code Quality] Incomplete Repo information

Your repo does not comply to the standards we defined in the lab.
Make sure for your repo to have:

• a description including the github handle of the owner
• a > 3 line README.md

If you don't update your repo, it will be disabled and then archived.