bayer-group / cellenium

Cellenium is a FAIR and scalable interactive visual analytics app for scRNA-Seq data (single-cell RNA sequencing).

cellenium

Cellenium is a FAIR and scalable interactive visual analytics app for scRNA-Seq data. It allows to:

• organize and semantically find scRNA studies with ontologized metadata for tissues and diseases
• explore cell types and other cell annotations in UMAP space
• find differentially expressed genes based on clusters of annotated cells
• view the expression of a single gene (or a few selected genes) in the UMAP plot or as grouped violin plots
• draw coexpression plots for pairs of genes, explore the cell types contained in the plots
• add new cell annotations based on plot selections, see differentially expressed genes for a selected group of cells
• find genes which expression is highly correlated to a query gene
• find marker genes in all imported studies and qualitatively compare gene expression across studies

Link to publication: https://doi.org/10.1093/bioinformatics/btad349

Link to showcase: https://youtu.be/U71qIK-Mqlc

System Overview

Cellenium imports scRNA expression data and cell annotations in H5AD format. We provide jupyter notebooks for downloading some publicly available scRNA studies, normalize the data if necessary, and calculate differentially expressed genes, a UMAP projection and other study data that is needed for Cellenium's features to work.

Cellenium is a web application that accesses a PostgreSQL database via GraphQL API. Some API features, like server-side rendered plots, depend on Python stored procedures. The graphql_api_usage folder contains a couple of example queries to illustrate the API capabilities.

The setup steps below automate the download and creation of appropriate H5AD files, docker image build, database schema setup and data ingestion.

Setting up

Preparation of CellO data files (workaround for deweylab/CellO#29 ):

mkdir scratch/cello_resources
curl https://deweylab.biostat.wisc.edu/cell_type_classification/resources_v2.0.0.tar.gz >scratch/cello_resources/resources_v2.0.0.tar.gz
tar -C scratch/cello_resources -zxf scratch/cello_resources/resources_v2.0.0.tar.gz

Cellenium setup, including execution of study data processing notebooks (initially, this will take a couple of hours to run).

# builds docker images and runs the whole stack
# until you run the "make reset_database" step below, error messages about the missing "postgraphile" user pile up... you can ignore them for now.
docker compose up
conda env create -f data_import/environment.yml
conda activate cellenium_import
# 'test_studydata' should contain data to cover all application features, but is small enough to be imported in a few minutes
make reset_database test_studydata_import

# 'normal_studydata': real life studies (i.e. with full amount of cells and genes)
make normal_studydata_import

# we have one for atac
make atac_studydata_import

# and one for cite
make cite_studydata_import

The GraphQL API explorer is available at http://localhost:5000/postgraphile/graphiql . Postgraphile will listen to changes in the database schema and the updated API is visible immediately.

The cellenium webapp 'production build' static site is hosted in the 'client' container, see http://localhost:6002/ . For development, you run (cd client && yarn && yarn start) to install the webapp's dependencies and have a hot-reloaded webapp.

Before you process and import the huge example study (there are two additional make targets for that), edit the beginning of heart_failure_reichart2022*.ipynb and define the download URL as described in the notebooks.

manually executing the study data preparation jupyter notebooks

The notebooks are run in headless mode by make. To create new notebooks and explore datasets:

(cd data_import && PYTHONPATH=$(pwd) jupyter-lab)

cellenium's Issues

log of pvalues = 0 is inf, so they are not shown in volcano plot

color change of selected annotation group after gene expression is plotted on top (e.g. pancreas data set, with CellO CD14-positive and VCAN gene)

scale violinplot down to 100% of availabe space

clustering for the rna part of the PBMC citeSeq data set looks wrong although correct in jupyter

study list filter: search for words in title, description; Pubmed ID

user prepared study upload

• S3 as h5ad study import source, in addition to local file system
• mechanism for granting temp credentials to S3 prefix to users, for upload
• simple admin UI for maintaining study metadata after initial import, and associating user groups for access rights
• discovery of uploaded S3 files
• docker container that can run unattended, imports discovered studies

open task prior to release

• #7
• #6
• #8
• #9
• #10
• #11
• #12
• #13
• #14
• #15

extend gene IDs, symbols, descriptions with further species

Using the Biomart client, we could add data for drosophila, macaca fascicularis and maybe more. This could be done by invoking the get_gene_mappings for these Tax IDs in addition, so that data for these species is added to omics_gene and omics_base just like the other species.

In case species aren't covered by Biomart and we want an "on the fly" definition of any gene IDs, symbols and gene descriptions from the gene annotation dataframe of an imported h5ad file, the import process needs to be extended. Currently, it would just raise any error message for unknown species or unknown genes.

add CITE-Seq and ATAC-Seq studies

make the correlation values of pairwise scatter and correlation mining correspond (do not remove 0)

cross-study heatmap should differentiate "low expression" vs. "cell type not measured at all in this study"

e.g. here we do have a very small fraction (0.002150) of cells of astrocyte cell type, but it isn't visible:

remove annotation select in user annotation view

expression plots: option to combine 'equivalent' annotations in one category

Example: many different CD8 T cell subtypes (memory, effector, naive ...) can be all grouped in one CD8 T cell celltype

Maybe there is a lightweight way for a user to reuse a grouping once it is defined (browser local storage?)

user-defined cell annotations: no DEG calculation for multi-omics (h5mu) studies

• first of all, the study h5mu file must be opened the the muon library
• instead of h5ad_obs_index, we should use h5ad_obs_key to safely identify samples
• multiple differential-anything (genes, regulatory regions etc) can be calculated using the modalities in h5mu

small support python package for study preparation

Users who are preparing h5ad files for cellenium should use the code in h5ad_preparation.py. There should be a python library built, and being installable via PyPi, for this use case. While the library isn't there, users can copy and update h5ad_preparation.py on their own

align ts-vector vs ts-query dictionary (e.g. mouse)

make the autocomplete nicer so they show the ontology type

multiple data layers (e.g. another matrix with imputed values)

Database, API and client already work with study_layer_id to distinguish multiple layers per study. For scRNA data, multiple layers get imported but only the first one is displayed in the UI.

• UI selection for layers (at least in Distribution Analysis view)
• for multiple modalities (h5mu files), the layers are imported from all modalities where they exist
• decide if multiple modalities can be in the same layer - currently it is done that way and it is convenient, but the database column study_layer.omics_type is out of line with the implementation

user annotation coexpression plot should show a (0,0) sample for selecting this case

for generated cell types using the user annotation the generated DEGs are not shown

it's for sure calculated and written into the database. must be due to some changes for multiomics

expression plots: show amount of samples next to the boxes / violins

display species in study overview card

filter studies before fetching data for cross-study view, e.g. filter by tissue

user requested gene correlation does not yet work for multiomics

as the h5mu is access the modality needs to be handed over for e.g. mdata.mod['gene']

enable filtering by cell ontology and species