waldronlab / hmp16sdata

16S rRNA Sequencing Data from the Human Microbiome Project

HMP16SData

HMP16SData is a Bioconductor ExperimentData package of the Human Microbiome Project (HMP) 16S rRNA sequencing data for variable regions 1–3 and 3–5. Raw data files are provided in the package as downloaded from the HMP Data Analysis and Coordination Center. Processed data is provided as SummarizedExperiment class objects via ExperimentHub.

HMP16SData can be installed using BiocManager as follows.

BiocManager::install("HMP16SData")

Once installed, HMP16SData provides two functions to access data – one for variable region 1–3 and another for variable region 3–5. When called, as follows, the functions will download data from an ExperimentHub Amazon S3 (Simple Storage Service) bucket over https or load data from a local cache.

V13()

## class: SummarizedExperiment 
## dim: 43140 2898 
## metadata(2): experimentData phylogeneticTree
## assays(1): 16SrRNA
## rownames(43140): OTU_97.1 OTU_97.10 ... OTU_97.9997 OTU_97.9999
## rowData names(7): CONSENSUS_LINEAGE SUPERKINGDOM ... FAMILY GENUS
## colnames(2898): 700013549 700014386 ... 700114963 700114965
## colData names(7): RSID VISITNO ... HMP_BODY_SUBSITE SRS_SAMPLE_ID

V35()

## class: SummarizedExperiment 
## dim: 45383 4743 
## metadata(2): experimentData phylogeneticTree
## assays(1): 16SrRNA
## rownames(45383): OTU_97.1 OTU_97.10 ... OTU_97.9998 OTU_97.9999
## rowData names(7): CONSENSUS_LINEAGE SUPERKINGDOM ... FAMILY GENUS
## colnames(4743): 700013549 700014386 ... 700114717 700114750
## colData names(7): RSID VISITNO ... HMP_BODY_SUBSITE SRS_SAMPLE_ID

The two data sets are represented as SummarizedExperiment objects, a standard Bioconductor class that is amenable to subsetting and analysis. To maintain brevity, details of the SummarizedExperiment class are not outlined here but the SummarizedExperiment package provides an excellent vignette.

For a complete explanation of the features of HMP16SData, see the package vignette or read the American Journal of Epidemiology article.

---

Schiffer, L. et al. HMP16SData: Efficient Access to the Human Microbiome Project through Bioconductor. Am. J. Epidemiol. (2019).

Griffith, J. C. & Morgan, X. C. Invited Commentary: Improving accessibility of the Human Microbiome Project data through integration with R/Bioconductor. Am. J. Epidemiol. (2019).

Waldron, L. et al. Improving Accessibility of the Human Microbiome Project Data Through Integration With R/Bioconductor. Am. J. Epidemiol. (2019).

hmp16sdata's Issues

Raw 16S data

Can you please add a repository with raw 16S sequencing data (fastq) used in this package?

Add Export Code for SAS, SPSS, and STATA

Add the following code to the primary vignette.

library(HMP16SData)
library(haven)
library(readr)

taxonomic_counts <-
    V13() %>%
    assay() %>%
    as.data.frame()

colnames(taxonomic_counts) <-
    colnames(taxonomic_counts) %>%
    paste0("_", .)

taxonomic_classification <-
    V13() %>%
    rowData() %>%
    as.data.frame()

participant_data <-
    V13() %>%
    colData() %>%
    as.data.frame()

write_sas(taxonomic_counts, "~/taxonomic_counts.sas7bdat")
write_sas(taxonomic_classification, "~/taxonomic_classification.sas7bdat")
write_sas(participant_data, "~/participant_data.sas7bdat")

write_sav(taxonomic_counts, "~/taxonomic_counts.sav")
write_sav(taxonomic_classification, "~/taxonomic_classification.sav")
write_sav(participant_data, "~/participant_data.sav")

write_dta(taxonomic_counts, "~/taxonomic_counts.dta")
write_dta(taxonomic_classification, "~/taxonomic_classification.dta")
write_dta(participant_data, "~/participant_data.dta")

write_csv(taxonomic_counts, "~/taxonomic_counts.csv")
write_csv(taxonomic_classification, "~/taxonomic_classification.csv")
write_csv(participant_data, "~/participant_data.csv")

V13() %>%
    as_phyloseq() %>%
    psmelt() %>%
    write_csv("~/V13_phyloseq.csv")

V35() %>%
    as_phyloseq() %>%
    psmelt() %>%
    write_csv("~/V35_phyloseq.csv")

ExperimentHub error

When I tried to install the package I got this error:

** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘HMP16SData’:
.onLoad failed in loadNamespace() for 'HMP16SData', details:
call: h(simpleError(msg, call))
error: error in evaluating the argument 'x' in selecting a method for function 'query': DEFUNCT: As of ExperimentHub (>1.17.2), default caching location has changed.
Problematic cache: /home/msubirana/.cache/ExperimentHub
See https://bioconductor.org/packages/devel/bioc/vignettes/ExperimentHub/inst/doc/ExperimentHub.html#default-caching-location-update

load from local file?

load("~/20180312.V13.rda") etc, looks unintended for the package?

https://github.com/waldronlab/HMP16SData/blame/master/vignettes/HMP16SData.Rmd#L47

Import all of SummarizedExperiment

Import all of SummarizedExperiment to make the exploration of objects easier.

Add the ability to convert to ExpressionSet

Draft code written with @lgeistlinger to convert to ExpressionSet and eventually phyloseq class objects.

V13 <- V13()
names(assays(V13))[1] <- "exprs"
V13.eset <- as(V13, "ExpressionSet")
ExpressionSet2phyloseq(V13.eset)

attach_dbGaP and download_dbGaP not working under latest version (2.10.7) SRA toolkit

Hello! The 2.10.7 version of SRA Toolkit no longer supports "vdb-config --import" to import ngc files. Instead, tools such as prefetch, fastq-dump, and vdb-decrypt now have a --ngc option to specify the ngc file for accessing control accessed data. Therefore, attach_dbGaP and download_dbGaP functions need to be updated.

Thanks!

Decorate to match curatedMetagenomicData

Add a function that will modify the objects to match those present in the curatedMetagenomicData package.

sra toolkit installation for attach_dbGaP

Document how to install sra toolkit in the help page and the vignette. I see it's there as an error, but should be in the docs too.

Existing code for importing HMP 16S data into phyloseq

@schifferl may be helpful, includes importing of phylogenetic tree:

http://joey711.github.io/phyloseq-demo/HMP_import_example.html

Specific documentation for matching samples to curatedMetagenomicData

Documentation is instead generic:

HMP16SData/R/V13.R

Line 72 in 8693301

#' @source If you're interested in joint analysis of 16S and shotgun metagenomic