HMP2Data
October 31, 2019
Abstract
HMP2Data is a Bioconductor package providing the data of the integrative Human Microbiome Project (iHMP), also known as the second phase of HMP project (HMP2). It contains 16S rRNA sequencing data from three longitudinal studies, 1) MOMS-PI, pregnancy and preterm birth; 2) IBD, gut disease onset, inflammatory bowel disease; and 3) T2D, onset of type 2 diabetes and respiratory viral infection. Where available, other data is also provided, such as cytokine measures. Raw data files were downloaded from the HMP Data Analysis and Coordination Center. Processed data is provided as matrices, SummarizedExperiment, MultiAssayExperiment, and phyloseq class objects.Package
HMP2Data 1.1.0
Contents
1 Prerequisites
HMP2Data can be installed using BiocManager as follows.
# Check if BiocManager is installed
if (!requireNamespace("BiocManager", quietly = TRUE))
install.packages("BiocManager")
# Install HMP2Data package using BiocManager
BiocManager::install("HMP2Data")Or the development version of the package can be installed from GitHub as shown below.
BiocManager::install("jstansfield0/HMP2Data")The following packages will be used in this vignette to provide demonstrative examples of what a user might do with HMP2Data.
library(HMP2Data)
library(phyloseq)
library(SummarizedExperiment)
library(MultiAssayExperiment)
library(dplyr)
library(ggplot2)
library(UpSetR)Once installed, HMP2Data provides functions to access the HMP2 data.
2 MOMS-PI
The MOMS-PI data can be loaded as follows.
2.1 16S data
Load 16S data as a matrix, rows are Greengene IDs, columns are sample names:
data("momspi16S_mtx")
momspi16S_mtx[1:5, 1:3]
#> EP003595_K10_MV1D EP003595_K100_BRCD EP003595_K90_BCKD
#> 807795 20 0 0
#> 134726 1 0 0
#> 215097 2 0 0
#> 542066 1 0 0
#> 851634 1 0 0Load the Greengenes taxonomy table as a matrix, rows are Greengene IDs, columns are taxonomic ranks:
data("momspi16S_tax")
colnames(momspi16S_tax)
#> [1] "Kingdom" "Phylum" "Class" "Order" "Family" "Genus" "Species"
momspi16S_tax[1:5, 1:3]
#> Kingdom Phylum Class
#> 807795 "Bacteria" "Firmicutes" "Bacilli"
#> 134726 "Bacteria" "Firmicutes" "Bacilli"
#> 215097 "Bacteria" "Proteobacteria" "Betaproteobacteria"
#> 542066 "Bacteria" "Actinobacteria" "Actinobacteria"
#> 851634 "Bacteria" "Bacteroidetes" "Bacteroidia"Load the 16S sample annotation data as a matrix, rows are samples, columns are annotations:
data("momspi16S_samp")
colnames(momspi16S_samp)
#> [1] "file_id" "md5" "size"
#> [4] "urls" "sample_id" "file_name"
#> [7] "subject_id" "sample_body_site" "visit_number"
#> [10] "subject_gender" "subject_race" "study_full_name"
#> [13] "project_name"
momspi16S_samp[1:5, 1:3]
#> file_id
#> EP003595_K10_MV1D e50d0c183689e4053ccb35f8b21a49f3
#> EP003595_K100_BRCD e50d0c183689e4053ccb35f8b2764c15
#> EP003595_K90_BCKD e50d0c183689e4053ccb35f8b21cadf2
#> EP003595_K90_BRCD e50d0c183689e4053ccb35f8b220d6c8
#> EP004835_K10_MCKD 6d91411d5ede0689305232cc77a70550
#> md5 size
#> EP003595_K10_MV1D 414907dbe12def470ca5395c6ac52947 45248
#> EP003595_K100_BRCD 90fa414ae826af7ed836e26d34244e2c 41152
#> EP003595_K90_BCKD 46516ead384e98552532f8980c9db06c 37056
#> EP003595_K90_BRCD eea4e103b75e2dc94e1a0657840e3edc 37056
#> EP004835_K10_MCKD 3b0b16edee96c3d3429408140c2a7f0e 105809
# Check if sample names match between the 16S and sample data
# all.equal(colnames(momspi16S_mtx), rownames(momspi16S_samp)) # Should be TRUEThe momspi16S function assembles those matrices into a phyloseq object.
momspi16S_phyloseq <- momspi16S()
momspi16S_phyloseq
#> phyloseq-class experiment-level object
#> otu_table() OTU Table: [ 7665 taxa and 9107 samples ]
#> sample_data() Sample Data: [ 9107 samples by 13 sample variables ]
#> tax_table() Taxonomy Table: [ 7665 taxa by 7 taxonomic ranks ]Here we have a phyloseq object containing the 16S rRNA seq data for 7665 taxa and 9107 samples.
2.2 Cytokine data
The MOMS-PI cytokine data can be loaded as a matrix, rownames are cytokine names, colnames are sample names:
data("momspiCyto_mtx")
momspiCyto_mtx[1:5, 1:5]
#> EP004835_K10_MVAX EP004835_K20_MVAX EP004835_K30_MVAX
#> Eotaxin 68.05 60.08 37.84
#> FGF 265.45 -2.00 -2.00
#> G-CSF 624.79 2470.70 -2.00
#> GM-CSF 51.39 -2.00 42.01
#> IFN-g 164.58 193.22 184.53
#> EP004835_K40_MVAX EP004835_K50_MVAX
#> Eotaxin 31.87 23.75
#> FGF 108.76 78.31
#> G-CSF -2.00 -2.00
#> GM-CSF 28.75 -2.00
#> IFN-g 185.97 116.39The cytokine data has 29 rows and 1396 columns.
Load the cytokine sample annotation data as a matrix, rows are samples, columns are annotations:
data("momspiCyto_samp")
colnames(momspiCyto_samp)
#> [1] "file_id" "md5" "size"
#> [4] "urls" "sample_id" "file_name"
#> [7] "subject_id" "sample_body_site" "visit_number"
#> [10] "subject_gender" "subject_race" "study_full_name"
#> [13] "project_name"
momspiCyto_samp[1:5, 1:5]
#> file_id
#> EP004835_K10_MVAX 31b14547707f47bb99f1e1ab41000829
#> EP004835_K20_MVAX 31b14547707f47bb99f1e1ab41000b86
#> EP004835_K30_MVAX 31b14547707f47bb99f1e1ab41001aa5
#> EP004835_K40_MVAX 31b14547707f47bb99f1e1ab4100223f
#> EP004835_K50_MVAX 31b14547707f47bb99f1e1ab41002bf4
#> md5 size
#> EP004835_K10_MVAX 006d9076e88889e231fed6866f8743d8 409
#> EP004835_K20_MVAX 6cbbebbc9da2efdcf7280f3ebd00a897 409
#> EP004835_K30_MVAX f8c5541931022f8a32ed511d7ceeee75 401
#> EP004835_K40_MVAX a8478ba7c7bd1bd721fdc570e8879139 394
#> EP004835_K50_MVAX 36a8c5ba1490ba7132e2bfaaaf391283 392
#> urls
#> EP004835_K10_MVAX fasp://aspera.ihmpdcc.org/ptb/cytokine/host/analysis/EP004835_K10_MVAX.CytokineProfile.txt
#> EP004835_K20_MVAX fasp://aspera.ihmpdcc.org/ptb/cytokine/host/analysis/EP004835_K20_MVAX.CytokineProfile.txt
#> EP004835_K30_MVAX fasp://aspera.ihmpdcc.org/ptb/cytokine/host/analysis/EP004835_K30_MVAX.CytokineProfile.txt
#> EP004835_K40_MVAX fasp://aspera.ihmpdcc.org/ptb/cytokine/host/analysis/EP004835_K40_MVAX.CytokineProfile.txt
#> EP004835_K50_MVAX fasp://aspera.ihmpdcc.org/ptb/cytokine/host/analysis/EP004835_K50_MVAX.CytokineProfile.txt
#> sample_id
#> EP004835_K10_MVAX 6791a23c10c0086065c25315502e9555
#> EP004835_K20_MVAX 6791a23c10c0086065c25315502e9854
#> EP004835_K30_MVAX 6791a23c10c0086065c25315502e9908
#> EP004835_K40_MVAX 6791a23c10c0086065c25315502e9e9d
#> EP004835_K50_MVAX 6791a23c10c0086065c25315502ea468The function momspiCytokines will make a SummarizedExperiment containing cytokine data
momspiCyto <- momspiCytokines()
momspiCyto
#> class: SummarizedExperiment
#> dim: 29 1396
#> metadata(0):
#> assays(1): ''
#> rownames(29): Eotaxin FGF ... FGF basic IL-17
#> rowData names(1): cytokine
#> colnames(1396): EP004835_K10_MVAX EP004835_K20_MVAX ...
#> EP996091_K40_MVAX EP996091_K60_MVAX
#> colData names(13): file_id md5 ... study_full_name project_nameThe cytokine data contains data for 29 cytokines over 1396 samples.
2.3 MultiAssayExperiment
We can construct a MultiAssayExperiment that will contain 16S and cytokine data for the common samples. Use the momspiMultiAssay function.
momspiMA <- momspiMultiAssay()
momspiMA
#> A MultiAssayExperiment object of 2 listed
#> experiments with user-defined names and respective classes.
#> Containing an ExperimentList class object of length 2:
#> [1] 16S: matrix with 7665 rows and 9107 columns
#> [2] cytokines: matrix with 29 rows and 1396 columns
#> Features:
#> experiments() - obtain the ExperimentList instance
#> colData() - the primary/phenotype DFrame
#> sampleMap() - the sample availability DFrame
#> `$`, `[`, `[[` - extract colData columns, subset, or experiment
#> *Format() - convert into a long or wide DFrame
#> assays() - convert ExperimentList to a SimpleList of matrices2.3.1 Subsetting the MultiAssayExperiment object
The 16S rRNA data can be extracted from the MultiAssayExperiment object as follows.
rRNA <- momspiMA[[1L]]Or the cytokines data like so.
cyto <- momspiMA[[2L]]The sample data can be accessed with the colData command.
colData(momspiMA) %>% head()
#> DataFrame with 6 rows and 12 columns
#> file_id
#> <character>
#> EP003595_K10_MV1D e50d0c183689e4053ccb35f8b21a49f3
#> EP003595_K100_BRCD e50d0c183689e4053ccb35f8b2764c15
#> EP003595_K90_BCKD e50d0c183689e4053ccb35f8b21cadf2
#> EP003595_K90_BRCD e50d0c183689e4053ccb35f8b220d6c8
#> EP004835_K10_MCKD 6d91411d5ede0689305232cc77a70550
#> EP004835_K10_MRCD 6d91411d5ede0689305232cc77a4dd86
#> md5 size
#> <character> <integer>
#> EP003595_K10_MV1D 414907dbe12def470ca5395c6ac52947 45248
#> EP003595_K100_BRCD 90fa414ae826af7ed836e26d34244e2c 41152
#> EP003595_K90_BCKD 46516ead384e98552532f8980c9db06c 37056
#> EP003595_K90_BRCD eea4e103b75e2dc94e1a0657840e3edc 37056
#> EP004835_K10_MCKD 3b0b16edee96c3d3429408140c2a7f0e 105809
#> EP004835_K10_MRCD 5825f20630620036d9bea05646b1c90d 128316
#> urls
#> <character>
#> EP003595_K10_MV1D http://downloads.hmpdacc.org/ihmp/ptb/genome/microbiome/16s/analysis/hmqcp/EP003595_K10_MV1D.otu_table.biom,fasp://aspera.ihmpdcc.org/ptb/genome/microbiome/16s/analysis/hmqcp/EP003595_K10_MV1D.otu_table.biom
#> EP003595_K100_BRCD http://downloads.hmpdacc.org/ihmp/ptb/genome/microbiome/16s/analysis/hmqcp/EP003595_K100_BRCD.otu_table.biom,fasp://aspera.ihmpdcc.org/ptb/genome/microbiome/16s/analysis/hmqcp/EP003595_K100_BRCD.otu_table.biom
#> EP003595_K90_BCKD http://downloads.hmpdacc.org/ihmp/ptb/genome/microbiome/16s/analysis/hmqcp/EP003595_K90_BCKD.otu_table.biom,fasp://aspera.ihmpdcc.org/ptb/genome/microbiome/16s/analysis/hmqcp/EP003595_K90_BCKD.otu_table.biom
#> EP003595_K90_BRCD http://downloads.hmpdacc.org/ihmp/ptb/genome/microbiome/16s/analysis/hmqcp/EP003595_K90_BRCD.otu_table.biom,fasp://aspera.ihmpdcc.org/ptb/genome/microbiome/16s/analysis/hmqcp/EP003595_K90_BRCD.otu_table.biom
#> EP004835_K10_MCKD http://downloads.hmpdacc.org/ihmp/ptb/genome/microbiome/16s/analysis/hmqcp/EP004835_K10_MCKD.otu_table.biom,fasp://aspera.ihmpdcc.org/ptb/genome/microbiome/16s/analysis/hmqcp/EP004835_K10_MCKD.otu_table.biom
#> EP004835_K10_MRCD http://downloads.hmpdacc.org/ihmp/ptb/genome/microbiome/16s/analysis/hmqcp/EP004835_K10_MRCD.otu_table.biom,fasp://aspera.ihmpdcc.org/ptb/genome/microbiome/16s/analysis/hmqcp/EP004835_K10_MRCD.otu_table.biom
#> sample_id
#> <character>
#> EP003595_K10_MV1D 8938ac880f194a32bc6b736c15714d95
#> EP003595_K100_BRCD 8938ac880f194a32bc6b736c15717f36
#> EP003595_K90_BCKD 8938ac880f194a32bc6b736c157154e4
#> EP003595_K90_BRCD 8938ac880f194a32bc6b736c157164d6
#> EP004835_K10_MCKD e50d0c183689e4053ccb35f8b2de153a
#> EP004835_K10_MRCD e50d0c183689e4053ccb35f8b2de1d91
#> subject_id sample_body_site
#> <character> <character>
#> EP003595_K10_MV1D 858ed4564f11795ec13dda4c10a0d657 vagina
#> EP003595_K100_BRCD 858ed4564f11795ec13dda4c10a0d657 rectum
#> EP003595_K90_BCKD 858ed4564f11795ec13dda4c10a0d657 buccal mucosa
#> EP003595_K90_BRCD 858ed4564f11795ec13dda4c10a0d657 rectum
#> EP004835_K10_MCKD e50d0c183689e4053ccb35f8b29d98c8 buccal mucosa
#> EP004835_K10_MRCD e50d0c183689e4053ccb35f8b29d98c8 rectum
#> visit_number subject_gender subject_race study_full_name
#> <integer> <character> <character> <character>
#> EP003595_K10_MV1D 1 female unknown momspi
#> EP003595_K100_BRCD 10 female unknown momspi
#> EP003595_K90_BCKD 9 female unknown momspi
#> EP003595_K90_BRCD 9 female unknown momspi
#> EP004835_K10_MCKD 1 female unknown momspi
#> EP004835_K10_MRCD 1 female unknown momspi
#> project_name
#> <character>
#> EP003595_K10_MV1D Integrative Human Microbiome Project
#> EP003595_K100_BRCD Integrative Human Microbiome Project
#> EP003595_K90_BCKD Integrative Human Microbiome Project
#> EP003595_K90_BRCD Integrative Human Microbiome Project
#> EP004835_K10_MCKD Integrative Human Microbiome Project
#> EP004835_K10_MRCD Integrative Human Microbiome ProjectTo extract only the samples with both 16S and cytokine data we can use the intersectColumns function.
completeMA <- intersectColumns(momspiMA)
#> harmonizing input:
#> removing 2 sampleMap rows with 'colname' not in colnames of experiments
completeMA
#> A MultiAssayExperiment object of 2 listed
#> experiments with user-defined names and respective classes.
#> Containing an ExperimentList class object of length 2:
#> [1] 16S: matrix with 7665 rows and 0 columns
#> [2] cytokines: matrix with 29 rows and 0 columns
#> Features:
#> experiments() - obtain the ExperimentList instance
#> colData() - the primary/phenotype DFrame
#> sampleMap() - the sample availability DFrame
#> `$`, `[`, `[[` - extract colData columns, subset, or experiment
#> *Format() - convert into a long or wide DFrame
#> assays() - convert ExperimentList to a SimpleList of matrices3 IBD
Load 16S data as a matrix, rows are SILVA IDs, columns are sample names:
data("IBD16S_mtx")
IBD16S_mtx[1:5, 1:5]
#> 206534 206536 206538 206547 206548
#> IP8BSoli 0 0 0 0 0
#> UncTepi3 0 0 0 0 0
#> Unc004ii 0 0 0 0 0
#> Unc00re8 3 3 0 0 0
#> Unc018j2 0 0 0 0 0Load the SILVA taxonomy table as a matrix, rows are SILVA IDs, columns are taxonomic ranks:
data("IBD16S_tax")
colnames(IBD16S_tax)
#> [1] "Kingdom" "Phylum" "Class" "Order" "Family" "Genus"
IBD16S_tax[1:5, 1:5]
#> Kingdom Phylum Class
#> IP8BSoli "Bacteria" "Proteobacteria" "Alphaproteobacteria"
#> UncTepi3 "Bacteria" "Proteobacteria" "Betaproteobacteria"
#> Unc004ii "Bacteria" "Firmicutes" "Clostridia"
#> Unc00re8 "Bacteria" "Bacteroidetes" "Bacteroidia"
#> Unc018j2 "Bacteria" "Firmicutes" "Clostridia"
#> Order Family
#> IP8BSoli "Rhodospirillales" "Acetobacteraceae"
#> UncTepi3 "Burkholderiales" "Comamonadaceae"
#> Unc004ii "Clostridiales" "Christensenellaceae"
#> Unc00re8 "Bacteroidales" "Prevotellaceae"
#> Unc018j2 "Clostridiales" "FamilyXIII"Load the 16S sample annotation data as a matrix, rows are samples, columns are annotations:
data("IBD16S_samp")
colnames(IBD16S_samp) %>% head()
#> [1] "Project" "sample_id" "subject_id" "site_sub_coll"
#> [5] "data_type" "week_num"
IBD16S_samp[1:5, 1:5]
#> Project sample_id subject_id site_sub_coll data_type
#> 206534 M2008CSC3_BP 206534 M2008 M2008CSC3 biopsy_16S
#> 206536 M2008CSC1_BP 206536 M2008 M2008CSC1 biopsy_16S
#> 206538 M2008CSC2_BP 206538 M2008 M2008CSC2 biopsy_16S
#> 206547 M2014CSC2_BP 206547 M2014 M2014CSC2 biopsy_16S
#> 206548 M2014CSC1_BP 206548 M2014 M2014CSC1 biopsy_16SThe IBD phyloseq object can be loaded as follows.
IBD <- IBD16S()
IBD
#> phyloseq-class experiment-level object
#> otu_table() OTU Table: [ 982 taxa and 178 samples ]
#> sample_data() Sample Data: [ 178 samples by 490 sample variables ]
#> tax_table() Taxonomy Table: [ 982 taxa by 6 taxonomic ranks ]4 T2D
Load 16S data as a matrix, rows are Greengene IDs, columns are sample names:
data("T2D16S_mtx")
T2D16S_mtx[1:5, 1:5]
#> HMP2_J00825_1_ST_T0_B0_0120_ZN9YTFN-01_AA31J
#> 198059 4
#> 985239 2
#> 840914 256
#> 174924 7
#> 180898 2
#> HMP2_J00826_1_ST_T0_B0_0120_ZN9YTFN-1011_AA31J
#> 198059 0
#> 985239 1
#> 840914 5870
#> 174924 2
#> 180898 0
#> HMP2_J00827_1_ST_T0_B0_0120_ZN9YTFN-1012_AA31J
#> 198059 2
#> 985239 0
#> 840914 2759
#> 174924 0
#> 180898 0
#> HMP2_J00828_1_ST_T0_B0_0120_ZN9YTFN-1013_AA31J
#> 198059 0
#> 985239 0
#> 840914 6733
#> 174924 2
#> 180898 0
#> HMP2_J00829_1_ST_T0_B0_0120_ZN9YTFN-1014_AA31J
#> 198059 2
#> 985239 1
#> 840914 3028
#> 174924 4
#> 180898 2Load the Greengenes taxonomy table as a matrix, rows are Greengene IDs, columns are taxonomic ranks:
data("T2D16S_tax")
colnames(T2D16S_tax)
#> [1] "Kingdom" "Phylum" "Class" "Order" "Family" "Genus" "Species"
T2D16S_tax[1:5, 1:5]
#> Kingdom Phylum Class Order
#> 198059 "Bacteria" "Firmicutes" "Clostridia" "Clostridiales"
#> 985239 "Bacteria" "Firmicutes" "Clostridia" "Clostridiales"
#> 840914 "Bacteria" "Bacteroidetes" "Bacteroidia" "Bacteroidales"
#> 174924 "Bacteria" "Firmicutes" "Clostridia" "Clostridiales"
#> 180898 "Bacteria" "Firmicutes" "Clostridia" "Clostridiales"
#> Family
#> 198059 "Lachnospiraceae"
#> 985239 "Lachnospiraceae"
#> 840914 "Prevotellaceae"
#> 174924 "Ruminococcaceae"
#> 180898 "Lachnospiraceae"Load the 16S sample annotation data as a matrix, rows are samples, columns are annotations:
data("T2D16S_samp")
colnames(T2D16S_samp)
#> [1] "file_id" "md5" "size"
#> [4] "urls" "sample_id" "file_name"
#> [7] "subject_id" "sample_body_site" "visit_number"
#> [10] "subject_gender" "subject_race" "study_full_name"
#> [13] "project_name"
T2D16S_samp[1:5, 1:5]
#> file_id
#> HMP2_J00825_1_ST_T0_B0_0120_ZN9YTFN-01_AA31J 6cca313bce90a4392c3d5cf23fdb43db
#> HMP2_J00826_1_ST_T0_B0_0120_ZN9YTFN-1011_AA31J 6cca313bce90a4392c3d5cf23fda16c7
#> HMP2_J00827_1_ST_T0_B0_0120_ZN9YTFN-1012_AA31J 6cca313bce90a4392c3d5cf23fd946a5
#> HMP2_J00828_1_ST_T0_B0_0120_ZN9YTFN-1013_AA31J 6cca313bce90a4392c3d5cf23fdb6783
#> HMP2_J00829_1_ST_T0_B0_0120_ZN9YTFN-1014_AA31J 6cca313bce90a4392c3d5cf23fdaa426
#> md5
#> HMP2_J00825_1_ST_T0_B0_0120_ZN9YTFN-01_AA31J 52ae217fbb3b5888906574e7c0eda10a
#> HMP2_J00826_1_ST_T0_B0_0120_ZN9YTFN-1011_AA31J f7f85c444703ec4259d5358e2662fe72
#> HMP2_J00827_1_ST_T0_B0_0120_ZN9YTFN-1012_AA31J 4b9bf6bf9cc0e62372ecb73ebd303cae
#> HMP2_J00828_1_ST_T0_B0_0120_ZN9YTFN-1013_AA31J 30251158c0b00b8d15f0027786cad84d
#> HMP2_J00829_1_ST_T0_B0_0120_ZN9YTFN-1014_AA31J 8584c0c7319c3ac7654a3cf022329f95
#> size
#> HMP2_J00825_1_ST_T0_B0_0120_ZN9YTFN-01_AA31J 96000
#> HMP2_J00826_1_ST_T0_B0_0120_ZN9YTFN-1011_AA31J 86000
#> HMP2_J00827_1_ST_T0_B0_0120_ZN9YTFN-1012_AA31J 83000
#> HMP2_J00828_1_ST_T0_B0_0120_ZN9YTFN-1013_AA31J 98000
#> HMP2_J00829_1_ST_T0_B0_0120_ZN9YTFN-1014_AA31J 89000
#> urls
#> HMP2_J00825_1_ST_T0_B0_0120_ZN9YTFN-01_AA31J fasp://aspera.ihmpdcc.org/t2d/genome/microbiome/16s/analysis/hmqcp/HMP2_J00825_1_ST_T0_B0_0120_ZN9YTFN-01_AA31J.biom
#> HMP2_J00826_1_ST_T0_B0_0120_ZN9YTFN-1011_AA31J fasp://aspera.ihmpdcc.org/t2d/genome/microbiome/16s/analysis/hmqcp/HMP2_J00826_1_ST_T0_B0_0120_ZN9YTFN-1011_AA31J.biom
#> HMP2_J00827_1_ST_T0_B0_0120_ZN9YTFN-1012_AA31J fasp://aspera.ihmpdcc.org/t2d/genome/microbiome/16s/analysis/hmqcp/HMP2_J00827_1_ST_T0_B0_0120_ZN9YTFN-1012_AA31J.biom
#> HMP2_J00828_1_ST_T0_B0_0120_ZN9YTFN-1013_AA31J fasp://aspera.ihmpdcc.org/t2d/genome/microbiome/16s/analysis/hmqcp/HMP2_J00828_1_ST_T0_B0_0120_ZN9YTFN-1013_AA31J.biom
#> HMP2_J00829_1_ST_T0_B0_0120_ZN9YTFN-1014_AA31J fasp://aspera.ihmpdcc.org/t2d/genome/microbiome/16s/analysis/hmqcp/HMP2_J00829_1_ST_T0_B0_0120_ZN9YTFN-1014_AA31J.biom
#> sample_id
#> HMP2_J00825_1_ST_T0_B0_0120_ZN9YTFN-01_AA31J 932d8fbc70ae8f856028b3f67c4ef6ac
#> HMP2_J00826_1_ST_T0_B0_0120_ZN9YTFN-1011_AA31J 932d8fbc70ae8f856028b3f67c4f0716
#> HMP2_J00827_1_ST_T0_B0_0120_ZN9YTFN-1012_AA31J 932d8fbc70ae8f856028b3f67c4f21b0
#> HMP2_J00828_1_ST_T0_B0_0120_ZN9YTFN-1013_AA31J 932d8fbc70ae8f856028b3f67c4f35bd
#> HMP2_J00829_1_ST_T0_B0_0120_ZN9YTFN-1014_AA31J 932d8fbc70ae8f856028b3f67c4f4fe6The T2D phyloseq object can be loaded like so.
T2D <- T2D16S()
T2D
#> phyloseq-class experiment-level object
#> otu_table() OTU Table: [ 12062 taxa and 2208 samples ]
#> sample_data() Sample Data: [ 2208 samples by 13 sample variables ]
#> tax_table() Taxonomy Table: [ 12062 taxa by 7 taxonomic ranks ]5 Features
5.1 Frequency Table Generation
The sample-level annotation data contianed in HMP2Data can be summarized using the table_two function. First, we need to make a list of the phyloseq and SummarizedExperiment objects which can then be entered into the table_two() table generating function.
list("MOMS-PI 16S" = momspi16S_phyloseq, "MOMS-PI Cytokines" = momspiCyto, "IBD 16S" = IBD, "T2D 16S" = T2D) %>% table_two()| N | % | N | % | N | % | N | % | |
|---|---|---|---|---|---|---|---|---|
| Body Site | ||||||||
| buccal mucosa | 3313 | 36.38 | 311 | 22.28 | 0 | 0 | 0 | 0 |
| cervix of uterus | 162 | 1.78 | 0 | 0 | 0 | 0 | 0 | 0 |
| feces | 765 | 8.4 | 0 | 0 | 0 | 0 | 1041 | 47.15 |
| rectum | 2679 | 29.42 | 0 | 0 | 0 | 0 | 0 | 0 |
| unknown | 146 | 1.6 | 0 | 0 | 0 | 0 | 0 | 0 |
| vagina | 2042 | 22.42 | 979 | 70.13 | 0 | 0 | 0 | 0 |
| blood cell | 0 | 0 | 106 | 7.59 | 0 | 0 | 0 | 0 |
| nasal cavity | 0 | 0 | 0 | 0 | 0 | 0 | 1167 | 52.85 |
| Sex | ||||||||
| male | 0 | 0 | 0 | 0 | 84 | 47.19 | 1248 | 56.52 |
| female | 9107 | 100 | 1396 | 100 | 94 | 52.81 | 947 | 42.89 |
| unknown | 0 | 0 | 0 | 0 | 0 | 0 | 13 | 0.59 |
| Race | ||||||||
| african american | 0 | 0 | 0 | 0 | 0 | 0 | 117 | 5.3 |
| asian | 0 | 0 | 0 | 0 | 0 | 0 | 235 | 10.64 |
| caucasian | 0 | 0 | 0 | 0 | 0 | 0 | 1657 | 75.05 |
| ethnic other | 0 | 0 | 0 | 0 | 0 | 0 | 73 | 3.31 |
| hispanic or latino | 0 | 0 | 0 | 0 | 0 | 0 | 126 | 5.71 |
| american indian or alaska native | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| unknown | 9107 | 100 | 1396 | 100 | 178 | 100 | 0 | 0 |
| total samples | 9107 | 100 | 1396 | 100 | 178 | 100 | 2208 | 100 |
We can also create a table of the breakdown of samples by visit number quantile.
list("MOMS-PI 16S" = momspi16S_phyloseq, "MOMS-PI Cytokines" = momspiCyto, "IBD 16S" = IBD, "T2D 16S" = T2D) %>% visit_table()| N | % | N | % | N | % | N | % | |
|---|---|---|---|---|---|---|---|---|
| First quantile | 8911 | 97.85 | 1126 | 80.66 | 163 | 91.57 | 1610 | 72.92 |
| Second quantile | 130 | 1.43 | 230 | 16.48 | 15 | 8.43 | 105 | 4.76 |
| Third quantile | 66 | 0.72 | 40 | 2.87 | NA | NA | 493 | 22.33 |
The patient-level characteristics can be summarized using the patient_table() function.
list("MOMS-PI 16S" = momspi16S_phyloseq, "MOMS-PI Cytokines" = momspiCyto, "IBD 16S" = IBD, "T2D 16S" = T2D) %>% patient_table()| N | % | N | % | N | % | N | % | |
|---|---|---|---|---|---|---|---|---|
| Sex | ||||||||
| male | 0 | 0 | 0 | 0 | 41 | 50.62 | 38 | 46.34 |
| female | 596 | 100 | 235 | 100 | 40 | 49.38 | 39 | 47.56 |
| unknown | 0 | 0 | 0 | 0 | 0 | 0 | 5 | 6.1 |
| Race | ||||||||
| african american | 0 | 0 | 0 | 0 | 0 | 0 | 4 | 4.88 |
| asian | 0 | 0 | 0 | 0 | 0 | 0 | 14 | 17.07 |
| caucasian | 0 | 0 | 0 | 0 | 0 | 0 | 52 | 63.41 |
| ethnic other | 0 | 0 | 0 | 0 | 0 | 0 | 9 | 10.98 |
| hispanic or latino | 0 | 0 | 0 | 0 | 0 | 0 | 3 | 3.66 |
| american indian or alaska native | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| unknown | 596 | 100 | 235 | 100 | 81 | 100 | 0 | 0 |
| Total patients | 596 | 100 | 235 | 100 | 81 | 100 | 82 | 100 |
5.2 Visits Histograms
Here we plot histograms of the number of samples at each visit for the data MOMS-PI 16S data. Note that there are 196 samples at visit numbers over 20.
# set up ggplots
p1 <- ggplot(momspi16S_samp, aes(x = visit_number)) +
geom_histogram(bins = 20, color = "#00BFC4", fill = "lightblue") +
xlim(c(0,20)) + xlab("Visit number") + ylab("Count")
# theme(panel.background = element_blank(), panel.grid = element_blank())
p1
We can plot the histogram of the number of samples at each visit for all studies. Note that the X-axis is capped at count of 40 for better visualization.
# make data.frame for plotting
plot_visits <- data.frame(study = c(rep("MOMS-PI Cytokines", nrow(momspiCyto_samp)),
rep("IBD 16S", nrow(IBD16S_samp)),
rep("T2D 16S", nrow(T2D16S_samp))),
visits = c(momspiCyto_samp$visit_number,
IBD16S_samp$visit_number,
T2D16S_samp$visit_number))
p2 <- ggplot(plot_visits, aes(x = visits, fill = study)) +
geom_histogram(position = "dodge", alpha = 0.7, bins = 30, color = "#00BFC4") + xlim(c(0, 40)) +
theme(legend.position = c(0.8, 0.8)) +
xlab("Visit number") + ylab("Count")
p2
Note that there are 677 samples at visit numbers over 40.
5.3 UpsetR plots
5.3.1 MOMS-PI 16S rRNA data
Here we plot the body sites which samples were taken from for each patient in the MOMS-PI 16S data.
# set up data.frame for UpSetR
momspi_upset <- aggregate(momspi16S_samp$sample_body_site, by = list(momspi16S_samp$subject_id), table)
tmp <- as.matrix(momspi_upset[, -1])
tmp <- (tmp > 0) *1
momspi_upset <- data.frame(patient = momspi_upset$Group.1, tmp)
# plot
upset(momspi_upset, order.by = 'freq', matrix.color = "blue", text.scale = 2)
5.3.2 MOMS-PI Cytokines data
Here we plot the body sites which samples were taken from for each patient in the MOMS-PI cytokines data.
# set up data.frame for UpSetR
momspiCyto_upset <- aggregate(momspiCyto_samp$sample_body_site, by = list(momspiCyto_samp$subject_id), table)
tmp <- as.matrix(momspiCyto_upset[, -1])
tmp <- (tmp > 0) *1
momspiCyto_upset <- data.frame(patient = momspiCyto_upset$Group.1, tmp)
# plot
upset(momspiCyto_upset, order.by = 'freq', matrix.color = "blue", text.scale = 2)
5.3.3 IBD data
The IBD patients only had samples taken from the feces.
5.3.4 T2D data
Here we plot the body sites which samples were taken from for each patient in the T2D 16S rRNA data.
# set up data.frame for UpSetR
T2D_upset <- aggregate(T2D16S_samp$sample_body_site, by = list(T2D16S_samp$subject_id), table)
tmp <- as.matrix(T2D_upset[, -1])
tmp <- (tmp > 0) *1
T2D_upset <- data.frame(patient = T2D_upset$Group.1, tmp)
# plot
upset(T2D_upset, order.by = 'freq', matrix.color = "blue", text.scale = 2)
6 Exporting Data to CSV Format
To our knowledge, R and Bioconductor provide the most and best methods for the analysis of microbiome data. However, we realize they are not the only analysis environments and wish to provide methods to export the data from HMP16SData to CSV format.
6.1 Preparing the data
For SummarizedExperiment objects, we will need to separate the data and metadata first before exportation. First, make a data.frame for participant data.
momspi_cytokines_participants <- colData(momspiCyto)
momspi_cytokines_participants[1:5, ]
#> DataFrame with 5 rows and 13 columns
#> file_id
#> <character>
#> EP004835_K10_MVAX 31b14547707f47bb99f1e1ab41000829
#> EP004835_K20_MVAX 31b14547707f47bb99f1e1ab41000b86
#> EP004835_K30_MVAX 31b14547707f47bb99f1e1ab41001aa5
#> EP004835_K40_MVAX 31b14547707f47bb99f1e1ab4100223f
#> EP004835_K50_MVAX 31b14547707f47bb99f1e1ab41002bf4
#> md5 size
#> <character> <integer>
#> EP004835_K10_MVAX 006d9076e88889e231fed6866f8743d8 409
#> EP004835_K20_MVAX 6cbbebbc9da2efdcf7280f3ebd00a897 409
#> EP004835_K30_MVAX f8c5541931022f8a32ed511d7ceeee75 401
#> EP004835_K40_MVAX a8478ba7c7bd1bd721fdc570e8879139 394
#> EP004835_K50_MVAX 36a8c5ba1490ba7132e2bfaaaf391283 392
#> urls
#> <character>
#> EP004835_K10_MVAX fasp://aspera.ihmpdcc.org/ptb/cytokine/host/analysis/EP004835_K10_MVAX.CytokineProfile.txt
#> EP004835_K20_MVAX fasp://aspera.ihmpdcc.org/ptb/cytokine/host/analysis/EP004835_K20_MVAX.CytokineProfile.txt
#> EP004835_K30_MVAX fasp://aspera.ihmpdcc.org/ptb/cytokine/host/analysis/EP004835_K30_MVAX.CytokineProfile.txt
#> EP004835_K40_MVAX fasp://aspera.ihmpdcc.org/ptb/cytokine/host/analysis/EP004835_K40_MVAX.CytokineProfile.txt
#> EP004835_K50_MVAX fasp://aspera.ihmpdcc.org/ptb/cytokine/host/analysis/EP004835_K50_MVAX.CytokineProfile.txt
#> sample_id file_name
#> <character> <character>
#> EP004835_K10_MVAX 6791a23c10c0086065c25315502e9555 EP004835_K10_MVAX
#> EP004835_K20_MVAX 6791a23c10c0086065c25315502e9854 EP004835_K20_MVAX
#> EP004835_K30_MVAX 6791a23c10c0086065c25315502e9908 EP004835_K30_MVAX
#> EP004835_K40_MVAX 6791a23c10c0086065c25315502e9e9d EP004835_K40_MVAX
#> EP004835_K50_MVAX 6791a23c10c0086065c25315502ea468 EP004835_K50_MVAX
#> subject_id sample_body_site
#> <character> <character>
#> EP004835_K10_MVAX e50d0c183689e4053ccb35f8b29d98c8 vagina
#> EP004835_K20_MVAX e50d0c183689e4053ccb35f8b29d98c8 vagina
#> EP004835_K30_MVAX e50d0c183689e4053ccb35f8b29d98c8 vagina
#> EP004835_K40_MVAX e50d0c183689e4053ccb35f8b29d98c8 vagina
#> EP004835_K50_MVAX e50d0c183689e4053ccb35f8b29d98c8 vagina
#> visit_number subject_gender subject_race study_full_name
#> <integer> <character> <character> <character>
#> EP004835_K10_MVAX 1 female unknown momspi
#> EP004835_K20_MVAX 2 female unknown momspi
#> EP004835_K30_MVAX 3 female unknown momspi
#> EP004835_K40_MVAX 4 female unknown momspi
#> EP004835_K50_MVAX 5 female unknown momspi
#> project_name
#> <character>
#> EP004835_K10_MVAX Integrative Human Microbiome Project
#> EP004835_K20_MVAX Integrative Human Microbiome Project
#> EP004835_K30_MVAX Integrative Human Microbiome Project
#> EP004835_K40_MVAX Integrative Human Microbiome Project
#> EP004835_K50_MVAX Integrative Human Microbiome ProjectThen we need to pull out the data matrix.
momspi_cytokines <- assay(momspiCyto)
momspi_cytokines[1:5, 1:5]
#> EP004835_K10_MVAX EP004835_K20_MVAX EP004835_K30_MVAX
#> Eotaxin 68.05 60.08 37.84
#> FGF 265.45 -2.00 -2.00
#> G-CSF 624.79 2470.70 -2.00
#> GM-CSF 51.39 -2.00 42.01
#> IFN-g 164.58 193.22 184.53
#> EP004835_K40_MVAX EP004835_K50_MVAX
#> Eotaxin 31.87 23.75
#> FGF 108.76 78.31
#> G-CSF -2.00 -2.00
#> GM-CSF 28.75 -2.00
#> IFN-g 185.97 116.39For phyloseq objects the data, metadata, and taxonomy can be separated as follows. First, pull out metadata.
momspi_16S_participants <- sample_data(momspi16S_phyloseq)Next, we can save the counts data as a matrix.
momspi16s_data <- as.matrix(otu_table(momspi16S_phyloseq))Finally, the taxonomy table can be extracted.
momspi16s_taxa <- tax_table(momspi16S_phyloseq) %>% as.data.frame()6.2 Save data as CSV
The data can be saved as .csv files as follows.
library(readr)
write_csv(data.frame(file_id = rownames(momspi_cytokines_participants), momspi_cytokines_participants), "momspi_cytokines_participants.csv")
write_csv(data.frame(momspi_cytokines), "momspi_cytokines.csv")7 Session Info
sessionInfo()
#> R Under development (unstable) (2019-10-24 r77329)
#> Platform: x86_64-pc-linux-gnu (64-bit)
#> Running under: Ubuntu 18.04.3 LTS
#>
#> Matrix products: default
#> BLAS: /home/biocbuild/bbs-3.11-bioc/R/lib/libRblas.so
#> LAPACK: /home/biocbuild/bbs-3.11-bioc/R/lib/libRlapack.so
#>
#> locale:
#> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
#> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=C
#> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
#> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
#> [9] LC_ADDRESS=C LC_TELEPHONE=C
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
#>
#> attached base packages:
#> [1] parallel stats4 stats graphics grDevices utils datasets
#> [8] methods base
#>
#> other attached packages:
#> [1] UpSetR_1.4.0 ggplot2_3.2.1
#> [3] dplyr_0.8.3 MultiAssayExperiment_1.13.0
#> [5] SummarizedExperiment_1.17.0 DelayedArray_0.13.0
#> [7] BiocParallel_1.21.0 matrixStats_0.55.0
#> [9] Biobase_2.47.0 GenomicRanges_1.39.1
#> [11] GenomeInfoDb_1.23.0 IRanges_2.21.1
#> [13] S4Vectors_0.25.0 BiocGenerics_0.33.0
#> [15] phyloseq_1.31.0 HMP2Data_1.1.0
#> [17] BiocStyle_2.15.0
#>
#> loaded via a namespace (and not attached):
#> [1] colorspace_1.4-1 XVector_0.27.0
#> [3] rstudioapi_0.10 bit64_0.9-7
#> [5] interactiveDisplayBase_1.25.0 AnnotationDbi_1.49.0
#> [7] xml2_1.2.2 codetools_0.2-16
#> [9] splines_4.0.0 knitr_1.25
#> [11] zeallot_0.1.0 ade4_1.7-13
#> [13] jsonlite_1.6 cluster_2.1.0
#> [15] dbplyr_1.4.2 shiny_1.4.0
#> [17] BiocManager_1.30.9 readr_1.3.1
#> [19] compiler_4.0.0 httr_1.4.1
#> [21] backports_1.1.5 assertthat_0.2.1
#> [23] Matrix_1.2-17 fastmap_1.0.1
#> [25] lazyeval_0.2.2 later_1.0.0
#> [27] htmltools_0.4.0 tools_4.0.0
#> [29] igraph_1.2.4.1 gtable_0.3.0
#> [31] glue_1.3.1 GenomeInfoDbData_1.2.2
#> [33] reshape2_1.4.3 rappdirs_0.3.1
#> [35] Rcpp_1.0.2 vctrs_0.2.0
#> [37] Biostrings_2.55.0 multtest_2.43.0
#> [39] ape_5.3 nlme_3.1-141
#> [41] ExperimentHub_1.13.0 iterators_1.0.12
#> [43] xfun_0.10 stringr_1.4.0
#> [45] rvest_0.3.4 mime_0.7
#> [47] AnnotationHub_2.19.0 zlibbioc_1.33.0
#> [49] MASS_7.3-51.4 scales_1.0.0
#> [51] hms_0.5.2 promises_1.1.0
#> [53] biomformat_1.15.0 rhdf5_2.31.0
#> [55] yaml_2.2.0 curl_4.2
#> [57] gridExtra_2.3 memoise_1.1.0
#> [59] stringi_1.4.3 RSQLite_2.1.2
#> [61] highr_0.8 BiocVersion_3.11.0
#> [63] foreach_1.4.7 permute_0.9-5
#> [65] rlang_0.4.1 pkgconfig_2.0.3
#> [67] bitops_1.0-6 evaluate_0.14
#> [69] lattice_0.20-38 purrr_0.3.3
#> [71] Rhdf5lib_1.9.0 labeling_0.3
#> [73] bit_1.1-14 tidyselect_0.2.5
#> [75] plyr_1.8.4 magrittr_1.5
#> [77] bookdown_0.14 R6_2.4.0
#> [79] DBI_1.0.0 pillar_1.4.2
#> [81] withr_2.1.2 mgcv_1.8-30
#> [83] survival_2.44-1.1 RCurl_1.95-4.12
#> [85] tibble_2.1.3 crayon_1.3.4
#> [87] BiocFileCache_1.11.0 rmarkdown_1.16
#> [89] grid_4.0.0 data.table_1.12.6
#> [91] blob_1.2.0 vegan_2.5-6
#> [93] digest_0.6.22 webshot_0.5.1
#> [95] xtable_1.8-4 httpuv_1.5.2
#> [97] munsell_0.5.0 viridisLite_0.3.0
#> [99] kableExtra_1.1.0