1 Quick Start

1.1 Package Installation and Loading

1.1.1 Download and Installation

To install the latest version of esATAC, you will need to be using the latest version of R. esATAC is part of Bioconductor project starting from Bioc 3.6 built on R 3.4. Please check your current Bioconductor version and R version first. Similar to other bioconductor package, you can download install esATAC and all its dependencies software like this:

if (!requireNamespace("BiocManager", quietly=TRUE))
    install.packages("BiocManager")
BiocManager::install("esATAC")

NOTE: We recommend to use this package in RStudio. Or you have to install pandoc yourself if you use R terminal

Acquiring more installation detail information for Linux, Windows and macOS, you can visit esATAC Installation Tutorial.

Note: By default, esATAC will build bowtie2 index if there is no bowtie2 index, which may take several hours with single thread. If you want to download or use your own bowtie2 index instead, please see section 1.4

NOTE: Annotation Packages for specific genomes are shown below:

genome BSgenome TxDb OrgDb
hg19 BSgenome.Hsapiens.UCSC.hg19 TxDb.Hsapiens.UCSC.hg19.knownGene org.Hs.eg.db
hg38 BSgenome.Hsapiens.UCSC.hg38 TxDb.Hsapiens.UCSC.hg38.knownGene org.Hs.eg.db
mm9 BSgenome.Mmusculus.UCSC.mm9 TxDb.Mmusculus.UCSC.mm9.knownGene org.Mm.eg.db
mm10 BSgenome.Mmusculus.UCSC.mm10 TxDb.Mmusculus.UCSC.mm10.knownGene org.Mm.eg.db

They will not be downloaded and installed during executing BiocManager::install("esATAC") because these package file are very big. They will be downloaded and installed according to the genome parameter when calling the preset pipeline for the first time.

1.1.2 Loading

Just like other R packages, you need to load esATAC like this each time before using the package.

library(esATAC)

1.2 Datasets

Most of the test datasets in this package (esATAC/extdata/) are generated from GEO: SRR891271 (from GSE47753)[7]. The data is ATAC-seq paired end sequencing for GM12878 cell line. We random sampling 20000 mapped fragments from chr20 and rebuild raw paired-end FASTQ files(file names with chr20 prefix). We also subsample the reads in SAM file and peak calling BED result files. Besides, the files in “uzmg” and “bt2” are the test files from AdapterRemoval and Bowtie2. For detail, you can read the subsequent sections.

1.3 Starting from Scratch

esATAC provides an easy-to-use entry, you only need to provide your ATAC-seq sequencing files (FASTQ format), and assign the spaces and genome assembly, it will do everything for you.

1.3.1 For Case-control Analysis

The R scripts below are ready to run. No more edit is needed.

Customize the code commented with “MODIFY” if you need to run on your own data.

Need to be prepared for your own data:

  • FASTQ files paths (may be .gz or .bz2 zipped FASTQ files)
    • case:
      • fastqInput1: mate 1 FASTQ file(s)
      • fastqInput2: mate 2 FASTQ file(s)
    • control:
      • fastqInput1: mate 1 FASTQ file(s)
      • fastqInput2: mate 2 FASTQ file(s)
  • genome: genome version may be one of these
    • hg19
    • hg38
    • mm9
    • mm10
  • refdir:(optional) Directory for installing genome reference and storage for reuse. Default: “./esATAC_pipeline/refdir” will be created if not exist.
  • tmpdir:(optional) Directory for intermediate files and results storage. Default: “./esATAC_pipeline/intermediate_results” will be created if not exist.
  • threads:(optional) The max threads allowed to be created. Default: 2.

For other genomes, you can built your own pipeline through Customized Pipeline presently. More genome will be supported in the future.

Here, we show an simple runnable example for case-control analysis. Case and control test sample are both paired-end data. Each of them contains two gzipped FASTQ files. The test file paths can are obtained like this: system.file(package="esATAC", "extdata", "chr20_1.1.fq.gz"). They are under folder path “R library path/esATAC/extdata/”.

library(esATAC)

# call pipeline
# all human motif in JASPAR will be processed
conclusion <- 
    atacPipe2(
# MODIFY: Change these paths to your own case files!
# e.g. fastqInput1 = "your/own/data/path.fastq"
        case=list(fastqInput1 = system.file(package="esATAC", "extdata", "chr20_1.1.fq.gz"),
                  fastqInput2 = system.file(package="esATAC", "extdata", "chr20_2.1.fq.gz")), 
# MODIFY: Change these paths to your own control files!
# e.g. fastqInput1 = "your/own/data/path.fastq"
        control=list(fastqInput1 = system.file(package="esATAC", "extdata", "chr20_1.2.fq.bz2"),
                     fastqInput2 = system.file(package="esATAC", "extdata", "chr20_2.2.fq.bz2")),
# MODIFY: Change this path to an permanent path to be used in future!
#       refdir = "./esATAC_pipeline/refdir",
#       tmpdir = "./esATAC_pipeline/intermediate_results", 
# MODIFY: Set the genome for your data
        genome = "hg19")

Note: By default, esATAC will build bowtie2 index if there is no bowtie2 index under the path set by argument refdir, which may take several hours. If you want to download bowtie2 index instead, please see section 1.4

Note: By default, esATAC will perform footprint analysis for all the TF motif PWM matrix in JASPAR database. This step may take a few hours to 2-days for human genome analysis depends on your hardware. If you only want to analyze a specific motif or your own PWMs, please see section 1.5

Note: For replicates sample, see section 1.8: Gallery for Analysis Result of Real ATAC-seq Dataset

The reference data will be installed in “./esATAC_pipeline/refdir/” and all of temporary data and result will be stored under “./esATAC_pipeline/intermediate_results”. The final brief results like HTML report will be stored under “./esATAC_pipeline/esATAC_results” and “./esATAC_pipeline/esATAC_report”

If you run the scripts above without modification, you are able to obtain default HTML example report.

Real data test

Download raw data from GEO(assertion number GSE88987 - GSM2356780: SRR4435490.sra) and (GSM2356795: SRR4435505.sra)).And then, Use NCBI SRA Toolkit to extract fastq files with command like fastq-dump --split-3 SRR44354XX.sra. Four files will be generated (SRR4435490_1.fastq, SRR4435490_2.fastq, SRR4435505_1.fastq, SRR4435505_2.fastq). Modify and run the scripts above like this example scripts, you will obtain HTML report.

1.3.2 For Single Sample Analysis

The R scripts below are ready to run. No more edit is needed.

Customize the code commented with “MODIFY” if you need to run on your own data.

Need to be prepared for your own data:

  • FASTQ files paths (may be .gz or .bz2 zipped FASTQ files)
    • fastqInput1: mate 1 FASTQ file(s)
    • fastqInput2: mate 2 FASTQ file(s)
  • genome: may be one of these
    • hg19
    • hg38
    • mm9
    • mm10
  • refdir:(optional) Directory for installing genome reference and storage for reuse. Default: “./esATAC_pipeline/refdir” will be created if not exist.
  • tmpdir:(optional) Directory for intermediate files and results storage. Default: “./esATAC_pipeline/intermediate_results” will be created if not exist.
  • threads:(optional) The max threads allowed to be created. Default: 2.

For other genomes, you can built your own pipeline through Customized Pipeline presently. More genome will be supported in the future.

Here, we show an simple runnable example for single sample analysis. We just use case data in case-control section.

library(esATAC)

# call pipeline
# for overall example(all human motif in JASPAR will be processed)
conclusion <- 
    atacPipe(
# MODIFY: Change these paths to your own case files!
# e.g. fastqInput1 = "your/own/data/path.fastq"
        fastqInput1 = system.file(package="esATAC", "extdata", "chr20_1.1.fq.gz"),
        fastqInput2 = system.file(package="esATAC", "extdata", "chr20_2.1.fq.gz"),
# MODIFY: Change this path to an permanent path to be used in future!
#       refdir = "./esATAC_pipeline/refdir",
#       tmpdir = "./esATAC_pipeline/intermediate_results", 
# MODIFY: Set the genome for your data
        genome = "hg19")

Note: By default, esATAC will build bowtie2 index if there is no bowtie2 index under the path set by argument refdir, which may take several hours. If you want to download bowtie2 index instead, please see section 1.4

Note: By default, esATAC will perform footprint analysis for all the TF motif PWM matrix in JASPAR database. This step may take a few hours to 2-days for human genome analysis depends on your hardware. If you only want to analyze a specific motif or your own PWMs, please see section 1.5

Note: For replicates sample, see section 1.8: Gallery for Analysis Result of Real ATAC-seq Dataset

The reference data will be installed in “./esATAC_pipeline/refdir/” and all of temporary data and result will be stored under “./esATAC_pipeline/intermediate_results”. The final brief results like HTML report will be stored under “./esATAC_pipeline/esATAC_results”, “./esATAC_pipeline/esATAC_report”

If you run the scripts above without modification, you are able to obtain default HTML Example report

Real data test

Download raw data from GEO (assersion number GSE47753 - GSM1155957(SRR891268.sra)). And then, Use NCBI SRA Toolkit to extract fastq files with command like fastq-dump --split-3 SRR8912XX.sra. Two files will be generated (SRR891268_1.fastq, SRR891268_2.fastq). Modify the scripts above like this example scripts, you will obtain HTML report for GSM1155957 data.

esATAC will download the genome sequence and annotation files, build bowtie2 index, mapping the reads, do the quality control analysis, find peak regions, perform GO analysis and motif enrichment analysis, etc. automatically. Finally, you will get an report file in html format included to the analysis results.

1.4 Already Have Genome Bowtie2 Index

Build2 bowtie index may take some time. If you already have bowtie2 index files or you want to download instead of building, you can let esATAC skip the steps by renaming them following the format (genome+suffix) and put them in reference installation path (refdir).

Example: hg19 bowtie2 index files

  • hg19.1.bt2
  • hg19.2.bt2
  • hg19.3.bt2
  • hg19.4.bt2
  • hg19.rev.1.bt2
  • hg19.rev.2.bt2

bowtie2 index download path:

ftp://ftp.ccb.jhu.edu/pub/data/bowtie2_indexes

Modify the “refdir” in Starting from Scratch, you can run the example code.

1.5 Search a Specific Motif Set

By default, esATAC will perform footprint analysis for all the TF motif PWM matrix in JASPAR database. This step may take a few hours to 2-days for human genome analysis depends on your hardware. If you only want to analyze a specific motif or your own PWMs, you can simple do it like this:

1.5.1 Case-control

### 
library(esATAC)
conclusion2 <- 
    atacPipe2(
        case=list(fastqInput1 = system.file(package="esATAC", "extdata", "chr20_1.1.fq.gz"),
                  fastqInput2 = system.file(package="esATAC", "extdata", "chr20_2.1.fq.gz")), 
        control=list(fastqInput1 = system.file(package="esATAC", "extdata", "chr20_1.2.fq.bz2"),
                     fastqInput2 = system.file(package="esATAC", "extdata", "chr20_2.2.fq.bz2")),
        genome = "hg19",
        motifs = getMotifInfo(motif.file = system.file("extdata", "CustomizedMotif.txt", package="esATAC")))

1.5.2 single sample

library(esATAC)
conclusion <- 
    atacPipe(
        fastqInput1 = system.file(package="esATAC", "extdata", "chr20_1.1.fq.gz"),
        fastqInput2 = system.file(package="esATAC", "extdata", "chr20_2.1.fq.gz"),
        genome = "hg19",
        motifs = getMotifInfo(motif.file = system.file("extdata", "CustomizedMotif.txt", package="esATAC")))

1.6 Use “Conclusion” for Further Analysis

The conclusion returned by atacPipe or atacPipe2 contains all configuration and result information generated by the pipeline.

1.6.1 Case-control Sample Conclusion

There are three slots in conclusion list:

You can use $ operator to obtain value. For example, conclusion$caselist, conclusion$caselist$atacProcs$renamer, etc.

1.6.2 Single Sample Conclusion

There are four slots in conclusion list:

  • filelist # a dataframe of fastq file paths
  • wholesummary # a dataframe of summary table
  • atacProcs # ATACProc objects list, see Query and Object Operations for object operation
    • unzipAndMerge
    • renamer
    • removeAdapter
    • bowtie2Mapping
    • libComplexQC
    • sam2Bed
    • bedToBigWig
    • tssqc100
    • peakCalling
    • DHSQC
    • blacklistQC
    • fripQC
    • atacQC
  • filtstat # a dataframe of reads or peaks filtering summary table

Slot atacProcs is also an list. Similarly, use $ to access them: conclusion$atacProcs$renamer

1.7 Resume Your Analysis

To run the whole pipeline for a typical ATACseq data set of human sample may take ~2 days on a personal computer with single thread. If your thread or R has been stopped during the process. You can simplely resume the analysis by retyping your command line in Starting from Scratch. The program will automatically check the steps that have been finished and continue the analysis.

2 Introduction

The esATAC package provides a dataflow graphs organized end-to-end pipeline for quantifying and annotating ATAC-seq and DNase-seq Reads in R, which integrate the functionality of several R packages (such as Rsamtools, ChIPpeakAnno and so on) and external softwares (e.g. AdapterRemoval[1], bowtie2[2], through the Rowtie2 package and Fseq[3]). Users could process raw FASTQ files through preset pipeline or customize their own workflow starting from any intermediate stages easily and flexibly in a single R script. That will be convenient to migrate, share and reproduce all details such as parameters settings, intermediate result and so on. Besides, a pretty quality control report file in HTML, which is able to be viewed in web browser, will be created in preset pipelines.

esATAC can be easily installed on various operator system platforms (Windows, Linux, Mac OS). All functions in package consume up to 16G memory. Most function only consume less than 8G. So the package is available for not only servers but also most of PC.

esATAC supports analysis of both single end reads and paired-end reads ATAC-seq data generated by Illumina sequencing platform. It can directly process raw datasets (FASTQ files) from GEO. Other standard format intermediate result files (FASTQ, SAM, BAM, BED file) generated by other programs (such as BAM BED files from ENCODE) are also tested by rebuilt sub-pipeline.

2.1 Flowchart and overview

If you do not know where to start with ATAC-seq or DNase-seq data, you can print flowchart like this:

library(esATAC)
printMap()

Following the flowchart, related functions could be found in manual. For example,if you want to query functions related to “SamToBed” in the flowchart, you can query “SamToBed” directly like this:

?SamToBed

If you know exactly function name, you can add “atac” prefix to query mannual like this:

?atacSamToBed

or use the lowercase of the initial letter

?samToBed

The workflow start with “UnzipAndMerge” function atacUnzipAndMerge. It unzips and merges the replicates into one FASTQ file(two for paired end file). Names of reads will be renamed as numbers: 1,2,3,… by calling “Renamer” function atacRenamer. The file will be smaller for further analysis. Adapter of reads may be found and removed by “RemoveAdapter” function atacRemoveAdapter. Then reads are ready for mapping to reference genome. “Bowtie2Mapping” mapping function atacBowtie2Mapping can do this job. “SamToBam”, “Rsortbam”,“BamToBed”,“SamToBed” and “BedUtils” provide general processing methods for SAM file including converting format into BAM or BED file, sorting according to chromosome/start site/end site, reads conditional filtering, reads shifting and so on. The ready-use reads in BED file may call peak by “PeakCallingFseq” function atacPeakCallingFseq.

For preset pipeline (see Quick Start), several summary tables will be shown in an HTML file(Example report) like this:

Item Case Control Reference
Sequence Files Type paired end (PE) paired end (PE) SE / PE
Original total reads 54.1M 56.5M
– Reads after adapter removing (ratio) 54.1M (100.00%) 56.5M (100.00%) >99%
– – Total mapped reads (ratio of original reads) 52.7M (97.53%) 55.1M (97.63%) >95%
– – – Unique locations mapped uniquely by reads 27.3M 25.6M
– – – Non-Redundant Fraction (NRF) 0.73 0.7 >0.7
– – – Locations with only 1 reads mapping uniquely 24.1M 23.1M
– – – Locations with only 2 reads mapping uniquely 2.5M 1.9M
– – – PCR Bottlenecking Coefficients 1 (PBC1) 0.88 0.9 >0.7
– – – PCR Bottlenecking Coefficients 2 (PBC2) 9.71 12.28 >3
– – – Non-mitochondrial reads (ratio) 37.4M (70.87%) 36.6M (66.31%) >70%
– – – – Unique mapped reads (ratio) 29.1M (55.26%) 26.6M (48.21%)
– – – – – Duplicate removed reads (final for use) 25.9M (49.13%) 24.2M (43.97%) >25M
– – – – – – Nucleosome free reads (<100bp) 10.5M (40.47%) 8.2M (33.92%)
– – – – – – – Total peaks 118157 116686
– – – – – – – Peaks overlaped with union DHS ratio 76.00% 79.00%
– – – – – – – Peaks overlaped with blacklist ratio 0.10% 0.10%
– – – – – – Fraction of reads in peaks (FRiP) 52.00% 66.50%

The pipeline also provide quality control elements (e.g.“FragLenDistr”, “FastQC”, ) and some general genome function analysis elements (e.g. “RMotifScan”,“RPeakAnno”). For more detail, you can see the manual or the examples in following sections.

Fragments Length Distribution Example

Fragments Length Distribution Example

Fourier Transformation Analysis of Distribution

Fourier Transformation Analysis of Distribution

2.2 Contact

This package is developed and maintained by members of

Xiaowo Wang Lab

Ministry of Education Key Laboratory of Bioinformatics,

Center for Synthetic and Systems Biology,

Department of Automation,

Tsinghua University, Beijing, 100084, China

email:{wei-z14,w-zhang16}(at)mails.tsinghua.edu.cn

3 Customized pipeline

All sub-processes are available for recombine new whole pipeline or sub-pipeline easily and flexibly. They are also able to be called individually. We just show some functions and their combinations from the package. For detail, the users can read the manual.

3.1 Preparation

3.1.1 Loading

Just like other R package, you need to load esATAC like this each time before using the package.

library(esATAC)

If you need to use fseq, we recommend to set max memory size for java (8G, 8000M in the example). Or rJava will use the default parameter for fseq.

options(java.parameters = "-Xmx8000m")

3.1.2 Loading Recommend Package

The BSgenome package, TxDb known gene package and OrgDb annotation package for some functions are required. We recommend to install (use BiocManager::install("packageName")) and load the specific species related packages before using the packages.

library(magrittr)
library(BSgenome.Hsapiens.UCSC.hg19)
library(TxDb.Hsapiens.UCSC.hg19.knownGene)
library(org.Hs.eg.db)
library(R.utils)

Packages for specific genomes are shown below:

genome BSgenome TxDb OrgDb
hg19 BSgenome.Hsapiens.UCSC.hg19 TxDb.Hsapiens.UCSC.hg19.knownGene org.Hs.eg.db
hg38 BSgenome.Hsapiens.UCSC.hg38 TxDb.Hsapiens.UCSC.hg38.knownGene org.Hs.eg.db
mm9 BSgenome.Mmusculus.UCSC.mm9 TxDb.Mmusculus.UCSC.mm9.knownGene org.Mm.eg.db
mm10 BSgenome.Mmusculus.UCSC.mm10 TxDb.Mmusculus.UCSC.mm10.knownGene org.Mm.eg.db
danRer10 BSgenome.Drerio.UCSC.danRer10 TxDb.Drerio.UCSC.danRer10.refGene org.Dr.eg.db
galGal5 BSgenome.Ggallus.UCSC.galGal5 TxDb.Ggallus.UCSC.galGal5.refGene org.Gg.eg.db
galGal4 BSgenome.Ggallus.UCSC.galGal4 TxDb.Ggallus.UCSC.galGal4.refGene org.Gg.eg.db
rheMac3 BSgenome.Mmulatta.UCSC.rheMac3 TxDb.Mmulatta.UCSC.rheMac3.refGene org.Mmu.eg.db
rheMac8 BSgenome.Mmulatta.UCSC.rheMac8 TxDb.Mmulatta.UCSC.rheMac8.refGene org.Mmu.eg.db
rn6 BSgenome.Rnorvegicus.UCSC.rn6 TxDb.Rnorvegicus.UCSC.rn6.refGene org.Rn.eg.db
rn5 BSgenome.Rnorvegicus.UCSC.rn5 TxDb.Rnorvegicus.UCSC.rn5.refGene org.Rn.eg.db
sacCer3 BSgenome.Scerevisiae.UCSC.sacCer3 TxDb.Scerevisiae.UCSC.sacCer3.sgdGene org.Sc.sgd.db
sacCer2 BSgenome.Scerevisiae.UCSC.sacCer2 TxDb.Scerevisiae.UCSC.sacCer2.sgdGene org.Sc.sgd.db
susScr3 BSgenome.Sscrofa.UCSC.susScr3 TxDb.Sscrofa.UCSC.susScr3.refGene org.Ss.eg.db

3.1.3 Configure Other Parameters

These configurations are also optional. “tmpdir” is the path to save all of the temporary data and the default result storage path. If it is not configured, current work directory will be set as “tmpdir”. “threads” is the maximum threads allowed to be created for data processing. The default value is 1. More thread will consume more memery in some processes.

# we use temp directiory "td" here 
# Change it to your directiory because the intermediate file may be huge
td<-tempdir()
options(atacConf=setConfigure("tmpdir",td))
options(atacConf=setConfigure("threads",8))

3.1.4 Reference Data Installation and Loading

We strongly recommend to install reference data first before using the package although it is optional. “refdir” is the folder that will save all of the reference data. “genome” is the genome name like hg19, hg38, mm10, mm9 and so on. The program will detect the elements that have not been installed and install them. Some resources need to be downloaded from internet. So don’t forget to connect internet during installation. Or the installation will be failed. If all of the reference data was installed, these two lines still need to be called for configuring the reference data path and genome.

options(atacConf=setConfigure("refdir","path/to/refdatafolder"))
options(atacConf=setConfigure("genome","hg19"))

NOTE: The installation will consume several hours for data download and building bowtie2 index depending on computer performance and network bandwidth.

NOTE: The installation is network based. Please keep your network connection. But you don need to worry about disconnect. The program will continue to check finished part and only build unfinish part.

WARNNING: If the reference data is not configured, the related reference argument of functions has to be set manually during using.

3.1.5 Already Have Genome Bowtie2 Index

Build bowtie index may take some time. If you already have bowtie2 index files or you want to download instead of building, you can let esATAC skip the steps by renaming them following the format (genome+suffix) and put them in reference installation path (refdir).

Example: hg19 bowtie2 index files

  • hg19.1.bt2
  • hg19.2.bt2
  • hg19.3.bt2
  • hg19.4.bt2
  • hg19.rev.1.bt2
  • hg19.rev.2.bt2

bowtie2 index download path:

ftp://ftp.ccb.jhu.edu/pub/data/bowtie2_indexes

3.2 An Customized Pipeline Example

Example: a simplified atacPipe from single sample (FASTQ) to library quality control and peak calling

library(esATAC)
library(magrittr)


dir.create("./esATAC_pipeline")
dir.create("./esATAC_pipeline/refdir")
dir.create("./esATAC_pipeline/result")

#configure reference path, result path, the max number of threads, genome
options(atacConf=setConfigure("refdir","esATAC_pipeline/refdir"))
options(atacConf=setConfigure("tmpdir","esATAC_pipeline/result"))
options(atacConf=setConfigure("threads",2))
options(atacConf=setConfigure("genome","hg19"))

#raw reads
fastqInput1 = system.file(package="esATAC", "extdata", "chr20_1.1.fq.gz")
fastqInput2 = system.file(package="esATAC", "extdata", "chr20_2.1.fq.gz") 

#pipeline           
atacUnzipAndMerge(fastqInput1 = fastqInput1,fastqInput2 = fastqInput2) %T>%
atacQCReport %>%
atacRenamer %>%
atacRemoveAdapter %>%
atacBowtie2Mapping %T>% 
atacLibComplexQC %>% 
atacSamToBed(maxFragLen = 2000) %T>%
atacBedToBigWig %T>% 
atacFragLenDistr %>%
atacBedUtils(maxFragLen = 100, chrFilterList = NULL) %>%
atacPeakCalling

3.2.1 Configuration Details

There are 4 configurable parameters (“refdir”,“tmpdir”,“threads”,“genome”) in this package. None of them is required because users can also pass them to the function as arguments based on needs. For convenience, by configuring them in advance, these arguments for functions will not need to be set repeatedly. Users can focus more on data flow pipeline implementation rather than dependency. These are the details for the parameters:

“genome”, the genome like “hg19”,“mm10”, etc.

“refdir”, the path for genome reference and annotation data storage.

“tmpdir”, the path for intermediate files and result storage. Default, “./”, current working directory

“threads”, the max number of threads allowed to be created. Default, 1

dir.create("./esATAC_pipeline")
dir.create("./esATAC_pipeline/refdir")
dir.create("./esATAC_pipeline/result")

#configure reference path, result path, the max number of threads, genome
options(atacConf=setConfigure("refdir","esATAC_pipeline/refdir"))
options(atacConf=setConfigure("tmpdir","esATAC_pipeline/result"))
options(atacConf=setConfigure("threads",2))
options(atacConf=setConfigure("genome","hg19"))

In this example, we create folder “./esATAC_pipeline/refdir” for “refdir” and “./esATAC_pipeline/result” for “tmpdir”. Two threads are allowed. Genome reference and annotations will be install under “./esATAC_pipeline/refdir” if they are not complete.

3.2.2 Traverse the Dataflow Subgraph

Pipe operator in R (%>% and %T>%) may help users build pipeline more easily. Parameters are passed seamlessly between upstream and downstream unit.

#raw reads
fastqInput1 = system.file(package="esATAC", "extdata", "chr20_1.1.fq.gz")
fastqInput2 = system.file(package="esATAC", "extdata", "chr20_2.1.fq.gz") 

#pipeline           
atacUnzipAndMerge(fastqInput1 = fastqInput1,fastqInput2 = fastqInput2) %T>%
atacQCReport %>%
atacRenamer %>%
atacRemoveAdapter %>%
atacBowtie2Mapping %T>% 
atacLibComplexQC %>% 
atacSamToBed(maxFragLen = 2000) %T>%
atacBedToBigWig %T>% 
atacFragLenDistr %>%
atacBedUtils(maxFragLen = 100, chrFilterList = NULL) %>%
atacPeakCalling

3.3 Call Functions Individually

3.3.1 Preprocess

Users can use %>% to build a pipeline to obtain merged, renamed and adapter removed clean reads fastq file(s) that is ready for mapping.

# Identify adapters
prefix<-system.file(package="esATAC", "extdata", "uzmg")
(reads_1 <-file.path(prefix,"m1",dir(file.path(prefix,"m1"))))
(reads_2 <-file.path(prefix,"m2",dir(file.path(prefix,"m2"))))

reads_merged_1 <- file.path(td,"reads1.fastq")
reads_merged_2 <- file.path(td,"reads2.fastq")
atacproc <- 
atacUnzipAndMerge(fastqInput1 = reads_1,fastqInput2 = reads_2) %>%
atacRenamer %>% atacRemoveAdapter

If you want to modify the parameters of AdapterRemoval, you have to refer to Rbowtie2 package:

library(Rbowtie2)
adapterremoval_usage()

3.3.2 Mapping

If the reference has not been configured, the bowtie2 index should be built first. Then bowtie2 mapping functions could used to map reads to reference genome.

## Building a bowtie2 index
library("Rbowtie2")
refs <- dir(system.file(package="esATAC", "extdata", "bt2","refs"),
full=TRUE)
bowtie2_build(references=refs, bt2Index=file.path(td, "lambda_virus"),
"--threads 4 --quiet",overwrite=TRUE)
## Alignments
reads_1 <- system.file(package="esATAC", "extdata", "bt2", "reads",
"reads_1.fastq")
reads_2 <- system.file(package="esATAC", "extdata", "bt2", "reads",
"reads_2.fastq")
if(file.exists(file.path(td, "lambda_virus.1.bt2"))){
    (bowtie2Mapping(bt2Idx = file.path(td, "lambda_virus"),
       samOutput = file.path(td, "result.sam"),
       fastqInput1=reads_1,fastqInput2=reads_2,threads=3))
    head(readLines(file.path(td, "result.sam")))
}

If you want to modify the parameters of bowtie2, you have to refer to Rbowtie2 package:

library(Rbowtie2)
bowtie2_usage()

3.3.3 Convert SAM File to BED File

The mapping results are stored in a SAM file. SamToBed functions can covert it into BED file. During converting, the operation like sorting, shifting, filtering chromosome and so on can also be set to do in the meantime.

sambzfile <- system.file(package="esATAC", "extdata", "Example.sam.bz2")
samfile <- file.path(td,"Example.sam")
bunzip2(sambzfile,destname=samfile,overwrite=TRUE,remove=FALSE)
samToBed(samInput = samfile)

3.3.4 Filtering Reads and Calling

Filter the nucleosome free reads(<100bp) for peak calling.

bedbzfile <- system.file(package="esATAC", "extdata", "chr20.50000.bed.bz2")
bedfile <- file.path(td,"chr20.50000.bed")
bunzip2(bedbzfile,destname=bedfile,overwrite=TRUE,remove=FALSE)

bedUtils(bedInput = bedfile,maxFragLen = 100, chrFilterList = NULL) %>%
atacPeakCalling

3.3.5 ATAC-seq Peak Annotation

ATAC-seq peak locate at open chromatin regions. Annotating these peak could find whether they locate at functional regions(such as promoter and enhancer).

Function “atacPeakAnno” and “peakanno” use function “annotatePeak” in package “ChIPseeker” to annotate ATAC-seq peak. for more information about package “ChIPseeker”, please click here[4].

Function “atacPeakAnno” and “peakanno” accept a bed file path as an input, users can change the parameters like “tssRegion”, “TxDb” according to their require. Now, bioconductor offers many species’ annotation database, click here to search more.

The following example is to exhibit how to annotate a UCSC bed file.

## extract example peak file from package "esATAC"
p1bz <- system.file("extdata", "Example_peak1.bed.bz2", package="esATAC")
peak1_path <- as.vector(bunzip2(filename = p1bz,
destname = file.path(getwd(), "Example_peak1.bed"),
ext="bz2", FUN=bzfile, overwrite=TRUE, remove = FALSE))
## run peakanno to annotate peaks
AnnoInfo <- peakanno(peakInput = peak1_path, TxDb = TxDb.Hsapiens.UCSC.hg19.knownGene, annoDb = "org.Hs.eg.db")

The output contains a pie chart in pdf format like below. It reports the percentage of peaks located in different functional regions.

The function also generate a file(with suffix .txt) contains annotation for all peaks. It is converted from dataframe in R, and users could open it with text editor or excel. Below is a part of the output.

chromatin start end annotation geneStart geneEnd geneId distanceToTSS SYMBOL
chr1 416606 416895 Distal Intergenic 367659 368597 729759 48947 OR4F29
chr1 2313275 2313587 Intron (uc001ajb.1/79906, intron 6 of 13) 2252696 2322993 79906 9406 MORN1
chr1 2516858 2518618 Promoter 2517899 2522908 127281 0 PRXL2B
chr1 2685755 2686581 Intron (uc021oey.1/100287898, intron 4 of 6) 2572807 2706230 100287898 19649 TTC34
chr1 3418588 3418822 Intron (uc001akk.3/1953, intron 14 of 29) 3404506 3448012 1953 29190 MEGF6

3.3.6 Go Analysis

GO analysis is performing enrichment analysis on gene sets. It establishes the relationship between gene sets and functions, and report the most significant function to users.

Function “atacGOAnalysis” and “goanalysis” use function “enrichGO” in package “clusterProfiler” to do GO analysis. for more information about package “clusterProfiler”, please click here[5].

The function need gene Id set as input. User could choose different GO terms(molecular function, biological process and cellular component) according to different input of parameter “ont”.

The following example is to exhibit how to do GO analysis on a gene set.

## extract gene ID
geneId <- as.character(sample(seq(10000), 100))
## do GO analysis
goAna <- goanalysis(gene = geneId, OrgDb = "org.Hs.eg.db", keytype = "ENTREZID", ont = "MF")

The output file(suffix .txt) contains the GO term sorted by p-value, below is a part of the output.

ID Description GeneRatio pvalue qvalue
1 GO:0004713 protein tyrosine kinase activity 6/65 0.0000096 0.0019593
2 GO:0019199 transmembrane receptor protein kinase activity 4/65 0.0002025 0.0205714
NA NA NA NA NA NA
NA.1 NA NA NA NA NA
NA.2 NA NA NA NA NA

3.3.7 Motif Scan

This function search motif occurrence in the given regions.

Function “atacMotifScan” and “motifscan” use function “matchMotifs” in package “motifmatchr”, for more parameters and usage, click here[6].

Multi-motif is supported, and the output file is named by your input motif list. The input could be PFMatrix, PFMatrixList, PWMatrix, PWMatrixList from TFBSTools.

Using “motifscan” function to search motif in given genome regions, UCSC bed file is recommended.

sample.path <- system.file("extdata", "chr20_sample_peak.bed.bz2", package="esATAC")
sample.path <- as.vector(bunzip2(filename = sample.path,
destname = file.path(getwd(), "chr20_sample_peak.bed"),
ext="bz2", FUN=bzfile, overwrite=TRUE, remove = FALSE))

motif <- readRDS(system.file("extdata", "MotifPFM.rds", package="esATAC"))

motif.data <- motifscan(peak = sample.path, genome = BSgenome.Hsapiens.UCSC.hg19, motifs = motif)

This function reports the exact motif position in the given genome like below(motif: CTCF).

chromatin start end strand score sequence
chr20 189774 189792 + 0.8794931 ACTCCTCTAGAGGGTGCTC
chr20 239773 239791 + 0.9003697 TTGCCACTGGGGGGAGACA
chr20 247783 247801 - 0.9337214 CTGCCGGCAGATGGCGGTA
chr20 281074 281092 - 0.8511201 TTGCCTGCAGGGGTGGGAA

3.3.8 Plot Footprint

The interaction between TF and DNA would leave a “footprint” in motif position, but it is not evident in a single site, so integrated footprint is necessary. In addition, we only consider Tn5 cut site. This function is based on the motif scan.

First, collecting all cut site from the bed file(Note: every line in the bed file is a DNA fragment) and save them.

## extract cut site position from bed file
fra_path <- system.file("extdata", "chr20.50000.bed.bz2", package="esATAC")
frag <- as.vector(bunzip2(filename = fra_path,
destname = file.path(getwd(), "chr20.50000.bed"),
ext="bz2", FUN=bzfile, overwrite=TRUE, remove = FALSE))
cs.data <- extractcutsite(bedInput = frag, prefix = "ATAC")

Next, plot footprint for different motifs.

In the motif scan, we get a variable named “motif.data”, is contains multi-motif information. In order to plot footprint of these motif in a single procedure, we will use the output of function motifscan, here is “motif.data”.

fp <- atacCutSiteCount(atacProcCutSite = cs.data, atacProcMotifScan = motif.data)

The following is CTCF footprint using example data.

Note: we only using a small part of the chromatin 20 as example.

4 Query and Object Operations

4.1 Workflow Map

esATAC is organized in data flow graph. Except for referring manual, the user may print the map to know workflow order.

bedbzfile <- system.file(package="esATAC", "extdata", "chr20.50000.bed.bz2")
bedfile <- file.path(td,"chr20.50000.bed")
bunzip2(bedbzfile,destname=bedfile,overwrite=TRUE,remove=FALSE)

peakproc <-bedUtils(bedInput = bedfile,maxFragLen = 100, chrFilterList = NULL) %>%
atacPeakCalling 

peakproc %>%  printMap

By printing the map, it is easy to know what valid processes are available to call next and what preprocess has been done before.

4.2 Parameters

It is easy to query the parameters set for the process with ATACProc objects. You can query available parameters like this:

#query all of available parameters
getParamItems(peakproc)
## [1] "bedInput"     "bedFileList"  "inBedDir"     "bedOutput"   
## [5] "outTmpDir"    "fragmentSize" "fileformat"   "verbose"

The value of a specific parameter can be obtain like this:

#query a parameter value
getParam(peakproc,"fragmentSize")
## [1] 0

4.3 Report Value

Similarly, it is also easy to query the report value calculated by the process with ATACProc objects

sambzfile <- system.file(package="esATAC", "extdata", "Example.sam.bz2")
samfile <- file.path(td,"Example.sam")
bunzip2(sambzfile,destname=samfile,overwrite=TRUE,remove=FALSE)
samToBedProc<-samToBed(samInput = samfile)

When the ATACProc objects are obtained, you can query all of available report items.

getReportItems(samToBedProc)
## [1] "report"                     "total"                     
## [3] "save"                       "filted"                    
## [5] "extlen"                     "unique"                    
## [7] "multimap"                   "non-mitochondrial"         
## [9] "non-mitochondrial-multimap"

The value of specific report item can be get like this:

#query a parameter value
getReportVal(samToBedProc,"report")

4.4 Clear the Cache and Redo Process

If the user call a process function that was called last time and finished, the process function will not redo the process. So if users need to redo the process, they have to clear the cache like this:

clearProcCache(peakproc)
process(peakproc)

5 Acknowledgement

We would like to thank Huan Fang for package testing and valuable suggestions,

and Kui Hua for providing package testing on Macbook.

6 Reference

[1] Langmead, B., & Salzberg, S. L. (2012). Fast gapped-read alignment with Bowtie 2. Nature methods, 9(4), 357-359.

[2] Schubert, Lindgreen, and Orlando (2016). AdapterRemoval v2: rapid adapter trimming, identification, and read merging. BMC Research Notes, 12;9(1):88.

[3] Boyle, A. P., Guinney, J., Crawford, G. E., & Furey, T. S. (2008). F-Seq: a feature density estimator for high-throughput sequence tags. Bioinformatics, 24(21), 2537-2538.

[4] Yu G, Wang L and He Q (2015). “ChIPseeker: an R/Bioconductor package for ChIP peak annotation, comparison and visualization.” Bioinformatics, 31(14), pp. 2382-2383. doi: 10.1093/bioinformatics/btv145.

[5] Yu G, Wang L, Han Y and He Q (2012). “clusterProfiler: an R package for comparing biological themes among gene clusters.” OMICS: A Journal of Integrative Biology, 16(5), pp. 284-287. doi: 10.1089/omi.2011.0118.

[6] Schep A (2017). motifmatchr: Fast Motif Matching in R. R package version 1.0.1.

[7] Buenrostro, J. D., Giresi, P. G., Zaba, L. C., Chang, H. Y., & Greenleaf, W. J. (2013). Transposition of native chromatin for fast and sensitive epigenomic profiling of open chromatin, DNA-binding proteins and nucleosome position. Nature methods, 10(12), 1213-1218.

7 Session Infomation

sessionInfo()
## R version 3.6.1 (2019-07-05)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Ubuntu 18.04.3 LTS
## 
## Matrix products: default
## BLAS:   /home/biocbuild/bbs-3.10-bioc/R/lib/libRblas.so
## LAPACK: /home/biocbuild/bbs-3.10-bioc/R/lib/libRlapack.so
## 
## locale:
##  [1] LC_CTYPE=en_US.UTF-8          LC_NUMERIC=C                 
##  [3] LC_TIME=en_US.UTF-8           LC_COLLATE=C                 
##  [5] LC_MONETARY=en_US.UTF-8       LC_MESSAGES=en_US.UTF-8      
##  [7] LC_PAPER=en_US.UTF-8          LC_NAME=en_US.UTF-8          
##  [9] LC_ADDRESS=en_US.UTF-8        LC_TELEPHONE=en_US.UTF-8     
## [11] LC_MEASUREMENT=en_US.UTF-8    LC_IDENTIFICATION=en_US.UTF-8
## 
## attached base packages:
## [1] stats4    parallel  stats     graphics  grDevices utils     datasets 
## [8] methods   base     
## 
## other attached packages:
##  [1] ChIPseeker_1.22.0                      
##  [2] Rbowtie2_1.8.0                         
##  [3] R.utils_2.9.0                          
##  [4] R.oo_1.22.0                            
##  [5] R.methodsS3_1.7.1                      
##  [6] org.Hs.eg.db_3.10.0                    
##  [7] TxDb.Hsapiens.UCSC.hg19.knownGene_3.2.2
##  [8] GenomicFeatures_1.38.0                 
##  [9] AnnotationDbi_1.48.0                   
## [10] BSgenome.Hsapiens.UCSC.hg19_1.4.0      
## [11] BSgenome_1.54.0                        
## [12] rtracklayer_1.46.0                     
## [13] magrittr_1.5                           
## [14] esATAC_1.8.0                           
## [15] ShortRead_1.44.0                       
## [16] GenomicAlignments_1.22.0               
## [17] SummarizedExperiment_1.16.0            
## [18] DelayedArray_0.12.0                    
## [19] matrixStats_0.55.0                     
## [20] Biobase_2.46.0                         
## [21] BiocParallel_1.20.0                    
## [22] Rsamtools_2.2.0                        
## [23] Biostrings_2.54.0                      
## [24] XVector_0.26.0                         
## [25] GenomicRanges_1.38.0                   
## [26] GenomeInfoDb_1.22.0                    
## [27] IRanges_2.20.0                         
## [28] S4Vectors_0.24.0                       
## [29] BiocGenerics_0.32.0                    
## 
## loaded via a namespace (and not attached):
##   [1] tidyselect_0.2.5            poweRlaw_0.70.2            
##   [3] RSQLite_2.1.2               htmlwidgets_1.5.1          
##   [5] grid_3.6.1                  munsell_0.5.0              
##   [7] colorspace_1.4-1            GOSemSim_2.12.0            
##   [9] rgexf_0.15.3                highr_0.8                  
##  [11] knitr_1.25                  rstudioapi_0.10            
##  [13] rJava_0.9-11                DOSE_3.12.0                
##  [15] urltools_1.7.3              GenomeInfoDbData_1.2.2     
##  [17] hwriter_1.3.2               polyclip_1.10-0            
##  [19] bit64_0.9-7                 farver_1.1.0               
##  [21] downloader_0.4              vctrs_0.2.0                
##  [23] lambda.r_1.2.4              xfun_0.10                  
##  [25] BiocFileCache_1.10.0        R6_2.4.0                   
##  [27] graphlayouts_0.5.0          VGAM_1.1-1                 
##  [29] bitops_1.0-6                fgsea_1.12.0               
##  [31] gridGraphics_0.4-1          assertthat_0.2.1           
##  [33] scales_1.0.0                ggraph_2.0.0               
##  [35] enrichplot_1.6.0            gtable_0.3.0               
##  [37] tidygraph_1.1.2             seqLogo_1.52.0             
##  [39] rlang_0.4.1                 zeallot_0.1.0              
##  [41] splines_3.6.1               lazyeval_0.2.2             
##  [43] europepmc_0.3               brew_1.0-6                 
##  [45] BiocManager_1.30.9          yaml_2.2.0                 
##  [47] reshape2_1.4.3              backports_1.1.5            
##  [49] qvalue_2.18.0               clusterProfiler_3.14.0     
##  [51] DiagrammeR_1.0.1            tools_3.6.1                
##  [53] influenceR_0.1.0            ggplotify_0.0.4            
##  [55] ggplot2_3.2.1               gplots_3.0.1.1             
##  [57] RColorBrewer_1.1-2          ggridges_0.5.1             
##  [59] Rcpp_1.0.2                  plyr_1.8.4                 
##  [61] visNetwork_2.0.8            progress_1.2.2             
##  [63] zlibbioc_1.32.0             purrr_0.3.3                
##  [65] RCurl_1.95-4.12             prettyunits_1.0.2          
##  [67] openssl_1.4.1               viridis_0.5.1              
##  [69] cowplot_1.0.0               ggrepel_0.8.1              
##  [71] motifmatchr_1.8.0           data.table_1.12.6          
##  [73] futile.options_1.0.1        DO.db_2.9                  
##  [75] triebeard_0.3.0             hms_0.5.1                  
##  [77] evaluate_0.14               xtable_1.8-4               
##  [79] XML_3.98-1.20               VennDiagram_1.6.20         
##  [81] gridExtra_2.3               compiler_3.6.1             
##  [83] biomaRt_2.42.0              tibble_2.1.3               
##  [85] KernSmooth_2.23-16          crayon_1.3.4               
##  [87] htmltools_0.4.0             TFBSTools_1.24.0           
##  [89] tidyr_1.0.0                 DBI_1.0.0                  
##  [91] tweenr_1.0.1                formatR_1.7                
##  [93] corrplot_0.84               dbplyr_1.4.2               
##  [95] MASS_7.3-51.4               rappdirs_0.3.1             
##  [97] boot_1.3-23                 Matrix_1.2-17              
##  [99] readr_1.3.1                 JASPAR2016_1.13.0          
## [101] gdata_2.18.0                igraph_1.2.4.1             
## [103] pkgconfig_2.0.3             rvcheck_0.1.5              
## [105] TFMPvalue_0.0.8             xml2_1.2.2                 
## [107] annotate_1.64.0             DirichletMultinomial_1.28.0
## [109] stringr_1.4.0               digest_0.6.22              
## [111] CNEr_1.22.0                 rmarkdown_1.16             
## [113] fastmatch_1.1-0             Rook_1.1-1                 
## [115] curl_4.2                    gtools_3.8.1               
## [117] lifecycle_0.1.0             jsonlite_1.6               
## [119] futile.logger_1.4.3         viridisLite_0.3.0          
## [121] askpass_1.1                 pillar_1.4.2               
## [123] lattice_0.20-38             KEGGREST_1.26.0            
## [125] httr_1.4.1                  plotrix_3.7-6              
## [127] GO.db_3.10.0                glue_1.3.1                 
## [129] png_0.1-7                   bit_1.1-14                 
## [131] ggforce_0.3.1               stringi_1.4.3              
## [133] blob_1.2.0                  latticeExtra_0.6-28        
## [135] caTools_1.17.1.2            memoise_1.1.0              
## [137] dplyr_0.8.3