Introduction to calm

Introduction

Multiple testing procedures are commonly used in the analyses of genomic data, where thousands of tests are conducted simultaneously. As in many other scientific research areas, new knowledge is discovered on the top of existing ones, which can appear as various auxiliary/contextual information. Calm is an R package for multiple testing methods with auxiliary covariate information.

The auxiliary information is almost ubiquitous in scientific research. In genomic experiments, each test corresponds to a genetic entity, such as a gene or single nucleotide polymorphisms (SNP). Suppose that we try to find which genes are differentially expressed between two conditions (e.g., disease status). Each gene has many (physical) properties, such as the location and length of its coding region. Furthermore, the summary statistics from a related study (similar experimental condition or related disease) can be highly informative. Calm provides the ability to perform the multiple testing while utilizing the existing auxiliary covariate information to improve power.

Installation

if(!requireNamespace("BiocManager", quietly = TRUE))
    install.packages("BiocManager")
BiocManager::install("calm")

Conditional local FDR

In multiple testing applications with covariate information, the ranking and the significance of the test statistics become non-trivial. We will estimate the conditional local FDR, which is the posterior probability of being null given the test statistic and the covariate information.

Consider the RNA sequencing (RNA-seq) study of psoriasis vulgaris disease in Jabbari et al. (2012), where the expression levels of genes were measured on three pairs of lesional and nonlesional skin samples collected from three patients. The RNA-seq read count data can be obtained from the recount2 project (Collado-Torres et al. 2017) with accession number SRP016583. We focus on the 18151 protein coding genes with at least one non-zero read from any patient. To find the differentially expressed (DE) genes between lesional and nonlesional skin samples, we compute \(t\)-statistics using the limma-voom method (Law et al. 2014). Then we could easily compute \(p\)-values from the \(t\)-statistics and use the linear step-up procedure of Benjamini and Hochberg (1995), the BH procedure, to control the FDR at a certain target level.

However, the signs of the \(t\)-statistics suggest whether the corresponding genes are potentially up- or down-regulated, and the use of \(p\)-values leads to loss of such directional information. For modeling convenience, we work with \(z\)-values, which can be transformed from the \(t\)-statistics as follows: z <- qnorm(pt(t,df)), where df is the degrees of freedom.

Furthermore, scientific research is rarely done from scratch each time, and it is almost always possible to find relevant information to improve the power of the current study. In our psoriasis example, each hypothesis is associated with a human gene, whose coding region has a certain length. For each gene, the length of the coding region in the number of nucleotides can also be obtained from the recount2 project.

library(calm)
data("pso")
dim(pso)

## [1] 18151     3

head(pso)

##               zval len_gene   tval_mic
## A1BG    -0.4550995     4006  3.3145543
## A1CF    -0.7702155     9603 -1.0323351
## A2M     -1.6760701     6384 -0.2625656
## A2ML1    4.7265616     7303 22.4756616
## A3GALT2  1.5265343     1023         NA
## A4GALT  -0.3958471     2943 -2.1659572

summary(pso$len_gene)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##     117    2687    4443    5292    6902  118976

hist(pso$len_gene, main="")

We can see that the longest gene is almost 119K nucleotides long and is far longer than other genes. For this reason, we will use a normalized gene length in later analyses to improve estimation stability.

For any specific genetic study, there could be previous studies conducted under similar experimental conditions or on related diseases.
Gudjonsson et al. (2010) used microarray to study the gene expression differences between 58 pairs of lesional and nonlesional skin samples from psoriasis patients. Among the 18151 genes measured in the psoriasis RNA-seq study (Jabbari et al. 2012), only 16493 genes can be found in the microarray study of Gudjonsson et al. (2010) due to the differences between the microarray and RNA-seq platforms.

# number of genes with matching microarray data
sum(!is.na(pso$tval_mic))

## [1] 16493

Naturally, we divide the total of 18,151 genes into two groups: one group contains the 16,493 genes with matching microarray data and another group of 1,658 genes without matches. We will start from the smaller group of 1,658 genes with only the gene length covariate.

# indicator for RNA-seq genes without matching microarray data
ind.nm <- is.na(pso$tval_mic)
x <- pso$len_gene[ind.nm]
# normalize covariate
x <- rank(x)/length(x)
y <- pso$zval[ind.nm]
names(y) <- row.names(pso)[ind.nm]

fit.nm <- CLfdr(x=x, y=y)

## Overall pi0:  0.5971049 Wed May 19 19:14:27 2021 
## Initial bandwidth:  0.088971 0.4551709 
## Bandwidth:  0.1074827 0.1782413 Wed May 19 19:14:49 2021

fit.nm$fdr[1:5]

##      A3GALT2      AADACL3         AARD       AARSD1       ABCA10 
## 0.6141581541 0.0009945293 0.7154088171 0.8991229342 0.0685392778

Here we use the normalized ranks of len_gene as our covariate.
Similarly, we analyze the larger group of 16,493 genes with both microarray \(t\)-statistic and gene length as covariates.

# indicator for RNA-seq genes with matching microarray data
ind.m <- !ind.nm
# normalize covariate
m <- sum(ind.m)
x1 <- rank(pso$tval_mic[ind.m])/m
x2 <- rank(pso$len_gene[ind.m])/m

xmat <- cbind(x1, x2)
colnames(xmat) <- c("tval", "len")
y <- pso$zval[ind.m]
names(y) <- row.names(pso)[ind.m]

fit.m <- CLfdr(x=xmat, y=y, bw=c(0.028, 0.246, 0.253))

## Overall pi0:  0.6005578 Wed May 19 19:14:52 2021

For time consideration, we specify the bandwidth here. Without the explicit specification of the bandwidth, the CLfdr function will find the optimal bandwidth, and it would take 20-30 minutes. Finally, we combine the results from two groups.

fdr <- c(fit.m$fdr, fit.nm$fdr)
FDR <- EstFDR(fdr)
o <- order(FDR)
FDR[o][1:5]

##        KRT6A         GJB2     SERPINB3     SERPINB4        KRT16 
## 8.113107e-09 1.263499e-08 1.480300e-08 2.079942e-08 2.845496e-08

sum(FDR<0.01)

## [1] 1867

Session Information

sessionInfo()

## R version 4.1.0 RC (2021-05-10 r80283)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows Server x64 (build 17763)
## 
## Matrix products: default
## 
## locale:
## [1] LC_COLLATE=C                          
## [2] LC_CTYPE=English_United States.1252   
## [3] LC_MONETARY=English_United States.1252
## [4] LC_NUMERIC=C                          
## [5] LC_TIME=English_United States.1252    
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
## [1] calm_1.6.0
## 
## loaded via a namespace (and not attached):
##  [1] lattice_0.20-44   digest_0.6.27     grid_4.1.0        R6_2.5.0         
##  [5] nlme_3.1-152      jsonlite_1.7.2    magrittr_2.0.1    evaluate_0.14    
##  [9] highr_0.9         rlang_0.4.11      stringi_1.6.2     jquerylib_0.1.4  
## [13] Matrix_1.3-3      bslib_0.2.5.1     rmarkdown_2.8     splines_4.1.0    
## [17] tools_4.1.0       stringr_1.4.0     xfun_0.23         yaml_2.2.1       
## [21] compiler_4.1.0    mgcv_1.8-35       htmltools_0.5.1.1 knitr_1.33       
## [25] sass_0.4.0