Contents

1 Introduction

The information theory R package named Informeasure is to quantify non-linear association between variables in the inference of biological regulatory networks, especially multivariate regulatory networks. This package compiles most information measures currently available: mutual information (MI), conditional mutual information (CMI)[1], interaction information (II)[2], partial information decomposition (PID)[3] and part mutual information (PMI)[4], all of which end with .measure() in form. They are MI.measure() for MI, CMI.measure() for CMI, II.measure() for II, PID.measure() for PID and PMI.measure() for PMI. The first estimator is used to infer bivariate networks while the last four are dedicated to analyze trivariate networks.

2 Main functions demonstration

Informeasure implements five information measures. In the implementation process, each information measure has two discretization mehtods and three types of probability estimators to choose from. Specifically, two discretization methods are uniform width-based method (default) that divides the continuous data into N count bins with equal width and uniform frequency-based approach that determines the continuous data into N count bins with equal count number. Note that the number of bins in these two methods is initialized into a round-off value based on the square root of the data size. Three types of probability estimators referenced to the entropy package[5] that include the empirical estimator (default), the Dirichlet distribution estimator and the shrinkage estimator, While the Dirichlet distribution estimator also includes four different distribution with different prior values. These different probability estimators are showed in detail below.

method = “ML”: empirical estimator, also referred to maximum likelihood estimator,

method = “Jeffreys”: Dirichlet distribution estimator with prior a = 0.5,

method = “Laplace”: Dirichlet distribution estimator with prior a = 1,

method = “SG”: Dirichlet distribution estimator with prior a = 1/length(count table),

method = “minimax”: Dirichlet distribution estimator with prior a = sqrt(sum(count table))/length(count table),

method = “shrink”: shrinkage estimator.

2.1 MI.measure(): mutual information

In the case of two variables, the representative method is mutual information, used to measure the mutual dependence between two joint variables. For example, it can be used to identify dependencies between proteins.

library(Informeasure)
library(SummarizedExperiment)

load(system.file("extdata/tcga.brca.testdata.Rdata", package="Informeasure"))

mRNAexpression <- as.matrix(mRNAexpression)
se.mRNAexpression = SummarizedExperiment(assays = list(mRNAexpression = mRNAexpression))

assays(se.mRNAexpression)[["log2"]] <- log2(assays(se.mRNAexpression)[["mRNAexpression"]]+1)

x <- assays(se.mRNAexpression["BRCA1", ])$log2
y <- assays(se.mRNAexpression["BARD1", ])$log2

XY <- discretize2D(x,y)

MI.measure(XY)
##> [1] 0.6459387

2.2 CMI.measure(): conditional mutual informaiton

In the three-variable case, the most classic method is conditional mutual information. It is widely used to evaluate the expected mutual information between two random variables conditioned on the third one. Such characteristics of conditional mutual information are fully applicable to the ceRNA network inference.

library(Informeasure)
library(SummarizedExperiment)

load(system.file("extdata/tcga.brca.testdata.Rdata", package="Informeasure"))

lncRNAexpression <- as.matrix(lncRNAexpression)
se.lncRNAexpression = SummarizedExperiment(assays = list(lncRNAexpression = lncRNAexpression))

miRNAexpression <- as.matrix(miRNAexpression)
se.miRNAexpression = SummarizedExperiment(assays = list(miRNAexpression = miRNAexpression))

mRNAexpression <- as.matrix(mRNAexpression)
se.mRNAexpression = SummarizedExperiment(assays = list(mRNAexpression = mRNAexpression))

assays(se.lncRNAexpression)[["log2"]] <- log2(assays(se.lncRNAexpression)[["lncRNAexpression"]] + 1)

assays(se.miRNAexpression)[["log2"]] <- log2(assays(se.miRNAexpression)[["miRNAexpression"]] + 1)

assays(se.mRNAexpression)[["log2"]] <- log2(assays(se.mRNAexpression)[["mRNAexpression"]] + 1)


x <- assays(se.miRNAexpression["hsa-miR-26a-5p", ])$log2
y <- assays(se.mRNAexpression["PTEN", ])$log2
z <- assays(se.lncRNAexpression["PTENP1", ])$log2

XYZ <- discretize3D(x,y,z)

CMI.measure(XYZ)
##> [1] 0.7697107

2.3 II.measure(): interaction information

Interaction information, also known as co-information, measures the amount information contained in a set of variables beyond any subset of those variables. The number of variables here is limited to three. It can be applied to explore the cooperative or competitive regulation mechanism of two miRNAs on the common target mRNA.

library(Informeasure)
library(SummarizedExperiment)

load(system.file("extdata/tcga.brca.testdata.Rdata", package="Informeasure"))

miRNAexpression <- as.matrix(miRNAexpression)
se.miRNAexpression = SummarizedExperiment(assays = list(miRNAexpression = miRNAexpression))

mRNAexpression <- as.matrix(mRNAexpression)
se.mRNAexpression = SummarizedExperiment(assays = list(mRNAexpression = mRNAexpression))

assays(se.miRNAexpression)[["log2"]] <- log2(assays(se.miRNAexpression)[["miRNAexpression"]] + 1)

assays(se.mRNAexpression)[["log2"]] <- log2(assays(se.mRNAexpression)[["mRNAexpression"]] + 1)

x <- assays(se.miRNAexpression["hsa-miR-34a-5p", ])$log2
y <- assays(se.mRNAexpression["MYC", ])$log2
z <- assays(se.miRNAexpression["hsa-miR-34b-5p", ])$log2

XYZ <- discretize3D(x,y,z)

II.measure(XYZ)
##> [1] 0.4676038

2.4 PID.measure(): Partial information decomposition

Partial information decomposition decomposes two source information acting on the common target into four information parts: joint information (synergy), unique information from x, unique information from y and shared information (redundancy). It also can be applied to explore the cooperative or competitive regulation mechanism of two miRNAs on the common target mRNA.

library(Informeasure)
library(SummarizedExperiment)

load(system.file("extdata/tcga.brca.testdata.Rdata", package="Informeasure"))

miRNAexpression <- as.matrix(miRNAexpression)
se.miRNAexpression = SummarizedExperiment(assays = list(miRNAexpression = miRNAexpression))

mRNAexpression <- as.matrix(mRNAexpression)
se.mRNAexpression = SummarizedExperiment(assays = list(mRNAexpression = mRNAexpression))

assays(se.miRNAexpression)[["log2"]] <- log2(assays(se.miRNAexpression)[["miRNAexpression"]] + 1)

assays(se.mRNAexpression)[["log2"]] <- log2(assays(se.mRNAexpression)[["mRNAexpression"]] + 1)

x <- assays(se.miRNAexpression["hsa-miR-34a-5p", ])$log2
y <- assays(se.miRNAexpression["hsa-miR-34b-5p", ])$log2
z <- assays(se.mRNAexpression["MYC", ])$log2

XYZ <- discretize3D(x,y,z)

PID.measure(XYZ)
##>    Synergy  Unique_X    Unique_Y Redundancy      PID
##> 1 0.670815 0.1854147 0.003058109  0.2032112 1.062499

2.5 PMI.measure(): Part mutual information

Part mutual information devotes to measuring the non-linearly direct dependencies between two random variables given a third, especially when any one variable has a potentially strong correlation with the third one. Such characteristics of part mutual information are also fully applicable to the ceRNA network inference.

library(Informeasure)
library(SummarizedExperiment)

load(system.file("extdata/tcga.brca.testdata.Rdata", package="Informeasure"))

lncRNAexpression <- as.matrix(lncRNAexpression)
se.lncRNAexpression = SummarizedExperiment(assays = list(lncRNAexpression = lncRNAexpression))

miRNAexpression <- as.matrix(miRNAexpression)
se.miRNAexpression = SummarizedExperiment(assays = list(miRNAexpression = miRNAexpression))

mRNAexpression <- as.matrix(mRNAexpression)
se.mRNAexpression = SummarizedExperiment(assays = list(mRNAexpression = mRNAexpression))

assays(se.lncRNAexpression)[["log2"]] <- log2(assays(se.lncRNAexpression)[["lncRNAexpression"]] + 1)

assays(se.miRNAexpression)[["log2"]] <- log2(assays(se.miRNAexpression)[["miRNAexpression"]] + 1)

assays(se.mRNAexpression)[["log2"]] <- log2(assays(se.mRNAexpression)[["mRNAexpression"]] + 1)

x <- assays(se.miRNAexpression["hsa-miR-26a-5p", ])$log2
y <- assays(se.mRNAexpression["PTEN", ])$log2
z <- assays(se.lncRNAexpression["PTENP1", ])$log2

XYZ <- discretize3D(x,y,z)

PMI.measure(XYZ)
##> [1] 1.074813

3 Conclusions

This package provides implementations of five currently popular information measures. The base installation of this package allows users to approach these information measures to infer bivariate even multivariate biological regulatory networks. But please be noted that the provided package is not only limited to bioinformatics applications. Optionally other research fields can also employ this package to generally evaluate information relations between variables.

4 References

[1] Wyner A D. A definition of conditional mutual information for arbitrary ensembles[J]. Information & Computation, 1978, 38(1): 51-59.

[2] Mcgill W J. Multivariate information transmission[J]. Psychometrika, 1954, 19(2): 97-116.

[3] Williams P L, Beer R D. Nonnegative Decomposition of Multivariate Information[J]. arXiv: Information Theory, 2010.

[4] Zhao J, Zhou Y, Zhang X, et al. Part mutual information for quantifying direct associations in networks[J]. Proceedings of the National Academy of Sciences of the United States of America, 2016, 113(18): 5130-5135.

[5] Hausser J. and Strimmer K. Entropy inference and the James-Stein estimator, with application to nonlinear gene association networks[J]. The Journal of Machine Learning Research, 2009, 10, 1469-1484.

5 Session information

sessionInfo()
##> R version 4.0.3 (2020-10-10)
##> Platform: x86_64-pc-linux-gnu (64-bit)
##> Running under: Ubuntu 18.04.5 LTS
##> 
##> Matrix products: default
##> BLAS:   /home/biocbuild/bbs-3.12-bioc/R/lib/libRblas.so
##> LAPACK: /home/biocbuild/bbs-3.12-bioc/R/lib/libRlapack.so
##> 
##> locale:
##>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
##>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=C              
##>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
##>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
##>  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
##> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
##> 
##> attached base packages:
##> [1] parallel  stats4    stats     graphics  grDevices utils     datasets 
##> [8] methods   base     
##> 
##> other attached packages:
##>  [1] Informeasure_1.0.0          SummarizedExperiment_1.20.0
##>  [3] Biobase_2.50.0              GenomicRanges_1.42.0       
##>  [5] GenomeInfoDb_1.26.0         IRanges_2.24.0             
##>  [7] S4Vectors_0.28.0            BiocGenerics_0.36.0        
##>  [9] MatrixGenerics_1.2.0        matrixStats_0.57.0         
##> [11] BiocStyle_2.18.0           
##> 
##> loaded via a namespace (and not attached):
##>  [1] knitr_1.30             XVector_0.30.0         magrittr_1.5          
##>  [4] zlibbioc_1.36.0        lattice_0.20-41        rlang_0.4.8           
##>  [7] stringr_1.4.0          tools_4.0.3            grid_4.0.3            
##> [10] xfun_0.18              htmltools_0.5.0        yaml_2.2.1            
##> [13] digest_0.6.27          bookdown_0.21          Matrix_1.2-18         
##> [16] GenomeInfoDbData_1.2.4 BiocManager_1.30.10    bitops_1.0-6          
##> [19] RCurl_1.98-1.2         evaluate_0.14          rmarkdown_2.5         
##> [22] entropy_1.2.1          DelayedArray_0.16.0    stringi_1.5.3         
##> [25] compiler_4.0.3