1 Introduction

By design, the scope of this package is limited to defining the SingleCellExperiment class and some minimal getter and setter methods. For this reason, we leave it to developers of specialized packages to provide more advanced methods for the SingleCellExperiment class. If packages define their own data structure, it is their responsibility to provide coercion methods to/from their classes to SingleCellExperiment.

For developers, the use of SingleCellExperiment objects within package functions is mostly the same as the use of instances of the base SummarizedExperiment class. The only exceptions involve direct access to the internal fields of the SingleCellExperiment definition. Manipulation of these internal fields in other packages is possible but requires some caution, as we shall discuss below.

2 Using the internal fields

2.1 Rationale

We use an internal storage mechanism to protect certain fields from direct manipulation by the user. This ensures that only a call to the provided setter methods can change the size factors. The same effect could be achieved by reserving a subset of columns (or column names) as “private” in colData() and rowData(), though this is not easily implemented.

The internal storage avoids situations where users or functions can silently overwrite these important metadata fields during manipulations of rowData or colData. This can result in bugs that are difficult to track down, particularly in long workflows involving many functions. It also allows us to add new methods and metadata types to SingleCellExperiment without worrying about overwriting user-supplied metadata in existing objects.

Methods to get or set the internal fields are exported for use by developers of packages that depend on SingleCellExperiment. This allows dependent packages to store their own custom fields that are not meant to be directly accessible by the user. However, this requires some care to avoid conflicts between packages.

2.2 Conflicts between packages

The concern is that package A and B both define methods that get/set an internal field X in a SingleCellExperiment instance. Consider the following example object:

library(SingleCellExperiment)
counts <- matrix(rpois(100, lambda = 10), ncol=10, nrow=10)
sce <- SingleCellExperiment(assays = list(counts = counts))
sce
## class: SingleCellExperiment 
## dim: 10 10 
## metadata(0):
## assays(1): counts
## rownames: NULL
## rowData names(0):
## colnames: NULL
## colData names(0):
## reducedDimNames(0):
## spikeNames(0):
## altExpNames(0):

Assume that we have functions that set an internal field X in packages A and B.

# Function in package A:
AsetX <- function(sce) {
    int_colData(sce)$X <- runif(ncol(sce))
    sce
}

# Function in package B:
BsetX <- function(sce) {
    int_colData(sce)$X <- sample(LETTERS, ncol(sce), replace=TRUE)
    sce
}

If both of these functions are called, one will clobber the output of the other. This may lead to nonsensical results in downstream procedures.

sce2 <- AsetX(sce)
int_colData(sce2)$X
##  [1] 0.1558390 0.8582489 0.1092469 0.6707894 0.5429999 0.2728018 0.1106492
##  [8] 0.4274946 0.8258717 0.5589302
sce2 <- BsetX(sce2)
int_colData(sce2)$X
##  [1] "O" "A" "R" "T" "I" "Y" "G" "C" "Z" "C"

2.3 Using “Inception-style” nesting

We recommend using nested DataFrames to store internal fields in the column-level metadata. The name of the nested element should be set to the package name, thus avoiding clashes between fields with the same name from different packages.

AsetX_better <- function(sce) {
    int_colData(sce)$A <- DataFrame(X=runif(ncol(sce)))
    sce
}

BsetX_better <- function(sce) {
    choice <- sample(LETTERS, ncol(sce), replace=TRUE)
    int_colData(sce)$B <- DataFrame(X=choice)
    sce
}

sce2 <- AsetX_better(sce)
sce2 <- BsetX_better(sce2)
int_colData(sce2)$A$X 
##  [1] 0.92446836 0.39307308 0.12410404 0.37190763 0.25395559 0.28054722
##  [7] 0.77127703 0.06545623 0.50001883 0.24447166
int_colData(sce2)$B$X 
##  [1] "M" "O" "V" "J" "B" "K" "M" "Y" "C" "Q"

The same approach can be applied to the row-level metadata, e.g., for some per-row field Y.

AsetY_better <- function(sce) {
    int_elementMetadata(sce)$A <- DataFrame(Y=runif(nrow(sce)))
    sce
}

BsetY_better <- function(sce) {
    choice <- sample(LETTERS, nrow(sce), replace=TRUE)
    int_elementMetadata(sce)$B <- DataFrame(Y=choice)
    sce
}

sce2 <- AsetY_better(sce)
sce2 <- BsetY_better(sce2)
int_elementMetadata(sce2)$A$Y 
##  [1] 0.15474673 0.95059953 0.39093855 0.72689509 0.32408032 0.34374388
##  [7] 0.03141815 0.60165479 0.30680826 0.01989538
int_elementMetadata(sce2)$B$Y
##  [1] "J" "O" "Q" "E" "I" "Q" "R" "W" "N" "K"

For the object-wide metadata, a nested list is usually sufficient.

AsetZ_better <- function(sce) {
    int_metadata(sce)$A <- list(Z = "Aaron")
    sce
}

BsetZ_better <- function(sce) {
    int_metadata(sce)$B <- list(Z = "Davide")
    sce
}

sce2 <- AsetZ_better(sce)
sce2 <- BsetZ_better(sce2)
int_metadata(sce2)$A$Z
## [1] "Aaron"
int_metadata(sce2)$B$Z
## [1] "Davide"

In this manner, both A and B can set their internal X, Y and Z without interfering with each other. Of course, this strategy assumes that packages do not have the same names as some of the in-built internal fields (which would be very unfortunate).

3 Contacting us

If your package accesses the internal fields of the SingleCellExperiment class, we suggest you get into contact with us on GitHub. This will help us in planning changes to the internal organization of the class. It will also allow us to contact you with respect to changes or to get feedback.

We are particularly interested in scenarios where multiple packages are defining internal fields with the same scientific meaning. In such cases, it may be valuable to provide getters and setters for this field in SingleCellExperiment directly. This reduces redundancy in the definitions across packages and promotes interoperability. For example, methods from one package can set the field, which can then be used by methods of another package.

4 Other design decisions

4.1 What’s up with reducedDims?

We use a SimpleList as the reducedDims slot to allow for multiple dimensionality reduction results. One can imagine that different dimensionality reduction techniques will be useful for different aspects of the analysis, e.g., t-SNE for visualization, PCA for pseudo-time inference. We see reducedDims as a similar slot to assays() in that multiple matrices can be stored, though the dimensionality reduction results need not have the same number of dimensions.

4.2 Why derive from a RangedSummarizedExperiment?

We decided to extend RangedSummarizedExperiment rather than SummarizedExperiment because for certain assays it will be essential to have rowRanges(). Even for RNA-seq, it is sometimes useful to have rowRanges() and other classes to define the genomic coordinates, e.g., DESeqDataSet in the DESeq2 package. An alternative would have been to have two classes, SingleCellExperiment and RangedSingleCellExperiment. However, this seems like an unnecessary duplication as having a class with default empty rowRanges seems good enough when one does not need rowRanges.

4.3 Why not use a MultiAssayExperiment?

Another approach to storing alternative Experiments would be to use a MultiAssayExperiment. We do not do so as the vast majority of scRNA-seq data analyses operate on the endogenous genes. Switching to a MultiAssayExperiment introduces an additional layer of indirection with no benefit in most cases. Indeed, the methods of this class are largely unnecessary when the alternative Experiments contain data for the same samples. By storing nested Experiments, we maintain the familiar SummarizedExperiment interface for better compatibility and ease of use.

5 Session information

sessionInfo()
## R version 3.6.1 (2019-07-05)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows Server 2012 R2 x64 (build 9600)
## 
## Matrix products: default
## 
## locale:
## [1] LC_COLLATE=C                          
## [2] LC_CTYPE=English_United States.1252   
## [3] LC_MONETARY=English_United States.1252
## [4] LC_NUMERIC=C                          
## [5] LC_TIME=English_United States.1252    
## 
## attached base packages:
## [1] parallel  stats4    stats     graphics  grDevices utils     datasets 
## [8] methods   base     
## 
## other attached packages:
##  [1] SingleCellExperiment_1.8.0  SummarizedExperiment_1.16.0
##  [3] DelayedArray_0.12.0         BiocParallel_1.20.0        
##  [5] matrixStats_0.55.0          Biobase_2.46.0             
##  [7] GenomicRanges_1.38.0        GenomeInfoDb_1.22.0        
##  [9] IRanges_2.20.0              S4Vectors_0.24.0           
## [11] BiocGenerics_0.32.0         BiocStyle_2.14.0           
## 
## loaded via a namespace (and not attached):
##  [1] Rcpp_1.0.2             knitr_1.25             XVector_0.26.0        
##  [4] magrittr_1.5           zlibbioc_1.32.0        lattice_0.20-38       
##  [7] rlang_0.4.1            stringr_1.4.0          tools_3.6.1           
## [10] grid_3.6.1             xfun_0.10              htmltools_0.4.0       
## [13] yaml_2.2.0             digest_0.6.22          bookdown_0.14         
## [16] Matrix_1.2-17          GenomeInfoDbData_1.2.2 BiocManager_1.30.9    
## [19] bitops_1.0-6           RCurl_1.95-4.12        evaluate_0.14         
## [22] rmarkdown_1.16         stringi_1.4.3          compiler_3.6.1