(1) Installation and Quick-Start

This section provides instructions for installation and a quick working example to demonstrate the important functions of NxtIRF. NxtIRFcore is the command line utility for NxtIRF.

For detailed explanations of each step shown here, refer to chapter 2: “Explaining the NxtIRF workflow” in this vignette. For a list of ready-made “recipes” for typical-use NxtIRF in real datasets, refer to chapter 3: “NxtIRF cookbook”

Installation

To install NxtIRFcore, start R (version “4.1”) and enter:

(Optional) For MacOS users, make sure OpenMP libraries are installed correctly. We recommend users follow this guide, but the quickest way to get started is to install libomp via brew:

Building the NxtIRF reference

A NxtIRF reference requires a genome FASTA file (containing genome nucleotide sequences) and a gene annotation GTF file (preferably from Ensembl or Gencode).

NxtIRF provides an example genome and gene annotation which can be accessed via the NxtIRFdata package installed with NxtIRF:

Using these two files, we construct a NxtIRF reference as follows:

Running IRFinder

NxtIRF provides an example set of 6 BAM files to demonstrate its use via this vignette.

Firstly, retrieve the BAM files from ExperimentHub using the NxtIRF helper function NxtIRF_example_bams(). This makes a copy of the BAM files to the temporary directory:

Finally, run NxtIRF/IRFinder as follows:

Collate individual IRFinder runs to build a NxtIRF Experiment

First, collate the IRFinder output files using the helper function Find_IRFinder_Output()

This creates a 3-column data frame with sample name, IRFinder gzipped text output, and COV files. Compile these output files into a single experiment:

Importing the collated data as a NxtSE object:

The NxtSE is a data structure that inherits SummarizedExperiment

(2) Explaining the NxtIRF Workflow

This section explains the working example provided in the prior “Quick-Start” section, to demonstrate how to use NxtIRF.

Generating the NxtIRF reference

NxtIRF first needs to generate a set of reference files. The NxtIRF reference is used to quantitate IR and alternative splicing, as well as in downstream visualisation tools.

Building an example NxtIRF reference using BuildReference()

First, load the NxtIRF package:

NxtIRF generates a reference from a user-provided genome FASTA and genome annotation GTF file, and is optimised for Ensembl references but can accept other reference GTF files. Alternatively, NxtIRF accepts AnnotationHub resources, using the record names of AnnotationHub records as input.

We will first demonstrate a runnable example using the included example NxtIRF genome. This was created using 7 example genes (SRSF1, SRSF2, SRSF3, TRA2A, TRA2B, TP53 and NSUN5). The SRSF and TRA family of genes all contain poison exons flanked by retained introns. Additionally, NSUN5 contains an annotated IR event in its terminal intron. Sequences from these 7 genes were aligned into one sequence to create an artificial chromosome Z (chrZ). The gene annotations were modified to only contain the 7 genes with the modified genomic coordinates.

The first function, GetReferenceResource(), requires 3 arguments: (1) fasta The file (or url) of the genome FASTA file, (2) gtf The file (or url) of the gene annotation GTF file, and (3) reference_path The directory (ideally an empty directory) to create the NxtIRF reference

GetReferenceResource() processes the given source genome / annotation files, downloading them where necesssary, and saves a local copy of the GTF file as well as a compressed version of the FASTA sequence file (as a TwoBitFile which is a binary compressed version of the genome sequence).

The next function, BuildReference(), uses the resources to build the NxtIRF reference. NxtIRF builds its own version of alternative splicing annotations by comparing differential combinations of splice junctions between the transcripts of each gene. Events annotated include Skipped Exons (SE), Mutually Exclusive Exons (MXE), Alternative First / Last Exons (AFE / ALE), Alternative 5’ or 3’ Splice Sites (A5SS / A3SS), and Intron Retention (IR). For IR events, every intron is considered as a potential Intron Retention event, assuming most IR events are not annotated.

Additionally, BuildReference assesses the coding sequence changes that arise if each individual IR event occurs, annotating the IR events as NMD-inducing if the inclusion of the intron results in premature termination codons (PTCs, as defined by a stop codon located 55 bases upstream of the last exon junction).

Quantitate IR and Alternative Splicing in Aligned BAM files

NxtIRF adopts the IRFinder algorithm to measure IR in aligned BAM files. The IRFinder algorithm also provides spliced junction counts that is used by NxtIRF to quantitate alternate splicing events.

Running IRFinder() on example BAM files

In this vignette, we provide 6 example BAM files. These were generated based on aligned RNA-seq BAMs of 6 samples from the Leucegene AML dataset (GSE67039). Sequences aligned to hg38 were filtered to only include genes aligned to that used to create the chrZ chromosome. These sequences were then re-aligned to the chrZ reference using STAR.

First, we download the example bam files using the following convenience function:

Often, alignment pipelines process multiple samples. NxtIRF provides convenience functions to recursively locate all the BAM files in a given folder, and tries to ascertain sample names. Often sample names can be gleaned when: * The BAM files are named by their sample names, e.g. “sample1.bam”, “sample2.bam”. In this case, level = 0 * The BAM files have generic names but are contained inside parent directories labeled by their sample names, e.g. “sample1/Unsorted.bam”, “sample2/Unsorted.bam”. In this case, level = 1

This convenience function retrieves a data frame with the first and second columns as the sample names and paths of all the BAM files found. We use this to run IRFinder on all the BAM files using the following:

This runs IRFinder using 2 threads. Multi-threading is performed using OpenMP (where available). If OpenMP is not installed when NxtIRFcore was compiled, BiocParallel will be used instead. Setting overwrite = FALSE ensures if IRFinder files were generated by a previous run, these would not be overwritten.

Also (optionally), users can generate gene counts from the same reference using run_featureCounts = TRUE (requires the Rsubread package). The featureCounts output is stored in the “main.FC.Rds” file which can be retrieved using below:

Collating IRFinder output into a single dataset

IRFinder produces output on individual samples. NxtIRF makes it easy to collate the output from multiple samples. As some splicing events may occur in some samples but not others, it is important to unify the data and organise it into a single data structure. NxtIRF uses a specialized class (called the NxtSE object) to contain multiple arrays of data. This is used to contain Included and Excluded event counts as well as associated quality control data (including intronic “coverage” which is the fraction of the intron covered by RNA-seq reads in IR events).

Reviewing the IRFinder output:

Having run IRFinder on all individual BAM files of an experiment, we can collate this data using the CollateData() function. Again, we use a convenience function to generate a list of IRFinder output files:

expr is a 3-column data frame. The first two columns contain the sample names and IRFinder output file path (as a “.txt.gz” - gzipped text file). This contains all of the original output from vanilla IRFinder, as well as sample QC readouts). Feel free to unzip a file to see what output is generated.

The third column contains “COV” file paths. COV files are generated by NxtIRF and contains RNA-seq read coverage data in a novel compressed format. COV files compress data much better than BigWig files, and also contains RNA-seq coverage data from individual strands. Currently, only NxtIRF is able to read COV files but we anticipate other packages can use COV files via NxtIRF (in future NxtIRF releases)! We will demonstrate how to use COV files in a later section.

Using CollateData to parse IRFinder output from multiple files.

Now that we have an organised list of IRFinder output files, we parse this into CollateData()

CollateData extracts IR quantitation and junction counts and organises these counts into unified arrays of data. CollateData also compiles QC parameters for all samples, including read depth and strandedness (directionality).

IRMode is a parameter that specifies how IRFinder calculates percentage intron retention (PIR). Previously, IRFinder estimates spliced (or exonic) abundance by including junction reads that involve either flanking exon SpliceMax = max(SpliceLeft, SpliceRight). This is done to correct for the possibility of alternate exons flanking the intron being assessed. NxtIRF extends this estimate by accounting for the possibility that BOTH flanking exons may be alternate exons, thereby accounting for splice events that overlap the intron but does not involve the junctions of either flanking exons. We call this parameter SpliceOverMax, which includes SpliceMax with the addition of distant splice events.

Building a NxtSE object from collated data

Now that CollateData() has created the unified data in the directory, we can retrieve the data as a NxtSE object:

Two things of note:

  1. By default, MakeSE constructs the NxtSE object using all the samples in the collated data. It is possible (and particularly useful in large data sets) to read only a subset of samples. In this case, construct a data.frame object with the first column containing the desired sample names and parse this into the colData parameter as shown:
  1. In complex transcriptomes including those of human and mouse, alternative splicing implies that introns are often overlapping. Thus, algorithms run the risk of over-calling intron retention where overlapping introns are assessed. NxtIRF removes overlapping introns by considering only introns belonging to the major splice isoforms. It estimates a list of introns of major isoforms by assessing the compatible splice junctions of each isoform, and removes overlapping introns belonging to minor isoforms. To disable this functionality, set RemoveOverlapping = FALSE.

Filtering for Expressed Splicing Events

Often, the gene annotations contain isoforms for all discovered splicing events. Most annotated transcripts are not expressed, and their inclusion in differential analysis complicates results including adjusting for multiple testing. It is prudent to filter these out using various approaches, akin to removing genes with low gene counts in differential gene analysis. We suggest using the default filters which generally excludes splicing events whereby the total included / excluded event counts less than 20 RNA-seq reads. There are other quality-control filters included but these are out of the scope of this vignette (please see the documentation for details).

To filter the NxtSE object using default filters:

In the above code, get_default_filters() retrieves the default filters suggested by NxtIRF. apply_filters returns a vector containing which events are included if the filters are applied to the NxtSE object. Finally, we obtain a NxtSE object by subsetting the original NxtSE using the vector of expressed_events.

To view a description of what these filters actually do, simply display them:

get_default_filters()
#> [[1]]
#> NxtFilter Class: Data    Type: Depth
#> EventTypes: IR MXE SE A3SS A5SS AFE ALE RI
#> Filter must be passed in at least 80.0 percent of all samples
#> Minimum Event Depth: 20
#> Event Depth refers to number of aligned splice reads plus effective depth of their introns
#> 
#> [[2]]
#> NxtFilter Class: Data    Type: Coverage
#> EventTypes: IR RI
#> Filter must be passed in at least 80.0 percent of all samples
#> Minimum Coverage Percentage: 90.0
#> In retained introns, coverage refers to the proportion of the measured intron that is covered by at least 1 alignment
#> Event Depth below 5 are ignored
#> Event Depth refers to number of aligned splice reads plus effective depth of their introns
#> 
#> [[3]]
#> NxtFilter Class: Data    Type: Coverage
#> EventTypes: MXE SE AFE ALE A5SS A3SS
#> Filter must be passed in at least 80.0 percent of all samples
#> Minimum Coverage Percentage: 60.0
#> In splice events, coverage refers to the proportion of junction reads that belong to either the included or excluded isoforms of the given event
#> Event Depth below 20 are ignored
#> Event Depth refers to number of aligned splice reads plus effective depth of their introns
#> 
#> [[4]]
#> NxtFilter Class: Data    Type: Consistency
#> EventTypes: MXE SE RI
#> Filter must be passed in at least 80.0 percent of all samples
#> Sub-event minimum proportion: 0.2 of total isoform splice count
#> In retained introns, sub-events refer to 5'- and 3'-exon-intron spanning alignments
#> In mutually exclusive exons and skipped exons, sub-events refer to the upstream and downstream splice alignments
#> Event Depth below 20 are ignored
#> Event Depth refers to number of aligned splice reads plus effective depth of their introns

To construct custom filters, create a NxtFilter object as shown:

For more information about the filters available, refer to the manual for details:

Performing analysis of Differential Alternative Splicing Events

Before performing differential analysis, we must annotate our samples. Currently there are no annotations:

To populate the NxtSE object with annotations, we can modify colData just as we would for a SummarizedExperiment object.

We can use either limma to perform differential alternative splicing analysis

NB: we can also use DESeq2 or DoubleExpSeq to perform the same analysis using DESeq_ASE() or DoubleExpSeq_ASE(). The differences are as follows:

  • Limma: models counts as log-normal distributed
  • DESeq2: models counts as negative binomial distributed
  • DoubleExpSeq: models alternative splicing ratios as beta-binomial distributed

Visualisation of Differential ASE Analysis

The following guide demonstrates how to plot standard figures including scatter plots of average percent-spliced-in (PSI) values, volcano plots of differentially expressed IR / AS events, and heatmaps of PSI values.

Scatter plot of differential Alternate Splicing Events

These functions return a data frame with the differential analysis results. These include average percent-spliced-in values for both conditions, which can be used directly to produce a scatter plot comparing the two conditions:

Note the columns in the results are defined by the names of the conditions “A” and “B”.

To filter for specific splicing events (e.g. IR only), simply filter the results data.frame by EventType:

Volcano plot of differential Alternate Splicing Events

limma_ASE and DESeq_ASE contain direct output from limma or DESeq2. The P.Value and pvalue of limma and DESeq2 output denote nominal P values, whereas multiple-correction is performed via the adj.P.Val and padj columns of limma and DESeq2, respectively. To plot these:

Heatmap of differential events

NxtIRF provides convenience functions to retrieve the splicing ratios via the NxtSE object. To produce a heatmap, first we create a matrix of splicing ratios of the top 10 differential events:

make_matrix() has the option of supplying values as logit-transformed. This is useful to contrast where the majority of values are near the 0- or 1- boundary.

Also, when the dataset contains low-abundance splicing depths, samples with splicing depths below a certain threshold can be excluded (replaced with NA). Simply set depth_threshold to a level below which sample-events will be converted to NA. Also, events can be excluded from display if a certain fraction of samples have low coverage (i.e. return NA values). This filter can be set using na.percent.max. This is useful as events with high number of NA values can return errors in heatmap functions that also perform clustering

With the matrix of values, one can produce a heatmap as shown:

NxtIRF Coverage Plots

NxtIRF is able to produce RNA-seq coverage plots of analysed samples. Coverage data is compiled simultaneous to the IR and junction quantitation performed by the IRFinder C++ routine. This data is saved in “COV” files, which is a BGZF compressed and indexed file. COV files show compression and performance gains over BigWig files.

Additionally, NxtIRF performs coverage plots of multiple samples combined based on user-defined experimental conditions. This is a powerful tool to illustrate group-specific differential splicing or IR. NxtIRF does this by normalising the coverage depths of each sample based on transcript depth at the splice junction / intron of interest. By doing so, the coverage depths of constitutively expressed flanking exons are normalised to unity. As a result, the intron depths reflect the fraction of transcripts with retained introns and can be compared across samples.

We will first demonstrate by plotting the RNA-seq coverage of a single gene by a single sample. Plot_Coverage() performs the calculations and generates a compound object containing both static and interactive plots. We can coerce this to a static plot using as_egg_ggplot()

The interactive plot (not run here) can be displayed by directly calling the final_plot element of the object returned by Plot_Coverage

There are many transcripts in TP53! This is because by default, NxtIRF displays all annotated transcripts. For clarity, one can either collapse the transcripts at a per-gene level, by setting condense_tracks = TRUE:

Alternatively, for fine control, one can supply a vector containing the transcript names to be displayed:

In the heatmap in the previous section, we can see some retained introns in NSUN5 that are more highly expressed in “02H003” and “02H025”. To demonstrate this, we first introduce a new condition that groups these two samples:

Performing differential analysis will confirm this to be the case:

We can now visualise the top hit, NSUN5 intron 8. To visualise the IR event across conditions, we specify the type of condition to contrast. The names of the tracks will be the names of the nominator (test) and denominator (control) conditions

Note the lines represent mean coverages of all samples in the labelled experimental conditions. These are achieved by normalising their individual coverages by their transcript abundance, which is calculated by summing the abundances of spliced and intron-retaining transcripts. Thus, the normalised coverage at the specified splice junction is approximately 1.0.

The grey shading represent 95% confidence intervals. It is possible that the shaded areas are larger as we traverse away from the splice junction. This may be because of differences in 3’-sequencing bias between different samples of the same condition.

Although NSUN5 intron 8 is the most differentially retained in the analysis, the difference isn’t very apparent. This is because the difference between the absolute values is very small. In contrast, intron 2 looks more prominently different in the heatmap, so we can explore this as well.

We can further compare the coverage by plotting the two conditions on the same track. This done by setting stack_tracks = TRUE. Further, we can add a track that tests the per-nucleotide statistical difference between the normalised coverage between the two conditions. To do this, set t_test = TRUE:

On the first track, the different coloured traces represent the two conditions. The second track plots the -log10 transformed p values, using Students T-test to test the difference between the coverages between the conditions, at per-nucleotide resolution.

Note that the t-test track does not replace formal statistical analysis using the limma_ASE or DESeq_ASE functions. Instead, they provide a useful adjunct to assess the adequacy of the normalisation of the group coverages.

There are more complex customisation options with NxtIRF’s Plot_Coverage tool which will be explored in subsequent tutorials (TODO).

(3) NxtIRF cookbook

Reference Generation

First, define the path to the directory in which the reference should be stored. This directory will be made by NxtIRF, but its parent directory must exist, otherwise an error will be returned.

Create a NxtIRF reference from user-defined FASTA and GTF files locally:

Note that setting genome_path = "hg38" will prompt NxtIRF to use the default files for nonPolyA and Mappability exclusion references in the generation of its reference. Valid options for genome_path are “hg38”, “hg19”, “mm10” and “mm9”.

Create a NxtIRF reference using web resources from Ensembl’s FTP:

The following will first download the genome and gene annotation files from the online resource and store a local copy of it in a file cache, facilitated by BiocFileCache. Then, it uses the downloaded resource to create the NxtIRF reference.

Create a NxtIRF reference using AnnotationHub resources:

AnnotationHub contains Ensembl references for many genomes. To browse what is available:

For a more specific query:

We wish to fetch “AH65745” and “AH64631” which contains the desired FASTA and GTF files, respectively. To build a reference using these resources:

BuildReference will recognise the inputs of fasta and gtf as AnnotationHub resources as they begin with “AH”.

Create a NxtIRF reference from species other than human or mouse:

For human and mouse genomes, we highly recommend specifying genome_type as the default mappability file is used to exclude intronic regions with repeat sequences from intron retention analysis. For other species, one could generate a NxtIRF reference without this reference:

Alternatively, if STAR is available on the computer or server where R/RStudio is being run, we can use the one-line function BuildReference_Full. This function will: * Prepare the resources from the given FASTA and GTF files * Generate a STAR genome * Use the STAR genome and the FASTA file to de-novo calculate and define low mappability regions * Build the NxtIRF reference using the genome resources and mappability file

n_threads specify how many threads should be used to build the STAR reference and to calculate the low mappability regions

Finally, if STAR is not available, Rsubread is available on Bioconductor to perform mappability calculations. The example code in the manual is displayed here for convenience, to demonstrate how this would be done:

Running IRFinder on BAM files

To conveniently find all BAM files recursively in a given path:

This convenience function returns the putative sample names, either from BAM file names themselves (level = 0), or from the names of their parent directories (level = 1).

To use IRFinder using 4 OpenMP threads:

Creating COV files from BAM files without running IRFinder

Sometimes one may wish to create a COV file from a BAM file without running the IRFinder algorithm. One reason might be because a NxtIRF/IRFinder reference is not available.

To convert a list of BAM files, run BAM2COV(). This is a function structurally similar to IRFinder() but without the need to give the path to the NxtIRF reference:

Collating IRFinder data into a single experiment:

Assuming the NxtIRF reference is in ref_path, after running IRFinder as shown in the previous section, use the convenience function Find_IRFinder_Output() to tabulate a list of samples and their corresponding IRFinder outputs:

This data.frame can be directly used to run CollateData:

Then, the collated data can be imported as a NxtSE object, which is an object that inherits SummarizedExperiment and has specialized containers to hold additional data required by NxtIRF.

Downstream analysis using NxtIRFcore

Please refer to chapters 1 and 2 for worked examples using the NxtIRF example dataset.

SessionInfo

sessionInfo()
#> R version 4.1.1 (2021-08-10)
#> Platform: x86_64-pc-linux-gnu (64-bit)
#> Running under: Ubuntu 20.04.3 LTS
#> 
#> Matrix products: default
#> BLAS:   /home/biocbuild/bbs-3.14-bioc/R/lib/libRblas.so
#> LAPACK: /home/biocbuild/bbs-3.14-bioc/R/lib/libRlapack.so
#> 
#> locale:
#>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
#>  [3] LC_TIME=en_GB              LC_COLLATE=C              
#>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
#>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
#>  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
#> 
#> attached base packages:
#> [1] stats4    stats     graphics  grDevices utils     datasets  methods  
#> [8] base     
#> 
#> other attached packages:
#>  [1] AnnotationHub_3.2.0         BiocFileCache_2.2.0        
#>  [3] dbplyr_2.1.1                pheatmap_1.0.12            
#>  [5] ggplot2_3.3.5               DoubleExpSeq_1.1           
#>  [7] DESeq2_1.34.0               SummarizedExperiment_1.24.0
#>  [9] Biobase_2.54.0              MatrixGenerics_1.6.0       
#> [11] matrixStats_0.61.0          GenomicRanges_1.46.0       
#> [13] GenomeInfoDb_1.30.0         IRanges_2.28.0             
#> [15] S4Vectors_0.32.0            BiocGenerics_0.40.0        
#> [17] limma_3.50.0                NxtIRFcore_1.0.0           
#> [19] NxtIRFdata_0.99.6          
#> 
#> loaded via a namespace (and not attached):
#>   [1] colorspace_2.0-2              rjson_0.2.20                 
#>   [3] ellipsis_0.3.2                XVector_0.34.0               
#>   [5] farver_2.1.0                  bit64_4.0.5                  
#>   [7] interactiveDisplayBase_1.32.0 AnnotationDbi_1.56.0         
#>   [9] fansi_0.5.0                   splines_4.1.1                
#>  [11] R.methodsS3_1.8.1             sparseMatrixStats_1.6.0      
#>  [13] cachem_1.0.6                  geneplotter_1.72.0           
#>  [15] knitr_1.36                    jsonlite_1.7.2               
#>  [17] Rsamtools_2.10.0              annotate_1.72.0              
#>  [19] png_0.1-7                     R.oo_1.24.0                  
#>  [21] shiny_1.7.1                   HDF5Array_1.22.0             
#>  [23] BiocManager_1.30.16           compiler_4.1.1               
#>  [25] httr_1.4.2                    assertthat_0.2.1             
#>  [27] Matrix_1.3-4                  fastmap_1.1.0                
#>  [29] lazyeval_0.2.2                later_1.3.0                  
#>  [31] htmltools_0.5.2               tools_4.1.1                  
#>  [33] gtable_0.3.0                  glue_1.4.2                   
#>  [35] GenomeInfoDbData_1.2.7        dplyr_1.0.7                  
#>  [37] rappdirs_0.3.3                Rcpp_1.0.7                   
#>  [39] jquerylib_0.1.4               vctrs_0.3.8                  
#>  [41] Biostrings_2.62.0             rhdf5filters_1.6.0           
#>  [43] rtracklayer_1.54.0            crosstalk_1.1.1              
#>  [45] DelayedMatrixStats_1.16.0     xfun_0.27                    
#>  [47] stringr_1.4.0                 mime_0.12                    
#>  [49] lifecycle_1.0.1               restfulr_0.0.13              
#>  [51] XML_3.99-0.8                  zlibbioc_1.40.0              
#>  [53] scales_1.1.1                  BSgenome_1.62.0              
#>  [55] promises_1.2.0.1              parallel_4.1.1               
#>  [57] rhdf5_2.38.0                  RColorBrewer_1.1-2           
#>  [59] yaml_2.2.1                    curl_4.3.2                   
#>  [61] gridExtra_2.3                 memoise_2.0.0                
#>  [63] sass_0.4.0                    stringi_1.7.5                
#>  [65] RSQLite_2.2.8                 highr_0.9                    
#>  [67] BiocVersion_3.14.0            genefilter_1.76.0            
#>  [69] BiocIO_1.4.0                  filelock_1.0.2               
#>  [71] BiocParallel_1.28.0           rlang_0.4.12                 
#>  [73] pkgconfig_2.0.3               bitops_1.0-7                 
#>  [75] evaluate_0.14                 lattice_0.20-45              
#>  [77] purrr_0.3.4                   Rhdf5lib_1.16.0              
#>  [79] labeling_0.4.2                GenomicAlignments_1.30.0     
#>  [81] htmlwidgets_1.5.4             bit_4.0.4                    
#>  [83] tidyselect_1.1.1              magrittr_2.0.1               
#>  [85] R6_2.5.1                      generics_0.1.1               
#>  [87] DelayedArray_0.20.0           DBI_1.1.1                    
#>  [89] withr_2.4.2                   pillar_1.6.4                 
#>  [91] survival_3.2-13               KEGGREST_1.34.0              
#>  [93] RCurl_1.98-1.5                tibble_3.1.5                 
#>  [95] crayon_1.4.1                  utf8_1.2.2                   
#>  [97] plotly_4.10.0                 rmarkdown_2.11               
#>  [99] locfit_1.5-9.4                grid_4.1.1                   
#> [101] data.table_1.14.2             blob_1.2.2                   
#> [103] digest_0.6.28                 xtable_1.8-4                 
#> [105] numDeriv_2016.8-1.1           tidyr_1.1.4                  
#> [107] httpuv_1.6.3                  R.utils_2.11.0               
#> [109] munsell_0.5.0                 fst_0.9.4                    
#> [111] viridisLite_0.4.0             bslib_0.3.1                  
#> [113] egg_0.4.5