Annotation Enrichment Analysis - An Alternative Method for Evaluating the Functional Properties of Gene Sets

Overview

• Recent high-throughput methods (microarray, RNA-Seq, etc) made it easy to produce large datasets comparing samples in different conditions.
• The end result of many of these analyses, however, is often a large list of genes that are associated with one condition or the other.
• Numerous tools have been developed to look for "enrichment" in these resulting gene sets for genes associated with a particular known pathway or functional annotation.
• These methods (GSEA, etc) often use statistics which make some assumptions about the distribution of annotations which may not be valid.
• What are the effects of these assumptions the resulting interpretation?
• Can we do better?

One example of where this kind of enrichment analysis could be useful is for determining possible roles for clusters of co-expressed genes.

T. cruzi co-expression network modules detected by WGCNA

Gene Ontology structure

(source: http://www.geneontology.org/GO.ontology.structure.shtml)

Huang et al. (2009) Table 1

• Fisher's Exact Test (FET)
• Binomial test
• Hypergeometric test
• Chi-squared test

All of these methods assume that, under the null hypothesis, genes are equally likely to be selected.

• Most popular tool for enrichment analysis
• Uses variant of Kolmogorov–Smirnov test
  • Compares distributions of two samples
  • Null hypothesis: the samples were drawn from the same distribution
• Looks for enrichment in genes with a known property (e.g. GO annotation) at the top of a list of genes ranked by differential expression, etc.

Subramanian et al. (2005) Figure 1

• Beißbarth & Speed (2004)
• Computes frequencies of all GO terms in two sets of genes:
  • Experiment set
  • Reference set (e.g. entire GO db)
• Uses \(\chi^2\) and Fisher's Exact test to look for terms which are enriched in either gene set with respect to the other.
• Performs multiple testing correction using either Holm or Benjamini and Hochberg correction.
• Website: http://gostat.wehi.edu.au/
• Not to be confused with "GOstats", a Bioconductor package for working with GO and microarray data...

• Young et al. (2010) notice biases between gene length, number of reads, and differential expression.
• They show that GO categories also show a length bias (many categories have significantly longer or shorter genest than expected by chance), which indicates that enrichment results could in turn be skewed by DE length bias.
• GOSeq corrects for the length bias by random sampling of a fitted distribution based on length and DE proportion.

Young et al. (2010) Figure 2

Gene and term degree distributions

• Biological Process terms dominate the human annotations.
• Degree of term (\(k_t\)) distribution is "heavy-tailed"; most terms are associated with only a few genes, but some terms are used for a huge number of genes.

Biological Process

The remainder of the results are based on the biological process ontology:

• 656,783 annotations
• 15,213 genes (avg: 43.2 annotations)
• 10192 terms (avg: 64.4 annotations)

Experiment design:

• Created 200 random gene sets:
  • \(N_g\)=200 genes in each set (a "typical" gene set size)
  • Varied number of annotations (\(M_g\))
  • Determined FET enrichment score for each of the 10192 BP GO terms

Results

• Number of unique annotations \(\propto\) GO enrichment significance!

• Multiple testing correction alone is not enough to deal with this bias, although it does seem to severely reduce the problem.

FET enrichment bias towards well-annotated genes is also present in biological datasets:

Experiment design:

• For FET and AEA, each:
  • Selected ~40 GeneSigDB signatures with the most significant enrichment scores.
  • Select 40 GO terms with most significant enrichment scores across all signatures.
• Performed hierarchical clustering.

• Created random term sets with same number of unique genes annotated as found in real GO branches
• Measured FET/AEA enrichment in random and real go branches
• FET find similar numbers of "significant" term-signature associations in the real and random branches!

• Biases exist in GO and other annotation databases.
• These biases can affect the performance of statistics such as FET in predicting significant enrichment.
• Annotation Enrichment Analysis (AEA) accounts for these biases and is able is not as prone to detecting spurious enrichments.

• Performance of AEA only compared with Fisher's Exact Test (FET); how does the performance compare to other GO methods?
• Only looked at Biological Process ontology -- is the picture the same for the other GO sub-ontologies? Other annotation databases?
• Currently only implemented in C++ (R bindings would be nice.)
• Does not take into account any of the addition information about the members of the experimental gene set (e.g. DE fold-change, p-value, etc)