Gene Symbol and Ontology Drift

Zetobit · Bioinformatics Insight Series Annotation & Provenance

Bioinformatics Insight Series

Gene Symbol and Ontology Drift

Why the Same Analysis Compares Different Gene Sets in Different Years

Kanna Nandakumar, PhD · Zetobit, LLC

A gene symbol looks like the most solid thing in a results table. It is a short string, it names a real object, and it joins cleanly to anything else that uses the same string. Every step of that impression is wrong in a way that matters, and it matters most in exactly the situation where a symbol seems most useful: comparing something measured now against something measured, curated or published earlier.

Symbols are labels, and labels are maintained. They get renamed when a gene’s function is understood better, retired when a gene turns out not to exist, merged when two entries prove to be one, and — the case that causes real damage — used simultaneously as the approved symbol for one gene and an alias for a different one. Meanwhile the pathway and ontology databases that give those symbols meaning are revised continuously underneath any analysis that queries them.

None of this is a defect. A nomenclature that never changed would be a nomenclature that had stopped tracking biology. The problem is that a join on a text string cannot tell the difference between a label that means the same thing it used to and one that does not.

Four things that happen to a symbol, and only two of them are visible

The renames are the familiar case. GCN5L2 became KAT2A once the protein was understood as a histone acetyltransferase¹. MLL became KMT2A — a change that clinical laboratories still work around, reporting the new symbol with the old one in parentheses because oncologists and three decades of literature use the old one².

The spreadsheet mangling is the famous case. Excel autoformatting converts SEPT7 to a date and DEC1 to 1-Dec, an issue first reported in 2004³. In 2016, a survey of supplementary files from 3,597 published papers found roughly one in five affected⁴. A follow-up in 2021 reported that the errors continue to accumulate year on year and that the proportion of affected articles has been stable since 2014 — while noting the three properties that make spreadsheet errors particularly corrosive: they occur silently, they hide among thousands of rows, and they are difficult to audit⁵. HGNC has since renamed the affected genes (SEPT1 to SEPTIN1, MARCH1 to MARCHF1) so that new data is protected⁶. That fix does nothing for the archives, and it is itself a rename, which is the first problem.

Both of those failures are loud in the sense that counts: they produce no match. Your join drops the row, and if you look, you can count what you lost.

The other two failures produce a match. That is what makes them dangerous.

Merges and withdrawals. HGNC distinguishes a “symbol withdrawn” — a gene that now has a different approved symbol — from an “entry withdrawn,” a symbol for a gene that has since been shown not to exist⁷. A row carrying an entry-withdrawn symbol is a measurement of something the field no longer believes is there. Nothing about the string announces that. And ambiguity can run the other way: correcting the Excel-mangled 1-Mar yields two candidate genes, MARCHF1 and MTARC1, because both were mangled to the same date¹.

Reuse and collision. This is the sharpest case, and it is best shown by example. FHL1 is a valid approved symbol. It is also an alias for CFH, a completely different gene on a different chromosome. Given the string FHL1 and nothing else, a correction tool will return it as valid — correctly — even when the source data meant CFH; only supplying the chromosome disambiguates it¹. The reverse pattern exists too: NIP is not an approved symbol for anything, but is an alias for GIPC1, DUOXA1 and CRPPA simultaneously¹.

This is not a handful of curiosities. A 2023 evaluation of NCBI Gene and HGNC found 3,940 gene records (2.3%) whose aliases identically matched the primary symbol of a different gene record — the authors’ illustration being that KRAS is the primary symbol of one gene and an alias of the related but distinct NRAS⁸. Earlier text-mining work put the homonym problem higher still, finding that 33% of genes in a combined thesaurus had one or more ambiguous symbols⁹.

Figure 1. The four outcomes when an older gene list is joined to a current annotation. The examples are real; the framing is the point. Most tooling and most discussion of this problem addresses the top two rows, which are the two that announce themselves.

How much of an archive is affected

The numbers are larger than the anecdotes suggest, and they scale with the age of the data. Across 20,716 GEO platform annotation files, the proportion of invalid gene symbols averaged around 3% in recent platforms, rising to about 20% for 2010-era platforms and 30–40% in the earliest, from 2002–03. Overall, 79% of symbols were valid, rising to 92% after automated correction¹.

Read that as a statement about longitudinal work. If you compare a 2010 dataset against a 2024 dataset by joining on gene symbols, roughly a fifth of the older list does not join — and the genes that fail are not a random sample, because renaming is driven by which genes got studied enough for their function to be reclassified.

It is worth being concrete about what that looks like in practice, because the failure mode is undramatic. A script reads two expression tables, merges them on the symbol column, and returns a data frame with fewer rows than either input. Nothing errors. Nothing warns. The default behaviour of an inner join is to keep what matched, and every downstream step — normalisation, differential testing, multiple-testing correction, enrichment — operates correctly on the survivors. The multiple-testing denominator quietly shrinks. The enrichment background quietly changes. The result is internally consistent and answers a question about a gene set nobody chose.

Curated resources are not exempt. The same analysis found 850 invalid symbols among 38,040 in MSigDB 7.0; correction reduced that to 453, mostly lncRNAs and a few withdrawn symbols¹. The gene sets you test against carry the same drift as the data you test.

The place this compounds worst is signature reuse. A published prognostic signature is a list of symbols frozen at publication date; applying it to a new cohort is a join between that frozen list and a current expression matrix. If four genes out of forty fail to map, the signature has been silently redefined, and its score is no longer the score that was validated. The authors of the symbol-correction work name exactly these use cases — reanalysis of targeted gene panels, gene set enrichment using symbol-based databases, and systematic review or meta-analysis of published multi-gene signatures — as the places where correction should be routine¹.

There is also a bias in which genes get renamed that has a direct effect on enrichment analysis. Genes whose symbols differed between 2010 and 2016 had significantly fewer annotations in 2010 than consistently named genes — an average of 3.1 versus 9.3 annotations per gene¹⁰. Symbol instability concentrates in the poorly characterised part of the genome, which is the part where a new finding is most likely to be interesting.

The genes most likely to have been renamed are the genes we understood least at the time. Those are also the genes a new result is most likely to be about.

The database moves too

Symbols are half the problem. The other half is that the pathway and ontology resources those symbols are looked up in are revised continuously, so an identical gene list produces different enrichment results depending on when you ran it.

The clearest measurement of this is a 2016 survey that examined update times for 25 web-based pathway enrichment tools alongside citations in 3,879 publications. Most tools were outdated by years; ten (42%) by five or more, including DAVID, whose annotations dated from January 2010¹⁰. Testing essential genes from 77 breast cancer cell lines, 2010-era annotations missed 74% of the terms found with 2016 annotations (191 versus 695). On a set of 75 significantly mutated glioblastoma genes, 2010 annotations captured only about 20% of the 2016 results — 172 of 827 GO biological process terms, and 16 of 128 Reactome pathways¹⁰. The vocabulary of pathways and processes roughly doubled over seven years¹¹.

An independent analysis reached the same conclusion from the other direction. Using gene signatures from 104 disease analyses spanning more than 23,000 samples, one group found low consistency between enrichment results computed with early versus recent GO versions, and noted a persistent annotation bias in which 58% of annotations are attached to 16% of human genes¹². Their recommendation was blunt: re-examine previous analyses with the current GO version¹².

A caveat, because the literature is not unanimous. A large-scale study using a database of historical GO versions confirmed that enrichment results can be unstable, but also found the effects are not uniform across datasets nor easily predictable, and that changes over months are minor while changes over years are substantial¹³. Earlier work had reported stable results or characterised the changes as improvements from focused curation¹³. The honest summary is that annotation drift is real, its magnitude is dataset-specific, and much of it represents genuine knowledge accumulation. Which does not make a comparison across versions valid — it makes it a comparison of two different states of knowledge, presented as a comparison of two biological conditions.

Two datasets, two years, one join

Layer	What changes between versions	Effect on a longitudinal comparison
Gene symbols	Renames, merges, withdrawals, reuse of retired symbols	Rows drop out silently; some rows match the wrong gene
Genome annotation	Gene models added, boundaries revised, new biotypes included	Counts for a given gene change even with identical reads14
Gene set definitions	Sets added, split, merged; membership revised at each release	The tested hypothesis changes between runs
Ontology structure	Terms added and obsoleted; the hierarchy is reorganised	Multiple-testing burden and term-level p-values shift12
Annotation evidence	New annotations concentrated on already well-studied genes	Enrichment moves toward the better-annotated fraction12

Each layer versions independently and on its own schedule. A pipeline that pins one of them and lets the others float is common and is not reproducible in the sense that matters here.

The quantification layer versions too

Before any symbol is looked up, a count has to be produced against a gene model — and gene models are revised at every release. Comparing RefSeq and Ensembl annotations on the same data, one study found identical gene quantification results for only 16.3% of genes; about 28.1% differed by 5% or more, and for 9.3% of genes the relative expression differed by 50% or more¹⁴.

That comparison is between two annotation sources rather than two versions of one, so it is an upper bound on what a version bump does. But it establishes the mechanism: the number attached to a gene depends on which exons the annotation says belong to it, and that definition is revised as annotations grow. A longitudinal comparison built from counts generated years apart is comparing measurements made against different rulers.

MSigDB makes the coupling explicit in its own release notes, which advise users to remap symbols whenever their dataset was generated with a transcriptome other than the specific Ensembl and GENCODE releases that version of the database was built against¹⁵. The gene sets are versioned against an annotation release, and the release notes say so. Most methods sections that cite MSigDB do not record which one.

Why the obvious fix is only most of a fix

The correct answer is to stop joining on symbols and use stable identifiers — HGNC IDs, Ensembl gene IDs, NCBI Gene IDs — which are designed not to be recycled. A pathologist quoted on the nomenclature problem put it simply: having an invariant like the ID helps avoid problems². Do this wherever the data allow it.

Two honest limits. First, much data arrives with only symbols: legacy microarray platforms, published supplementary tables, targeted panel annotations, curated signature databases. Retrofitting IDs to those requires the same lossy mapping you were trying to avoid. Second, identifiers are stable but not inert — Ensembl IDs are versioned, retired and occasionally deprecated, and the mapping between identifier namespaces is itself maintained.

Automated symbol correction is the standard mitigation and it works well: in the GEO analysis, correction raised validity from 79% to 92%¹. The residual 8% is instructive. It comprised lncRNAs, withdrawn symbols, uncharacterised LOC-prefixed genes, non-human symbols, missing data, and commercial probe identifiers¹. And the deeper limitation is one the authors state directly: the tool cannot always determine which gene a symbol refers to, which is why they recommend supplying as much context as possible and remaining cautious about the output¹.

There is a subtlety here worth dwelling on. Correction converts an invisible failure into a visible one for the renamed and mangled cases — good. For the ambiguous cases it can do the opposite: FHL1 passes the check as valid, and the analysis proceeds with more confidence than before. Running a corrector does not remove the ambiguity; it removes the part of the ambiguity that was showing.

What to do about it

1. Join on stable identifiers wherever the data permit. Carry the symbol as a display label, not as the key. If you must map symbols to IDs, do it once, save the mapping table, and version it alongside the analysis.
2. Record the version of every annotation layer — genome build, gene annotation release, symbol database date, pathway or gene set database version, and the enrichment tool’s internal annotation date, which is frequently much older than the tool version. Five numbers, none of them optional.
3. Report the join loss. How many genes in the input failed to map, and what fraction of the list is that? A single line in the methods turns an invisible failure into a reported one.
4. Inspect the unmapped list, don’t just count it. Because renaming concentrates in poorly characterised genes¹⁰, the dropped set is enriched for exactly the genes a novel finding would involve.
5. Flag ambiguous symbols explicitly rather than resolving them silently. Where a symbol maps to more than one gene, or is both an approved symbol and an alias, that row needs chromosome or coordinate context before it can be used¹.
6. For any comparison across time, re-run both sides under one annotation version. Reprocessing the older data is the only way to make the comparison about biology rather than about curation. Where reprocessing is impossible, say so as a limitation.
7. Check the annotation date of your enrichment tool before trusting the result, and prefer tools that state it. The gap between a tool’s release date and its annotation date has historically been years¹⁰.
8. Pin gene set databases the way you pin software. A container that fixes package versions but pulls the current GO release at runtime is not reproducible; the scientific parameter is outside the lockfile.
9. Re-run old analyses before citing their conclusions as current, especially enrichment results more than a few years old¹². Re-analysis is cheap relative to what it protects.

The shape of the error

This series keeps returning to results that are correct measurements attached to inferences that no longer hold. The variant reclassification piece made the closest version of this argument: a pathogenicity call is a statement about the evidence available on the date it was made, and the sequence outlives the interpretation. Symbol and ontology drift is the same phenomenon operating on the vocabulary rather than on the conclusions.

What makes it distinctive is where the failure sits. A pipeline can be entirely correct, a variant caller can be perfectly calibrated, a differential expression model can be exactly right — and the comparison can still be between two different gene sets, because a string in one file matched a string in another file and nothing in either file recorded what year the string was written.

The measurement is real. The gene is real. The join is a claim that two labels refer to the same thing, made by software that has no way to check — and unlike most of the failures in this series, this one is fixable by writing down five version numbers.

References

1. Oh S, Abdelnabi J, Al-Dulaimi R, et al. HGNChelper: identification and correction of invalid gene symbols for human and mouse. F1000Research 2020;9:1493 (version 2, 2022). pmc.ncbi.nlm.nih.gov/articles/PMC7856679
2. Seeking stability in gene nomenclature. CAP TODAY. Trade publication; cited for practitioner accounts of the MLL/KMT2A transition. captodayonline.com — seeking stability in gene nomenclature
3. Zeeberg BR, Riss J, Kane DW, et al. Mistaken identifiers: gene name errors can be introduced inadvertently when using Excel in bioinformatics. BMC Bioinformatics 2004;5:80. pubmed.ncbi.nlm.nih.gov/15214961
4. Ziemann M, Eren Y, El-Osta A. Gene name errors are widespread in the scientific literature. Genome Biology 2016;17:177. pubmed.ncbi.nlm.nih.gov/27552985
5. Abeysooriya M, Soria M, Kasu MS, Ziemann M. Gene name errors: lessons not learned. PLOS Computational Biology 2021;17(7):e1008984. journals.plos.org — pcbi.1008984
6. Bruford EA, Braschi B, Denny P, et al. Guidelines for human gene nomenclature. Nature Genetics 2020;52(8):754–758. pubmed.ncbi.nlm.nih.gov/32747822
7. HUGO Gene Nomenclature Committee, frequently asked questions (definitions of “symbol withdrawn” and “entry withdrawn”). genenames.org/help/faq
8. Gene Normalizer: a tool to resolve genetic ambiguity through data harmonization. 2023. sciencedirect.com — S2210776223000935
9. Schijvenaars BJA, Mons B, Weeber M, et al. Thesaurus-based disambiguation of gene symbols. BMC Bioinformatics 2005;6:149. pmc.ncbi.nlm.nih.gov/articles/PMC1183190
10. Wadi L, Meyer M, Weiser J, Stein LD, Reimand J. Impact of outdated gene annotations on pathway enrichment analysis. Nature Methods 2016;13(9):705–706. nature.com/articles/nmeth.3963
11. Preprint version of ref. 10, from which the doubling of the pathway vocabulary over 2009–2016 is drawn. bioRxiv 049288. biorxiv.org — 049288
12. Haynes WA, Tomczak A, Khatri P. Interpretation of biological experiments changes with evolution of the Gene Ontology and its annotations. Scientific Reports 2018;8:1–8. nature.com/articles/s41598-018-23395-2
13. Monitoring changes in the Gene Ontology and their impact on genomic data analysis (GOTrack). bioRxiv 320861 (preprint; not peer reviewed). Cited for its mixed-findings summary and its own large-scale result. biorxiv.org — 320861
14. Zhao S, Zhang B. A comprehensive evaluation of Ensembl, RefSeq, and UCSC annotations in the context of RNA-seq read mapping and gene quantification. BMC Genomics 2015;16:97. pmc.ncbi.nlm.nih.gov/articles/PMC4339237
15. MSigDB release notes (symbol remapping advisories tying each release to a specific Ensembl/GENCODE version). docs.gsea-msigdb.org — MSigDB 7.5 release notes

Zetobit, LLC · Bioinformatics consulting · Lexington, KY · zetobit.com