Variant Normalization: Why the Same Variant Written Two Ways Silently Fails to Match

Zetobit Bioinformatics Insight Series

Variant Representation & Data Integrity

Variant Normalization: Why the Same Variant Written Two Ways Silently Fails to Match

A single insertion or deletion can be written in several equally correct VCF representations, at different positions, with different alleles. To a human they are obviously the same edit. To a computer doing an exact match against a clinical database, they are different variants — and the mismatch is completely silent.

By Kanna Nandakumar, PhD · Zetobit

Most failures in a genomics pipeline announce themselves: a job crashes, a file is empty, a metric looks wrong. Variant representation failures do the opposite. Everything runs, every file is well-formed, and the answer is quietly incomplete because two records that describe the same underlying change were never recognized as equal. A pathogenic variant sits in your VCF and in ClinVar in two different but valid spellings, the annotation step compares them character by character, finds no match, and reports nothing. No error is raised because, mechanically, nothing went wrong.

This is one of the most consequential and least visible problems in variant analysis, and it has a clean solution that is easy to skip. Understanding why the problem exists — and why the fix is not optional — starts with an uncomfortable fact about the VCF format: it does not enforce a unique way to write a variant.

The same edit, many spellings

Consider a deletion inside a short tandem repeat — say a run of CA dinucleotides. Deleting one CA unit produces the same resulting sequence no matter which copy of the unit you consider deleted, so the deletion can be legitimately anchored at several positions. Each anchoring yields a different POS, REF, and ALT, and every one of them is a valid VCF record describing the identical biological event. The repeat is not an exotic corner case; a single tandem-repeat variant such as a CA repeat has multiple possible VCF representations differentiated only by where the change is anchored.¹

The consequences are not academic. Because the number of ways to represent the same genomic variant is non-unique, it is mandatory to normalize before you can determine whether two representations are the same or different variants — and a failure to recognize this frequently results in inaccurate analyses.² Every operation that compares variants by identity inherits the problem: annotation against a database, merging call sets from two callers, deduplicating, looking up an allele frequency, matching a patient variant to a known pathogenic one. If the two sides spell the variant differently, the comparison fails and nothing warns you.

Normalization: two rules that produce one canonical form

Normalization is the procedure that collapses all valid spellings of a variant to a single canonical one, so that identity comparison becomes meaningful. It rests on two independent requirements that together pin down a unique representation.¹

The first is parsimony: a variant should be represented with as few bases as possible in REF and ALT, trimming any shared bases that are not needed to describe the change. A deletion written as REF=GCACA, ALT=GCA carries redundant flanking sequence; trimmed to its minimal form it says the same thing with fewer bases. Parsimony governs the length of the representation.

The second is left-alignment: among all the equivalent positions at which an indel could be anchored, choose the leftmost. A variant is left-aligned if and only if it is no longer possible to shift its position to the left while keeping the length of all its alleles constant.¹ Left-alignment governs the position. A variant that is both parsimonious and left-aligned is, by definition, normalized — and the deep result that makes this useful is a uniqueness guarantee: two variants have different normalized representations if and only if they are actually different variants.³ Normalize both sides and exact matching becomes correct. That biconditional is the entire reason the procedure works.

# the same one-unit deletion in a CA repeat — all valid, all different
chr1 112 CAC C ← as emitted by caller A
chr1 108 ACA A ← as stored in a database
chr1 110 ACAC AC ← non-parsimonious variant

# after normalization — one canonical record
chr1 107 GCA G ← left-aligned + parsimonious

Many anchorings collapse to one. In a repeat, the same one-unit deletion can be written at several positions, each a legal VCF record. Left-alignment picks the leftmost position and parsimony trims to the minimal alleles, so every valid spelling maps to a single canonical record — the precondition that makes exact matching correct.

Multiallelic sites: a second representation trap

Left-alignment and parsimony handle a single variant, but a related failure hides at sites where more than one alternate allele is called. VCF permits a multiallelic record — REF=A, ALT=C,G on one line — and many downstream tools silently mishandle these, comparing or annotating only the first allele or failing to match a biallelic record elsewhere. The standard fix is decomposition: split the multiallelic record into separate biallelic records before normalizing. Large aggregation projects do exactly this as a matter of course — Genomics England's aggV2, for instance, decomposes all multiallelic variants into biallelic form and left-aligns indels using vt, because many downstream tools rely on biallelic representation and it enables clean allelic comparison between call sets.⁴ Decomposition and normalization are a pair; doing one without the other leaves a gap.

There is a caveat worth stating plainly, because it comes from the decomposition tools themselves: information is generally lost when a multiallelic variant is decomposed vertically, so per-allele fields must be interpreted with care afterward.⁴ Normalization is not free of tradeoffs; it is a deliberate transformation whose downstream effects you should understand, not a cosmetic reformat.

Not all normalizers agree

It would be convenient if choosing any normalization tool guaranteed the same canonical output. It does not. The widely used implementations — vt normalize, bcftools norm, and GATK's LeftAlignAndTrimVariants — do not all produce identical results, and the differences fall exactly where they hurt: indels in repetitive regions. In the analysis accompanying the vt normalize method, GATK's LeftAlignAndTrimVariants failed to normalize 3.4% of the indels that vt normalize handled correctly.⁵ The gap persists in practice: on GATK's own left-alignment test file, an independent review found LeftAlignAndTrimVariants incorrectly normalized 3 of the 16 variants, including specific indels on chromosomes 20 and 21.³ If two call sets are normalized with different tools, a residue of true-positive matches can still be lost to representation differences neither tool resolved.

Normalization is a precondition, not a guarantee. Normalizing both sides of a comparison with the same tool makes exact matching correct for the cases that tool resolves. Normalizing with different tools can still leave real matches undetected — so for high-stakes comparison, normalize both call sets through one pipeline, and be aware that some complex variants in repeats have no unique representation at all.

What to check in any variant workflow

Because the failure is silent, the safeguards are procedural — things to confirm rather than errors to catch:

• Is every call set normalized before comparison? Both your variants and the reference database must be left-aligned and parsimonious against the same reference genome. An un-normalized side defeats the whole exercise.²
• Are multiallelic sites decomposed? Split to biallelic records before matching, and treat per-allele annotations with care afterward because decomposition loses information.⁴
• Same reference build and same tool on both sides? Normalization is reference-dependent; left-alignment shifts against the reference sequence, so a mismatched build silently changes the canonical form. Using one tool end-to-end avoids inter-tool disagreement.⁵
• Does your annotator normalize internally, or assume you did? Some tools normalize on ingest; others trust the input. Know which, because a silent no-match looks identical to a true absence.³
• Are complex variants in repeats flagged? A minority of variants have no unique representation even after normalization; comparison of these needs allele-aware methods rather than position matching.⁵

The through-line with the rest of this series is representation. A genome build is a specific sequence, not a neutral ruler; a VCF record is a specific spelling, not a neutral identity. In both cases the danger is assuming a canonical form exists where none is enforced. Normalization is the small, unglamorous step that manufactures the canonical form on purpose — and skipping it does not produce an error, only an answer that is quietly missing the variant you were looking for.

References

1. Variant Normalization. University of Michigan Genome Analysis Wiki (Tan, Abecasis, Kang). genome.sph.umich.edu/wiki/Variant_Normalization
2. Variant Normalization. OpenCB / OpenCGA documentation. github.com/opencb/opencga/wiki/Variant-Normalization
3. Littlefield M, et al. Streamlining variant normalization on large genomic datasets (Glow); on the uniqueness theorem of Tan et al. and LeftAlignAndTrimVariants test-file errors. Databricks / Glow documentation, 2019. glow.readthedocs.io/en/latest/blogs/variant-normalization-blog
4. AggV2 variant normalisation and representation. Genomics England Research Environment User Guide. re-docs.genomicsengland.co.uk/variant_normalisation
5. Bayat A, Gaëta B, Ignjatovic A, Parameswaran S. Improved VCF normalization for accurate VCF comparison. Bioinformatics. 2017;33(7):964–970. Discussing Tan et al. (2015), Bioinformatics 31:2202–2204 (vt normalize). academic.oup.com/bioinformatics/article/33/7/964

© Zetobit LLC · Bioinformatics Insight Series zetobit.com