False Positives in Metagenomic Classification: Why a Species in the Report Is Not Evidence It Was in the Sample
Bioinformatics Insight Series
False Positives in Metagenomic Classification: Why a Species in the Report Is Not Evidence It Was in the Sample
A metagenomic classifier assigns every read the best available label from its reference database. It has no category for organisms it has never seen, and no mechanism for reporting that nothing matched well. What comes back is a ranked list of the closest things it knows about.
In 2020, a group set out to test how much a metagenomic pipeline's output depends on the contents of its reference database rather than the contents of the sample. Their method was elegantly blunt.
They took 38 human stool metagenomes — 27 from Hadza hunter-gatherers in Tanzania, 11 from subjects in Italy — that had previously been analyzed with a fungal classification tool. Then they swapped out the fungal reference database and replaced it with genomes of 18 species of amphibians and reptiles: frogs, salamanders, snakes, lizards, geckos, turtles, and crocodiles. Every other parameter stayed at its default.
The pipeline identified amphibians and reptiles in all of the samples analysed, reporting turtles, bull frogs, and snakes as the most abundant taxa in the human gut.
Nothing malfunctioned. The tool did precisely what it was built to do: take reads, compare them to a database, and assign the best available label. The failure was not in the software. It was in the assumption that the software's output constitutes evidence of presence.
The closed-world problem
A classifier's reference database defines the universe of possible answers. Reads that derive from organisms outside that universe do not go unclassified — they get sorted into whichever database entry they resemble most, however weakly.
This is why the reptile experiment worked, and it is also visible in the original analysis it critiqued. The fungal tool reported detecting 65 of the 66 species in its own reference database across the sample set. A pipeline that finds essentially everything it was given to look for is not describing a sample; it is describing a database.
Two mechanisms drive the spurious matches, and both are properties of reference genomes rather than of the classifier. The first is conservation: genes involved in fundamental biological processes change little over evolutionary time and across taxa, so a read from a bacterium can align respectably to a homologous region in a crocodile. The second is that assembled reference genomes are themselves contaminated — sequence from other organisms is present in public assemblies, mislabeled as belonging to the host taxon.
That second problem is old and acknowledged. Kraken's original publication noted that contaminant sequences in reference data come from other bacteria, viruses, or even human genomes, producing incorrectly labelled k-mers in the database, and that although the authors attempted to remove them, some contaminants may still slip through any filters. Every classifier built on public genome collections inherits this.
Figure 1. Read count is the metric most classifiers report and the one least able to separate signal from artifact. Reads from an organism genuinely present distribute across its genome; false-positive reads typically concentrate in small regions — a conserved gene, a low-complexity stretch, or a contaminated segment of the assembly. Both scenarios produce identical entries in an abundance table.
The organisms that are really there, but not from the patient
Database artifacts are one source of false positives. The other is genuine DNA from organisms that were genuinely in the tube — just not in the person.
The definitive demonstration remains Salter and colleagues' 2014 work. They showed that contaminating DNA is ubiquitous in commonly used DNA extraction kits and other laboratory reagents, that it varies greatly in composition between different kits and kit batches, and that it critically affects results from samples with low microbial biomass. Their dilution experiment makes the mechanism unmistakable: starting from a pure Salmonella bongori culture and diluting ten-fold repeatedly, the proportion of reads from the actual organism fell while contaminating bacterial groups rose in relative abundance, until by the fifth dilution the contaminants dominated.
The critical insight is that contamination is roughly constant in absolute terms and therefore variable in relative terms. A high-biomass stool sample swamps the kit background. A low-biomass sample — CSF, blood, lung, joint fluid, tissue biopsy — does not. The same reagent, the same protocol, the same laboratory produces a clean-looking profile in one specimen type and a profile dominated by reagent organisms in another.
This has a consequence that is easy to state and frequently ignored: contamination is not a fixed error rate to be subtracted. It is a background whose relative weight is determined by how much real material is in the sample, which means the same threshold cannot serve both specimen types.
Salter's group also found that contamination can become the dominant driver of how samples cluster in ordination analyses. Once the contaminating taxa were identified and removed, the clustering pattern they had observed disappeared. A study that had not sequenced negative controls would have interpreted kit batch as biology.
Coverage, not counts
The most useful correction is conceptually simple: stop treating read count as evidence and start asking where the reads landed.
The logic behind this is the observation that motivated tools like KrakenUniq: false-positive reads typically match only small portions of a genome, so if a species' genome contains a low-complexity region and the only reads matching that species fall within it, the species was probably not present. Reads from microbes that are truly present should distribute relatively uniformly across the genome.
The effect of applying this is substantial. One recent evaluation found that in simulations, filtering identified species to only those with at least 0.5% reference genome coverage removed 99.7% of false-positive taxa. The same authors are careful about the caveat: applying this to real datasets requires a more nuanced approach accounting for sequencing depth, whether samples are high- or low-biomass, and how complete the database is with respect to the environment sampled. A 0.5% threshold is not a universal constant — it is an illustration that breadth carries information that depth does not.
There is also a database-design lesson embedded in the reptile experiment. The authors' conclusion was that databases containing all domains of life provide the most suitable option for metagenomic species profiling, particularly when targeting microbial eukaryotes. A taxon-restricted database forces reads to choose among the options it contains; a comprehensive one lets a bacterial read be assigned to a bacterium instead of to the most bacterium-like reptile available.
| Origin | Mechanism | What separates it out |
|---|---|---|
| Database artifact | Contaminated or mislabeled sequence in the reference assembly | Coverage breadth; curated or decontaminated databases |
| Conserved-region match | Reads align to homologous genes shared across distant taxa | Coverage breadth; comprehensive all-domain database |
| Reagent / kit background | Real DNA introduced by extraction kits, reagents, environment | Negative controls processed in the same batch and lot |
| Genuine organism | Nucleic acid from the specimen itself | Breadth plus abundance above batch-matched background |
How clinical assays handle this
Clinical metagenomics has had to formalize what research metagenomics often leaves to judgment, and the validated assays are instructive because they treat detection as a quantitative comparison rather than a binary presence call.
The UCSF clinical CSF assay is a well-documented example. Rather than reporting organisms by read count, it computes a reads-per-million ratio — the RPM in the sample divided by the RPM in the no-template control — so the background is built into the metric itself. Threshold selection was empirical: ROC analysis across 95 clinical CSF samples showed that an RPM ratio of 10 maximized accuracy, and that value became the reporting cutoff for bacteria, fungi, and parasites.
Two further design choices are worth borrowing. Spiked internal controls act as sensitivity indicators — if the internal control's RPM falls below 100, indicating high host background, or the preprocessed read count falls below five million, a negative result is reported with an explicit caveat rather than as a clean negative. And viral detections are assessed by mapping reads to the closest matched genome to identify non-overlapping regions, which is the coverage-breadth principle applied at the level of an individual call.
The scale at which this operates is worth noting for calibration. Across 4,828 CSF mNGS tests run between 2016 and 2023, 14.4% were positive for any microorganism. A pan-pathogen assay on a difficult specimen type returns a negative result most of the time — which is what a correctly calibrated metagenomic test looks like, and quite different from a research profile listing dozens of taxa per sample.
What to build in
Negative controls in every batch, sequenced and analyzed
Because contamination varies by kit and by lot, a control from a previous batch does not characterize the current one. The control has to travel with the samples through the same extraction, the same reagents, and the same run — and it has to be sequenced, not merely collected.
Report coverage breadth alongside read count
Percentage of reference genome covered, and whether coverage is distributed or concentrated, should sit next to abundance in any output a human will interpret. A taxon supported by reads stacked on one locus and a taxon supported by reads spread across a genome are different claims and should not look identical in a table.
Match the database to the question, and prefer breadth
A taxon-restricted database guarantees that every read finds a home somewhere in that taxon. Where the goal is to determine what is present rather than to quantify something already known to be present, a database spanning all domains — including host — gives reads somewhere correct to go.
Set thresholds per specimen type
Because the contamination-to-signal ratio depends on input biomass, a threshold validated on stool has no standing when applied to CSF or synovial fluid. Thresholds are properties of an assay–specimen pairing, not of a classifier.
Treat the classifier's output as a hypothesis list
The output of a metagenomic classifier is the set of organisms worth investigating, not the set of organisms present. Confirmation — targeted PCR, culture where feasible, coverage inspection, orthogonal assay — is what converts a candidate into a finding.
The short version
A metagenomic classifier answers "which database entry does this read most resemble," and that question has an answer for every read, including reads from organisms not in the database and reads from organisms not in the patient. Read counts alone cannot distinguish a genuine detection from a conserved-region match, a contaminated reference assembly, or the extraction kit.
An open-ended question by construction
Most assays in a clinical laboratory ask a closed question: is this specific target present, above this specific threshold. Metagenomics asks an open one — what is here — and open questions do not have the same relationship to negative evidence. A targeted PCR that fails to amplify tells you something definite. A metagenomic classifier that reports fifty taxa has not told you that fifty organisms were present; it has told you that fifty database entries attracted reads.
That distinction is the whole discipline of the field. The technology is genuinely capable of finding pathogens that nothing else would have found. It just cannot, on its own, tell you which of its findings are real — and the parts of the workflow that answer that question are the controls, the coverage metrics, and the thresholds, not the classifier.
Zetobit builds and validates CAP/CLIA-compliant NGS pipelines, including metagenomic workflows with batch-matched controls, coverage-based confirmation, and specimen-specific thresholds. If you are validating an mNGS assay or auditing a microbial profiling pipeline, we're happy to talk.
References
- Marcelino VR, Holmes EC, Sorrell TC. The use of taxon-specific reference databases compromises metagenomic classification. BMC Genomics 21:184 (2020). doi:10.1186/s12864-020-6592-2.
- Salter SJ, Cox MJ, Turek EM, et al. Reagent and laboratory contamination can critically impact sequence-based microbiome analyses. BMC Biology 12:87 (2014). doi:10.1186/s12915-014-0087-z.
- Wood DE, Salzberg SL. Kraken: ultrafast metagenomic sequence classification using exact alignments. Genome Biology 15:R46 (2014). doi:10.1186/gb-2014-15-3-r46.
- Breitwieser FP, Baker DN, Salzberg SL. KrakenHLL: confident and fast metagenomics classification using unique k-mer counts. bioRxiv preprint (2018). doi:10.1101/262956. Later published as KrakenUniq, Genome Biology 19:198 (2018).
- From classification to confirmation: verifying taxonomic classifications by mapping metagenomic reads to reference genomes. Microbial Genomics (2026). doi:10.1099/mgen.0.001739.
- Miller S, Naccache SN, Samayoa E, et al. Laboratory validation of a clinical metagenomic sequencing assay for pathogen detection in cerebrospinal fluid. Genome Research 29(5):831–842 (2019). doi:10.1101/gr.238170.118.
- Wilson MR, Sample HA, Zorn KC, et al. Clinical metagenomic sequencing for diagnosis of meningitis and encephalitis. New England Journal of Medicine 380:2327–2340 (2019). doi:10.1056/NEJMoa1803396.
- Metagenomic next-generation sequencing of cerebrospinal fluid for diagnosis of central nervous system infections: 7-year performance of a clinically validated test. medRxiv preprint (2024). doi:10.1101/2024.03.14.24304139. Preprint — check for the peer-reviewed version before citing.