The Confound You Designed In
The Confound You Designed In
Batch effects, the limits of correction, and the ten-minute conversation that has to happen before the samples are queued.
Every sequencing experiment has batches. Samples run on different days, on different flow cells, by different technicians, with different reagent lots. This is unavoidable — a study of any size cannot be processed in a single run.
What is avoidable is confounding those batches with the thing you're trying to measure. And that mistake is made at the bench, weeks before anyone opens R.
What a batch effect actually is
A batch effect is systematic technical variation that tracks with processing group rather than biology [1]. It is not noise. Noise is random and averages out; batch effects are structured, reproducible, and average in.
The sources are mundane and numerous: library prep kit lot, PCR cycle count, flow cell, lane, extraction date, RIN degradation during storage, ambient temperature, the specific person pipetting. Each contributes a small systematic shift. Collectively they can dominate the variance in your expression matrix — sometimes exceeding the biological signal you designed the study to find [2].
The classic demonstration remains sobering: cluster an RNA-seq dataset without correction and you frequently recover the processing date, not the phenotype [3].
The distinction that matters
There are two situations, and they are not equally recoverable.
Batch is a covariate. Your cases and controls are distributed across every batch. Batch contributes variance, but it is separable from the effect of interest because the design matrix has the rank to separate them. This is a modeling problem. It has a solution.
Batch is a confounder. All your cases were run in March and all your controls in June. The batch effect and the biological effect are the same column in your design matrix. No statistical method can separate them, because there is no information in the data that distinguishes them.
This second case is the one that reaches me as a consulting question, usually phrased as "which correction method should we use?" The honest answer is that the question has no good answer. ComBat will produce output [1]. limma's removeBatchEffect will produce output [4]. Every method will produce output. None of them will produce truth, because the information required to separate the signals was never collected.
A correction method cannot recover information the design destroyed. It can only redistribute what's there — and when batch and biology are collinear, redistributing means guessing.
Why correction can make things worse
Assume the recoverable case: batch is a covariate, not a confounder. Correction is still not free.
The two-step trap
The common workflow is to run ComBat, then feed the "corrected" matrix into a differential expression model. This understates variance and inflates false positives, because the downstream model treats corrected values as raw observations and never accounts for the uncertainty introduced by the correction itself [5]. The correct approach is to include batch as a term in the model — ~ batch + condition — so a single fit estimates both, and the standard errors reflect that estimation.
Use correction methods for visualization (PCA, clustering, heatmaps). Use model terms for inference. Conflating these is one of the most common errors I see in otherwise careful analyses.
Over-correction
If batch is even partially correlated with condition — not fully confounded, but unbalanced — aggressive correction removes real biological signal along with technical variation [5]. The result looks cleaner and is less true. Cleanliness is not a validation metric.
Small batches
Estimating a batch effect from three samples produces an estimate with wide uncertainty. Applying it as if it were known is overconfidence dressed as rigor.
The unsupervised case
For discovery work — where you don't have a known covariate to protect — surrogate variable analysis (SVA) [6] and RUV [7] estimate hidden structure from the data itself, using control genes or residual variation. These are useful tools and considerably better than ignoring the problem.
They also carry the sharpest version of the over-correction risk. A surrogate variable is estimated, not measured; if your biological effect is subtle and your technical structure is strong, the method may absorb the former into the latter. When the biological grouping is unknown a priori, there is no guardrail telling you it happened.
What actually works
- Randomize sample-to-batch assignment. Not convenience order. Not case-batch-first. Randomize, and do it before extraction [2].
- Block deliberately. If you know batches will be needed, balance every condition across every batch. A balanced design converts an unsolvable problem into a solved one.
- Record everything. Extraction date, kit lot, flow cell, operator, RIN, storage duration. You cannot model a variable you didn't write down. This is the cheapest insurance in the entire workflow, and it is routinely skipped.
- Include controls that span batches. A common reference sample in every batch gives you a direct measurement of the batch effect rather than an inference about it.
- Look before you model. PCA colored by every recorded technical variable, before any hypothesis testing. If PC1 is extraction date, you need to know that at the start, not in review.
The consulting reality
Most of the time, I am handed data that already exists. Randomization is no longer available; the samples were run months ago in the order they arrived.
In that situation the useful work is diagnostic, not corrective: determine whether batch and condition are separable at all. If they are, model batch as a term and report the design honestly. If they are not, say so — clearly, early, and in writing.
That last part is the hard one. A confounded design means the experiment cannot answer the question it was built to answer, and someone has to say it out loud. Producing a corrected matrix and a volcano plot is easier, faster, and more satisfying to everyone in the room. It is also, in that situation, a way of laundering an unanswerable question into a publishable figure.
The takeaway
Batch effects are not primarily a statistics problem. They are a design problem that becomes a statistics problem only when the design was sound enough to leave a path.
Correction methods are legitimate and useful — for the case where batch is a nuisance covariate, applied as a model term rather than a preprocessing step. They are not a repair mechanism for a confounded experiment, and treating them as one converts a known limitation into a hidden one.
The most valuable thing a bioinformatician can do for a study is often to be in the room before the samples are queued, asking which batch each one is going in. That conversation takes ten minutes. Nothing downstream can substitute for it.
References
- Johnson WE, Li C, Rabinovic A. Adjusting batch effects in microarray expression data using empirical Bayes methods. Biostatistics. 2007;8(1):118–127. — Biostatistics — ComBat empirical Bayes batch adjustment
- Leek JT, Scharpf RB, Bravo HC, et al. Tackling the widespread and critical impact of batch effects in high-throughput data. Nature Reviews Genetics. 2010;11(10):733–739. — Nature Reviews Genetics — the impact of batch effects
- Gilad Y, Mizrahi-Man O. A reanalysis of mouse ENCODE comparative gene expression data. F1000Research. 2015;4:121. — F1000Research — batch effects in the mouse ENCODE reanalysis
- Ritchie ME, Phipson B, Wu D, et al. limma powers differential expression analyses for RNA-sequencing and microarray studies. Nucleic Acids Research. 2015;43(7):e47. — Nucleic Acids Research — the limma framework
- Nygaard V, Rødland EA, Hovig E. Methods that remove batch effects while retaining group differences may lead to exaggerated confidence in downstream analyses. Biostatistics. 2016;17(1):29–39. — Biostatistics — why two-step correction inflates confidence
- Leek JT, Storey JD. Capturing heterogeneity in gene expression studies by surrogate variable analysis. PLoS Genetics. 2007;3(9):e161. — PLoS Genetics — surrogate variable analysis (SVA)
- Risso D, Ngai J, Speed TP, Dudoit S. Normalization of RNA-seq data using factor analysis of control genes or samples. Nature Biotechnology. 2014;32(9):896–902. — Nature Biotechnology — RUVSeq normalization
- Love MI, Huber W, Anders S. Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2. Genome Biology. 2014;15(12):550. — Genome Biology — DESeq2 and design formulas