Spatial Deconvolution: Why a Cell Type Map Shows You the Reference, Not the Tissue
Bioinformatics Insight Series
Spatial Deconvolution: Why a Cell Type Map Shows You the Reference, Not the Tissue
A Visium spot holds several cells, so every cell type map is a deconvolution against a single-cell reference. The proportions are constrained to sum to one across that reference's categories — which means a cell type the reference doesn't contain is never reported as missing. Its signal is redistributed into whatever resembles it.
Spatial transcriptomics produces the most immediately persuasive figure in modern genomics: a tissue section overlaid with colored pie charts or a smooth map of cell type abundance, cell populations arranged in recognizable anatomy. It looks like a photograph of where cells are.
It is a model output. On the standard Visium platform, each capture spot is 55 µm across, with 100 µm between spot centers, while a typical immune cell is around 10 µm in diameter — so a spot captures expression from multiple cells, typically 1 to 10. Nothing in the raw data assigns a cell type to anything. The map is produced afterward, by fitting each spot's expression as a mixture of cell type profiles drawn from a separate single-cell reference.
That fitting step imports every limitation of the reference into what looks like a direct observation of the tissue.
The constraint that hides the problem
Deconvolution estimates, for each spot, the proportion of each reference cell type. Those proportions sum to one. That is a sensible modeling choice and it has a consequence worth stating plainly: the model has no way to express "some of this signal came from something I don't have a profile for."
This is the same structural failure covered in this series for single-cell annotation — a classifier that must assign every cell to one of its known labels will confidently misassign a novel population rather than flag it. Deconvolution has the identical shape, made worse by mixing. In annotation, a novel cell gets one wrong label. In deconvolution, an unmodeled cell's transcripts are apportioned across several reference types according to which ones its expression happens to resemble, in a spot that also contains real cells of those types. The error does not appear as a distinct wrong entity. It appears as a plausible adjustment to the proportions of things genuinely present.
The effect is measurable. A benchmark of deconvolution pipelines found that failure to include cell types in the reference that are present in a mixture leads to substantially worse results, regardless of the previous choices — meaning no amount of care in normalization, marker selection, or method choice compensates for a reference that is missing a population.
What makes the missing type consequential rather than merely absent is where it goes. Work on missing cell types in deconvolution references found that when adipocytes and mesothelial cells were present at realistic proportions in mixtures but absent from the reference, deconvolution accuracy decreased across all methods. Their case is instructive for tumor work: high-grade serous ovarian carcinoma metastasizes to the omentum, a tissue rich in adipose, but adipocytes are notoriously difficult to recover in a single-cell suspension — so the reference systematically lacks a population the tissue systematically contains.
That last point generalizes beyond adipocytes. Any cell type that is fragile, large, adherent, or otherwise hard to isolate is underrepresented in scRNA-seq references while remaining fully present in an intact tissue section. Spatial data captures the tissue as it is; the reference captures what came through the dissociation protocol. The deconvolution reconciles the two by pushing the difference into the categories it has.
The encouraging part: it leaves a trace
The same work that characterized the problem also showed it is not undetectable. The residuals — the part of each mixture's expression that the fitted reference profiles fail to explain — carry information about what is missing. In their analysis, residual structure suggested adipocyte signatures, and incorporating adipocytes into the reference was enough to remove the difference they had observed between sample types.
This is the practical lever, and it is underused. Most spatial pipelines report proportions and stop. Almost none report goodness of fit per spot, or examine what the model could not account for. A spot whose expression is poorly explained by every reference profile is telling you something specific: either a cell type is missing, or the reference's version of that cell type does not match the tissue's.
Figure 1. Because proportions are constrained to sum to one over the reference's categories, a cell type absent from the reference cannot be reported as unexplained. Its share is absorbed into whichever profiles it partially resembles — inflating cell types that are genuinely present and producing an output indistinguishable in form from a correct one. Residual analysis is what surfaces the difference.
Choosing the method is the easier problem
The field has benchmarked deconvolution methods thoroughly. Li and colleagues evaluated 16 integration methods across 45 paired datasets and 32 simulated datasets, finding Cell2location, SpatialDWLS and RCTD the top performers for spot deconvolution. A separate benchmark of 18 methods across 50 real and simulated datasets recommended CARD, Cell2location and Tangram. The Spotless pipeline benchmarked 11 methods across 63 synthetic, 3 binned and 2 real datasets, with RCTD and Cell2location most recommended.
Two things stand out from reading those together. First, they broadly agree that a handful of methods lead — Cell2location and RCTD appear across all three. Second, the recommendations still differ, and a review comparing them attributes the inconsistency to a combination of different reference datasets, testing datasets, gold standards and evaluation metrics. Reference choice is not a nuisance parameter in these comparisons; it is one of the reasons they disagree.
The same review notes the more directly actionable finding: more relevant references result in more accurate deconvolution. Selecting a well-benchmarked method is worth doing and takes an afternoon. Assembling a reference that actually matches the tissue, condition, and platform is the harder task and has more leverage over the result.
Platform effects and the reference-free alternative
Even a complete reference is measured on a different instrument than the spatial data. The leading methods handle this explicitly — RCTD and Cell2location estimate gene-based platform effects within Bayesian models — which is a strong argument for using a method that models the discrepancy rather than one that assumes the two modalities are directly comparable.
Reference-free methods take the more radical route of not requiring an external reference at all. STdeconvolve applies latent Dirichlet allocation, treating genes as words and spots as documents, to infer transcriptionally distinct cell types and their proportions directly from the spatial data. Its authors report comparable performance to reference-based methods when a suitable reference exists, and potentially superior performance when one does not.
The tradeoff is honest and worth stating. Reference-free methods recover transcriptionally distinct programs, not named cell types — and naming them typically means correlating the recovered profiles against a reference anyway, which is why at least one comparative study reclassified STdeconvolve as reference-based for its own analysis. LDA also brings known limitations: sensitivity to sparse and noisy data, difficulty detecting rare topics, and difficulty distinguishing similar topics such as closely related subtypes. Reference-free approaches move the reference dependency from the fitting step to the interpretation step. That is a real improvement — it stops the reference from constraining what can be found — but it does not eliminate the dependency.
| Dependency | How it distorts the map | What helps |
|---|---|---|
| Reference completeness | Missing types redistributed into resembling ones; all proportions shift | Residual/goodness-of-fit inspection; tissue-matched reference |
| Dissociation bias | Fragile or large cells underrepresented in reference, present in section | Single-nucleus reference; check against H&E morphology |
| Annotation granularity | Reference's subtype resolution becomes the map's resolution | Decide granularity deliberately; report it |
| Platform effect | Reference measured on a different technology than the spatial assay | Methods that model platform effects explicitly |
| Method choice | Moderate differences among leading tools | Use a consistently top-ranked method; compare two |
What to do differently
Report goodness of fit per spot, not just proportions
Residuals are where an incomplete reference announces itself. A map of poorly-explained spots deserves to sit beside the cell type map, and it costs nothing to produce. Spatially clustered poor fit is a particularly strong signal — it usually indicates a structure the reference does not contain rather than random noise.
Ask what the reference could not contain before trusting the map
The question is not whether the reference is high quality but whether its dissociation protocol could have recovered the cell types the tissue is known to have. For tumor work: adipocytes, neurons, skeletal muscle, megakaryocytes, and large or fragile stromal populations are the usual suspects. A single-nucleus reference sidesteps much of this.
Treat proportions as compositional
Values constrained to sum to one cannot move independently — one type rising forces others down arithmetically, whether or not anything changed biologically. This is the same constraint that governs microbiome relative abundance, and it applies with equal force here. Differential abundance testing on raw spot proportions across conditions invites exactly that artifact.
Cross-check against the image
The H&E section underneath the spots is independent evidence and is usually ignored. Morphology will not resolve cell types finely, but it will readily contradict a map that places a population where the tissue plainly has none.
Run a reference-free method as a control
Not necessarily as the primary analysis, but as a check on what the reference is constraining. A transcriptionally distinct program recovered without supervision, which has no counterpart in the reference-based output, is worth investigating rather than dismissing.
The short version
Deconvolution answers "what mixture of my reference cell types best explains this spot," and that question has an answer for every spot — including spots containing cells the reference has never seen. Because proportions must sum to one, missing populations are not reported as missing; they are absorbed into the types that remain, inflating them. The residual is where the absence is visible, and almost no pipeline looks at it.
A map is a claim about the reference
The persuasiveness of a spatial figure comes from its resemblance to a photograph. Cell type colors laid over real tissue architecture, in anatomically sensible arrangements, read as observation. And a great deal of it is observation — the expression is measured, the coordinates are real, the tissue structure is genuinely there.
The cell type labels are not. They are the output of a model asked to explain measured expression using a fixed vocabulary assembled from a different experiment on different tissue processed a different way. Where that vocabulary is complete and well matched, the map is close to right. Where it isn't, the map is equally smooth, equally confident, and quietly describing the reference instead of the tissue.
Zetobit builds and validates NGS and spatial transcriptomics pipelines, including deconvolution workflows with residual diagnostics, reference-completeness auditing, and compositional-aware downstream testing. If you are standing up spatial analysis or reconciling cell type maps across references, we're happy to talk.
References
- Ivich A, Davidson NR, Grieshober L, et al. Missing cell types in single-cell references impact deconvolution of bulk data but are detectable. Genome Biology 26 (2025). doi:10.1186/s13059-025-03506-9.
- Avila Cobos F, Alquicira-Hernandez J, Powell JE, et al. Benchmarking of cell type deconvolution pipelines for transcriptomics data. Nature Communications 11:5650 (2020). doi:10.1038/s41467-020-19015-1.
- Li B, Zhang W, Guo C, et al. Benchmarking spatial and single-cell transcriptomics integration methods for transcript distribution prediction and cell type deconvolution. Nature Methods 19:662–670 (2022). doi:10.1038/s41592-022-01480-9.
- Li H, Zhou J, Li Z, et al. A comprehensive benchmarking with practical guidelines for cellular deconvolution of spatial transcriptomics. Nature Communications 14:1548 (2023). doi:10.1038/s41467-023-37168-7.
- Sang-aram C, Browaeys R, Seurinck R, Saeys Y. Spotless, a reproducible pipeline for benchmarking cell type deconvolution in spatial transcriptomics. eLife 12:RP88431 (2024). doi:10.7554/eLife.88431.
- Miller BF, Huang F, Atta L, Sahoo A, Fan J. Reference-free cell type deconvolution of multi-cellular pixel-resolution spatially resolved transcriptomics data. Nature Communications 13:2339 (2022). doi:10.1038/s41467-022-30033-z.
- 10x Genomics. Integrating single cell and Visium spatial gene expression data. Analysis guide, accessed July 2026.
- Deconvolution. In: Orchestrating Spatial Transcriptomics Analysis with Bioconductor (OSTA), accessed July 2026.