Where the Wet Lab Ends and Pipeline Validation Begins

Zetobit · The Validation File

The Validation File

Where the Wet Lab Ends and Pipeline Validation Begins

The boundary that matters is not a file format. It is a moment, and a sentence about scope.

Kanna Nandakumar, PhD

Ask a bioinformatician where the pipeline starts and you get a file name. The instrument writes base calls, a converter turns them into FASTQ, and everything downstream is the pipeline. It is a tidy answer, and for validation purposes it settles almost nothing. The question a validation plan has to answer is not which file marks the handoff. It is which evidence belongs to which side of it, what each kind of evidence can establish, and what has to be written down so that an inspector — or a client, or you in two years — can tell the difference.

The regulation stops at the bench

CLIA defines a test system as the instructions and all of the instrumentation, equipment, reagents and supplies needed to perform an assay and generate results.¹ Software is not on that list. In April 2022 a workgroup convened under the CLIA advisory committee met to consider, among other things, where CLIA coverage of the total testing process ought to begin and end; among its recommendations was that the definition of a test system be modified to include the algorithm or software used to generate a result.² The definition still reads the same way. As of July 2026 CMS and CDC are asking the same family of questions again in an open request for information — which NGS performance characteristics are not addressed by the current regulations, and whether facilities that only process analytical data need a CLIA certificate at all. Comments close on 14 September 2026.³

So the pipeline is regulated by inheritance rather than by name. Section 493.1253 requires a laboratory that has modified or developed a test system to establish its performance specifications before reporting patient results, and every number on the report is produced by the pipeline.⁴ The obligation reaches the code whether or not the noun appears in the text. What fills the space between that inheritance and a day's actual work is professional guidance: AMP's 2018 joint recommendation with CAP treats the pipeline as a component with its own design and development, optimization and familiarization, and validation phases, and works through each of them.⁵

The boundary is a moment

Which supplies the first half of a usable answer. Familiarization is the pilot phase, where a new or vendor-supplied pipeline is run end to end so that intermediate outputs can be inspected and unanticipated behavior found.⁵ Optimization is where you are allowed to look at results and change things: depth thresholds, strand-bias filters, the order of operations, the handling of degraded specimens. Validation is where you are not. As the step-by-step treatment in the CAP journal puts it, tuning the pipeline to reduce errors in poor-quality samples belongs in optimization, before validation begins; the workflow is locked down and then challenged.⁶

That makes the boundary temporal rather than spatial. It is not the FASTQ. It is the last moment at which a parameter could have moved in response to seeing a result. Before it you are developing; after it you are collecting evidence. The same fifty specimens will produce a persuasive sensitivity figure on either side of that line, and only one of the two figures is a performance specification — the other is a fit. A validation file that cannot show when lock-down occurred cannot show which one it is holding.

And a scope

The second half is the sentence nobody enjoys writing, because it limits the claim. A clinical pipeline is not a general-purpose instrument. It is tuned to one assay, and the most literal demonstration is duplicate removal. A hybrid-capture pipeline marks and discards reads that share start and stop positions, because in a capture library those are PCR duplicates. Hand the same pipeline — same version, running correctly — data from an amplicon assay, and it will throw away nearly everything, because in an amplicon library identical coordinates are the design.⁶ Nothing is broken. The code is answering a question about a library it was not built for.

This is why proficiency testing splits into two products, a physical specimen that exercises the whole workflow and sequence files that exercise only the informatics, and why the second is harder to design than it looks.⁶ It is also why a pipeline's established performance is never a property of the pipeline alone. It is a property of the pipeline and the configuration that produced its input: platform, chemistry, capture or amplicon, read length, specimen types, input mass, coverage target.

The asymmetry that decides what evidence you can generate

Once the scope is stated, the practical consequence follows. You can vary the pipeline while holding the specimen fixed: retained data can be reanalyzed, and variants of a specified type and allele fraction can be inserted into existing FASTQ or BAM files to challenge classes that no specimen in your validation set happened to contain. AMP, the Association for Pathology Informatics and CAP set out the forms this can take — purely simulated reads, mixed samples, downsampled files, manipulated assay data — and what each is good for.⁷ You cannot do the reverse. There is no way to vary the wet lab while holding the data fixed, because the data is what the wet lab produces.

That asymmetry is the reason in silico material is a legitimate supplement on one side of the boundary and no help at all on the other. Manipulated files test what the pipeline does with a variant; they say nothing about whether the assay would have delivered that variant to the pipeline in the first place. They also carry an assumption — that the wet-lab components have been established by other means — which is precisely the claim your configuration statement is making.

It also resolves a sample-count question that otherwise looks like a contradiction. Specimen requirements for NGS panels come from the 2017 AMP/CAP oncology-panel guideline; the 2018 pipeline guideline points back at those numbers and then adds that where the wet-lab minimum does not adequately exercise the variant types the assay claims to detect, the laboratory should add appropriate validation samples, commercially available reference materials, or verified in silico samples.^5,8 The two evidence bases genuinely separate at that point. The specimens establish what the assay does to patient material. The pipeline must additionally establish what it does to variant classes patient material did not supply.

Each evidence base answers a different question and produces a different document. None of them is transferable to another assay configuration, and none of them counts as validation until the pipeline is locked.

Attribution is optional; disclosure is not

One more consequence, and it is the one that shortens validation meetings. An end-to-end performance figure is a property of the pair. If a clinically relevant region falls below your coverage threshold because of chemistry rather than code, the pipeline is not at fault — and the disclosure obligation is identical either way. Regions that do not consistently meet quality standards have to be recorded at validation whichever component produced them, and arguing about which one did is not a substitute for writing them down.⁶

Precision is where the two sides come back together, and the design of that study is itself a disclosure. Replicates run by different technologists, on each instrument that will be used clinically, within and between runs, on each analysis environment, and by different analysts, tell you which of those variables the result is insensitive to.⁶ Whatever was held constant is a variable you have not characterized. That list belongs in the file next to the numbers, not in someone's memory.

The same logic governs ownership. Whether the analysis runs on the laboratory's own systems or is handed to an outside informatics provider, the laboratory is responsible for validating the accuracy of the entire process.² Splitting the work does not split the obligation. It only adds a section to the file — a subject large enough to deserve its own piece.

None of this calls for a new document set. It calls for two performance summaries that each name what they are conditional on, and one sentence joining them: these specifications were established on data produced by this configuration, with the pipeline locked on this date. Write that sentence and most of the arguments about where the wet lab ends stop being arguments.

What goes in the file

• A configuration statement. The wet-lab conditions the pipeline's specifications were established under: platform and chemistry, capture or amplicon, read length, specimen types, input mass and quality range, coverage target. This is the scope of every performance claim that follows.
• The lock-down record. Component and version list, and the date after which no parameter changed, with the optimization work that preceded it filed separately so the two are not confused.
• A validation sample inventory, split three ways. Clinical specimens through the whole assay; reference and engineered materials; in silico and manipulated files. For each generated file, its provenance and what it was designed to challenge.
• The precision study design. What was deliberately varied — operator, instrument, run, analysis environment, analyst — and, by omission, what was not.
• A limitations record. Regions and variant classes that did not consistently meet quality standards during validation, recorded without requiring an attribution to bench or code.
• A statement of what ran outside your systems and how the laboratory satisfied itself that it was accurate.

References

1. 42 CFR § 493.2, Definitions (“test system”). ecfr.gov
2. CLIA Regulations Assessment Workgroup, meeting summary, 1 April 2022. Centers for Disease Control and Prevention. stacks.cdc.gov
3. Request for Information: Clinical Laboratory Improvement Amendments of 1988 (CLIA) Regulations. 91 FR 43586, 16 July 2026 (CMS-3485-NC); comments due 14 September 2026. federalregister.gov
4. 42 CFR § 493.1253, Standard: Establishment and verification of performance specifications. ecfr.gov
5. Roy S, Coldren C, Karunamurthy A, et al. Standards and Guidelines for Validating Next-Generation Sequencing Bioinformatics Pipelines: A Joint Recommendation of the Association for Molecular Pathology and the College of American Pathologists. J Mol Diagn. 2018;20(1):4–27. doi:10.1016/j.jmoldx.2017.11.003
6. SoRelle JA, Wachsmann M, Cantarel BL. Assembling and Validating Bioinformatic Pipelines for Next-Generation Sequencing Clinical Assays. Arch Pathol Lab Med. 2020;144(9):1118–1130. doi:10.5858/arpa.2019-0476-RA
7. Duncavage EJ, Coleman JF, de Baca ME, et al. Recommendations for the Use of in Silico Approaches for Next-Generation Sequencing Bioinformatic Pipeline Validation: A Joint Report of the Association for Molecular Pathology, Association for Pathology Informatics, and College of American Pathologists. J Mol Diagn. 2023;25(1):3–16. doi:10.1016/j.jmoldx.2022.09.007
8. Jennings LJ, Arcila ME, Corless C, et al. Guidelines for Validation of Next-Generation Sequencing-Based Oncology Panels: A Joint Consensus Recommendation of the Association for Molecular Pathology and College of American Pathologists. J Mol Diagn. 2017;19(3):341–365. doi:10.1016/j.jmoldx.2017.01.011