Reportable Range Is Not the Gene List

Zetobit, LLC The Validation File

The Validation File

Reportable Range Is Not the Gene List

Three boundaries hide inside one phrase — what the assay interrogates, what the validation established, and what the pipeline will print. Only two of them stand still.

Kanna Nandakumar, PhD  ·  Zetobit, LLC

A panel is described as 523 genes. The requisition says 523 genes, the brochure says 523 genes, and the section of the SOP headed reportable range says 523 genes. None of those sentences is a reportable range. They describe what was ordered. A reportable range is a statement about what the laboratory is prepared to defend having said — and about the positions where it will decline to say anything at all.

CLIA lists reportable range among the characteristics a laboratory must establish for a modified or in-house test, alongside analytical sensitivity.¹ Both terms arrived with a different kind of assay in view. As Gargis and colleagues put it, the CLIA performance-characteristic definitions were written for quantitative single-analyte tests — the reportable range of a glucose assay is the span of concentrations it measures accurately — and establishing them for a method that interrogates an effectively unbounded number of targets is a translation problem each laboratory has to solve for itself and write down.²

The translation usually collapses three separate boundaries into one paragraph. They have different owners, rest on different evidence, and — the part that causes trouble — change at wildly different rates.

1. What the assay interrogates

Between the genes on the requisition and the positions where the laboratory has evidence sit two narrowings: from genes ordered to regions actually targeted by the probe or amplicon design, and from regions targeted to regions where performance was demonstrated. FDA keeps the ends of that chain apart in its labeling recommendations. For a targeted panel, list the genes on the panel; and, separately, identify the regions of the genome in which sequence meeting pre-specified performance specifications can be generated.³ Two sentences in one label, because they are two different claims: one about design intent, one about evidence.

The same guidance asks for something most validation packets do not contain. Known clinically relevant variants for the stated indication that fall outside the region the test detects should themselves be specified.³ That is an obligation to document the complement, not just the content — and it is the difference between a panel description and a scope statement. It also answers the question a clinician actually has, which is never “what does this cover” but “what would this have missed.”

2. What the assay can detect

The second boundary is a level rather than a territory, and it is a probability statement. FDA defines the lower limit of detection as the lowest concentration at which at least 95% of calls are positive, with an acceptable rate of invalid or no-call results, among the replicates tested at that concentration.³ That definition has a consequence: an LoD requires replicates clustered near the boundary. It is a different experiment from the accuracy study, and a validation assembled entirely from concordance against reference materials has not performed it. It is also inseparable from depth: AMP and CAP recommend a minimum of 250 reads per tested target for somatic variant detection,⁷ which means a stated allele fraction carries no information without the coverage floor it was measured at.

Three things routinely go missing. The first is stratification: FDA asks for LoD by variant type and in different sequence contexts, for an upper limit as well as a lower one, and for an acceptable DNA input range with a maximum as well as a minimum.³ Most packets carry a single lower LoD, no upper bound, and only an input floor.

The second is why context is not a technicality. Sequencing ultra-deeply — to 1,300,000× — Hadigol and Khiabanian found more than 100-fold variation in error rate within a single substitution type depending on the flanking bases: T>G transversions in a GTC trinucleotide occurred at 133.5 times the rate of the same substitution in ATA. Depth does not rescue this uniformly. The T>G error rate in GTC stopped improving beyond 10,000×, while in TTA it kept improving above 500,000×. Their stated conclusion is evidence against assigning a single allele-frequency threshold for calling.⁴ A single LoD is the limit of detection of an average sequence context, which no specific variant occupies.

The third is the germline case. In a constitutional assay the allele fraction is fixed by biology at roughly half or all of the reads; there is no low-fraction axis to have a limit on. The limiting quantities are input and coverage, which is exactly why FDA’s germline recommendation is an input range plus dilution studies for mosaic content.³ A packet that imports somatic LoD language into a germline validation has documented a characteristic the test does not have, and left the one it does have unestablished.

The three boundaries drawn on the two axes they actually live on. The hatched zones are where a laboratory is exposed: territory it interrogates but never validated, territory it validated but did not print, and the band between a filter threshold and the validated floor where results are issued on evidence that was never gathered.

3. What the pipeline will print

The third boundary does not appear in the validation report at all. It lives in a configuration file: minimum depth, minimum alternate reads, minimum allele fraction, minimum call quality, and the logic deciding which positions count as callable in this specimen. FDA treats these as design elements to be selected, justified, documented and versioned, with the filtering criteria and their purpose recorded explicitly.³ AMP and CAP say the same of the pipeline as a whole: it is a component with performance to be established, not plumbing that comes with the sequencer.⁵

Misalignment runs in two directions and neither shows up in a summary. A filter stricter than the validated LoD discards sensitivity the laboratory paid to establish, in exchange for specificity nobody specified. A filter looser than the validated LoD issues results from a region of the measurement space where no evidence was gathered. Neither is a scandal. Both are decisions, and decisions belong in the record as decisions rather than as defaults inherited from a tool’s documentation.

This is also the boundary that moves. FDA is explicit that variants should not be reported from regions of a run that fail quality thresholds, and that regions left uninterrogated for that reason should be reported as such.³ That makes the reportable range a per-run output, not a static paragraph. Two specimens run on the same panel in the same week have different reportable ranges, and the difference is a property of the specimen rather than of the test.

Boundary	Fixed by	Lives in	Changes when
Interrogated	Assay design	Design spec and target file, versioned to a reference build	Redesign — rare, deliberate
Validated	Validation study	Validation report, stratified by variant type and context	Revalidation — rare, deliberate
Printed	Pipeline config and run QC	Version-controlled filter spec plus the callable file issued per sample	Every run — and any config edit

The last column is the reason to hold the three apart. The first two change by deliberate act, and both announce themselves: somebody signs a redesign, somebody signs a revalidation. The third changes every time a specimen underperforms, and it can also change through a one-line edit to a threshold that no one classifies as a modification. A laboratory keeping all three in the same SOP paragraph has no mechanism for noticing when the third has drifted away from the second. What counts as a change, and what has to happen when one occurs, is the subject of a later piece in this series. The prerequisite is having three documents instead of one sentence.

For a CAP-accredited laboratory the operative version of all this is the checklist rather than the CFR, and the inspector reads the validation packet against it. CAP’s expectations for next-generation sequencing tests are also set out in the peer-reviewed literature, which is where they can be read by anyone weighing a laboratory’s documents from outside the accreditation process.⁶ A prospective client, a partner in a co-development deal and an inspector are all asking the same question about the reportable range, and only one of them can see the checklist.

What goes in the file

• A target definition as an actual interval file, versioned and tied to a named reference build — not a gene count, and not a gene list.
• An out-of-scope statement: clinically relevant variants for the stated indication that lie outside what the assay detects, and the reason in each case (not targeted, not coverable, not callable by this method).
• LoD studies stratified by variant type and sequence context, with the replicate design at the boundary shown, an upper limit where one applies, and the DNA input range with both a floor and a ceiling.
• A filter specification listing every threshold that determines whether a call reaches the report, each with a justification, under version control — and a written reconciliation against the validated limit of detection.
• The per-run callable output issued with each report, plus the report language stating which interrogated regions failed quality thresholds for that specimen.
• A named owner for each boundary and a rule for what happens when two of them disagree — because they will, and the moment of discovery should not be the inspection.

A quick test of whether these exist as documents or as institutional memory: hand someone the reportable-range paragraph from the SOP and the callable-region file issued with last week’s report, and ask whether they describe the same test. If answering requires finding the person who maintains the pipeline, the boundary is not documented. It is remembered, which is a different thing and does not survive their notice period.

References

1. 42 CFR §493.1253 — Standard: Establishment and verification of performance specifications. Electronic Code of Federal Regulations. ecfr.gov
2. Gargis AS, Kalman L, Lubin IM. Assuring the Quality of Next-Generation Sequencing in Clinical Microbiology and Public Health Laboratories. J Clin Microbiol. 2016;54(12):2857–2865. doi:10.1128/JCM.00949-16
3. US Food and Drug Administration. Considerations for Design, Development, and Analytical Validation of Next Generation Sequencing (NGS)-Based In Vitro Diagnostics (IVDs) Intended to Aid in the Diagnosis of Suspected Germline Diseases. Guidance for Stakeholders and FDA Staff, April 2018. fda.gov
4. Hadigol M, Khiabanian H. MERIT reveals the impact of genomic context on sequencing error rate in ultra-deep applications. BMC Bioinformatics. 2018;19:219. doi:10.1186/s12859-018-2223-1
5. Roy S, Coldren C, Karunamurthy A, et al. Standards and Guidelines for Validating Next-Generation Sequencing Bioinformatics Pipelines: A Joint Recommendation of the Association for Molecular Pathology and the College of American Pathologists. J Mol Diagn. 2018;20(1):4–27. doi:10.1016/j.jmoldx.2017.11.003
6. Aziz N, Zhao Q, Bry L, et al. College of American Pathologists’ Laboratory Standards for Next-Generation Sequencing Clinical Tests. Arch Pathol Lab Med. 2015;139:481–493.
7. Jennings LJ, Arcila ME, Corless C, et al. Guidelines for Validation of Next-Generation Sequencing–Based Oncology Panels: A Joint Consensus Recommendation of the Association for Molecular Pathology and College of American Pathologists. J Mol Diagn. 2017;19(3):341–365. doi:10.1016/j.jmoldx.2017.01.011

Educational content only; not clinical, diagnostic, legal or regulatory advice. Regulations, guidance documents and professional standards change — this piece reflects the state of those documents as of its publication date. All figures and examples are illustrative.