Skip to content
Methods

Reference checks

Reference checks: Drybench Pipeline

This page lists the reference checks published for Drybench Pipeline. Each set gives the methods it covers, what the expected values were computed with, the number of cases, the tolerance the app must meet, the date, and the file holding the inputs and expected outputs, so you can open a case and compare it with what the app gives.

Every case was chosen and recorded by the Drybench team; none has been independently reviewed. A reference check tests the calculation, not your data, your experiment or whether a model suits it. Methods without a set are listed on the methods page with their level.

Methods and levelsDocumentation

Published sets

No verification set has been published for this app yet.

Fixture tables, computed when the page was built

Status: draft. Expected values transcribed from FDA documents in the build session, 5 October 2026; not independently checked (Q-19). FDA precedent's release gate is not met until a reviewer with regulatory-affairs experience signs these fixtures.

  1. Review time and designations: products checked against their approval letters

    48 products

    Compares
    The join by application number, the receipt and approval dates, the review time in days and every designation flag's reading, for a seeded sample of FDA's compilation stratified by oncology, NDA or BLA and approval before or after 9 July 2012.
    Expected values from
    Each product's Drugs@FDA approval letter (its receipt sentence quoted with the page and the PDF's SHA-256), Drugs@FDA's approval date and review priority, and FDA's breakthrough lists.
    Tolerance
    Dates and days exact. Where an approval letter and the compilation differ, the difference is recorded and the app keeps the compilation's date, which the data dictionary defines; an unrecorded difference fails the test.
    Files
    src/components/tools/drug-pipeline-explorer/precedent/fixtures/products.json
    pipeline/drug-pipeline-explorer/precedent/fixtures/build_products.py
    src/components/tools/drug-pipeline-explorer/precedent/nme.test.ts

    237 candidates replaced in their stratum (no letter online, or no receipt date in its text); listed in the fixture.

    4 recorded disagreements between FDA documents:

    • Entereg: receipt, approval letter 2007-08-10, compilation 2004-06-25
    • Lymphoseek: receipt, approval letter 2012-10-31, compilation 2011-08-10
    • Baraclude: receipt, approval letter 2004-09-30, compilation 2004-09-29
    • Trelstar Depot: receipt, approval letter 1999-12-16, compilation 1996-06-26

  2. Complete response letters joined to the later approval

    20 letters

    Compares
    Whether each letter joins to the later approval of the same application, and the interval in days; at least five negative cases (a supplement, another letter type, an application never approved).
    Expected values from
    The application number printed on the letter's first page (or FDA's file name for a scanned letter), the date in FDA's file name, and the original approval in Drugs@FDA.
    Tolerance
    Exact.
    Files
    src/components/tools/drug-pipeline-explorer/precedent/fixtures/crl-joins.json
    pipeline/drug-pipeline-explorer/precedent/fixtures/build_crl.py
    src/components/tools/drug-pipeline-explorer/precedent/crl.test.ts

  3. Trials named in a label's Clinical Studies section

    30 labels, 10 naming no trial

    Compares
    The NCT numbers read from openFDA's section 14 text, in order, against those in the label's PDF.
    Expected values from
    DailyMed's PDF of the same label set id and version as openFDA's record.
    Tolerance
    Exact, including labels that name none.
    Files
    src/components/tools/drug-pipeline-explorer/precedent/fixtures/section14.json
    pipeline/drug-pipeline-explorer/precedent/fixtures/build_section14.py
    src/components/tools/drug-pipeline-explorer/precedent/pivotal.test.ts

  4. The seven example searches

    7 searches, 251 products

    Compares
    For every US product of each example search: the compilation row it joins to, its pathway, and its review time or the reason it has none.
    Expected values from
    A second implementation of the rules, in Python, reading FDA's compilation spreadsheet rather than the app's snapshot.
    Tolerance
    Exact.
    Files
    src/components/tools/drug-pipeline-explorer/precedent/fixtures/examples.json
    pipeline/drug-pipeline-explorer/precedent/fixtures/build_examples.py
    src/components/tools/drug-pipeline-explorer/precedent/examples.test.ts

Run by npm test in CI on every change.