Oracles and verification
Patent captured-source reference (18 October 2026)
pipeline/patent-exclusivity/patent_capture_reference_v1.py independently
transcribes capture 2d0b7be911499788d58924e6a16b145eeab5cb01 into the write-once
fixtures/capture-2026-10-17/reference-1.json, before engine corrections.
The 439 raw rows retain source/member hashes, exact text, original line and
duplicate occurrence, parsed-date observations and independent summary facts.
The two Eliquis NDAs, 20 NDA cases, ten PIV RLDs and ten BLAs are selected by
the 18 October CSO memo. PIV exact-key refusals include every one of the 76
unassigned rows; 22 numeric-brand adverse rows prevent first-digit matching.
Purple semantic CSV keys and historical raw-split capture keys are distinct.
Full --check rederives from all five pinned capture files and refuses any
different fixture bytes. CI rederives the retained excerpt observations and
checks the script/raw-row hashes; the full-capture check additionally proves
selection completeness and was run before commitment. --out refuses
overwrite. F1/F2/F3/F5 retain missing original prerequisites; F4/F6/F7 are
explicitly unavailable. These are mechanical source fixtures, not reviewed
regulatory interpretations; SCI-124/Q-19 remains open. Old fixtures remain
byte-for-byte unchanged.
The existing Python trace also produces the new
fixtures/synthetic-trace-2/expected.json from the unchanged synthetic inputs,
before the TypeScript fix. Its exact regression updates follow SCI-228/230:
the separate Purple reference-product column and exact NDA-only PIV joins.
The original expected JSON remains intact. The trace writer now refuses
overwrite rather than relying on callers to choose a new output path.
How the suite's scientific numbers are checked against references that do not come from the engine under test: the oracle scripts under pipeline/, the fixtures they write, the unit tests that read them, the hash checks that keep the fixtures honest, the copies published under public/verification/, the scheduled re-checks and the evidence records. Snapshots, data builds and source verification are covered in Data pipeline and are not repeated here.
Terms and rules
An oracle is a named tool or worked method, independent of the TypeScript engine, that produces the expected value a test compares the engine against. A fixture is the committed file holding those expected values. A verification set is a fixture published for one app release and listed in the registry.
The rules come from the design briefs and the board, and are labelled here as intent where the code does not enforce them:
- The oracle rule (
docs/design/brief-lab-calculators.md, "Three rules apply"): constants carry a source and date; fixtures come from a named external oracle with the script and version checked in and the fixture file hashed; no fixture is produced by the engine under test. - The badge rule (same section): no agreement claim without the oracle's name, version, fixture count, tolerance and date. The pattern is "= {oracle} {version} within {tolerance} on {N} fixtures ({date})". Copy says "checked against", never "validated".
- Oracle before engine: the assay, sequence, plasmid and bioreactor READMEs and script headers state that the fixtures were written before the engine as step 1 of each build.
- Rule G (
docs/design/brief-assay-analyser.mdsection 10.2,docs/design/brief-lab-calculators.mdsection 10.2): GPL or academic-only packages may be run offline as oracles and their numerical outputs committed as fixtures; their code, parameter tables and data are never shipped, ported or listed as dependencies. The legal analysis in those briefs is the design documents' position. Whether it holds for a sale is held for the General Counsel. - Roadmap rule 1 (
docs/roadmap.md, "The rules that keep breadth from breaking things"): "Every new method ships with an independent reference fixture and a methods entry, or it does not ship." Several shipped methods do not meet it; see Methods without an independent oracle.
Not every fixture in the repository comes from an oracle. The bioreactor golden runs (src/lib/hosts/golden/), the plasmid versioned data, the Assay project-file fixtures (pipeline/assay-analyser/fixtures/project-files/) and the MTT figure scene (pipeline/assay-analyser/fixtures/figure/mtt-scene-v1.json, labelled "written by the engine, not an oracle" in its about block) are written by the code itself and protect against regressions only. They are described below so they are not mistaken for independent checks.
Where verification is declared
| Place | What it holds | Checked by |
|---|---|---|
content/apps.ts, type Oracle |
Each app's oracles[]: name, version, licence, what it was used for, purpose ("oracle" or "build-time") and the date it was last run |
content/apps.test.ts ("oracles and verification sets are well formed") |
content/apps.ts, type VerificationSet |
Each app's verificationSets[]: id, method ids, the oracle's name (must be listed in oracles), fixture count, tolerance text, date, and the file name under public/verification/<slug>/<version>/ |
content/apps.test.ts; per-app tests listed below. For the Assay, content/apps.test.ts also requires every verified MethodRef in content/apps/assay-analyser.methods.ts to be named by a set and every set's method id to be a MethodRef (added 20 October 2026, when 22 verified methods were named by no set and the combination set named ids no MethodRef carried) |
content/apps/<slug>.verification.ts |
The dates, versions, counts, tolerances and badge sentences each app's registry entry, methods module and pages share. No imports, so the registry never pulls engines into a first load | Per-app tests compare the counts with the fixture files (for example src/lib/hosts/fixtures.test.ts, src/apps/sequence-toolkit/sequence.test.ts, src/apps/lab-calculators/calculators.test.ts) |
content/apps/<slug>.methods.ts |
METHODS, one MethodRef (src/components/suite/data/provenance.ts) per method, with verification: "verified" | "unit-tested" | "unverified" and a verificationNote |
src/apps/lab-calculators/calculators.test.ts and src/apps/sequence-toolkit/sequence.test.ts require a "verified" note to name what it was checked against |
content/apps/drug-pipeline-explorer.verification.ts |
The Pipeline app's precedent verification sets (a different shape: fixture, script, test, compares, tolerance, oracle) | The precedent tests named in each set |
content/apps.test.ts also fixes which apps publish verification sets: lab-calculators, sequence-toolkit, assay-analyser, plasmid-map, bioreactor-simulator, protein-explorer and molecule-workbench. Every other app must have none, and its docs page says none is published. The docs page's verification block (src/components/suite/marketing/docs-blocks.marketing.tsx) links each set to verificationFileUrl(slug, set.version ?? app.version, set.file) (content/apps/routes.ts), so each listed file must exist under its set's own version, or under the app's current version when the set names none; the same test checks it. A set's version may be later than the app's (Calc's 0.3.0 sets under app version 0.2.0; Bioreactor's 0.3.0 and 0.4.0 measured-comparison sets under 0.2.0), and the page then says which version it was recorded for.
scripts/licences.mjs lists every oracle whose licence matches GPL, academic or non-commercial with its purpose, and greps src/ and content/ for GPL headers and the identifiers of GPL tools (Primer3, SMS2, EMBOSS, pLannotate, CRISPOR), with named per-file exemptions for the sequence app's registry and copy that name Primer3 as an oracle. It does not scan pipeline/ or public/verification/. See Dependencies and licences.
The oracle environment
pipeline/oracle-env/ defines one container image for the R and Python oracles (CTO memo of 12 October 2026, ruling 3). Before it, each fixture recorded the machine that happened to run it: R 4.3.3 from Ubuntu packages for the version-1 Assay fixtures, no R at all for the three 0.4.0 fixtures whose cross-checks say "not run", and Python 3.11.17 where requirements-scipy.txt names 3.11.15.
| Layer | Contents | Pinned by |
|---|---|---|
| Base | rocker/r-ver:4.4.3 |
the FROM line |
| CRAN | Posit Package Manager's dated snapshot of 30 September 2026 (https://p3m.dev/cran/__linux__/<Ubuntu codename>/2026-09-30), the first that serves drc 4.0-0 |
ARG CRAN_SNAPSHOT |
| R packages | drc, car, chemCal, investr, outliers, multcomp, mvtnorm, jsonlite, digest and their dependencies; MASS and survival as R 4.4.3 ships them | the snapshot; the build fails if one is missing, and prints (does not fail on) a drc other than 4.0-0, which reference.R then refuses |
| Python | CPython 3.11.15 from uv 0.11.32 |
ARG PYTHON_VERSION, the uv image tag |
| Python environments | /opt/venv/numpy (requirements.txt), /opt/venv/scipy (requirements-scipy.txt), /opt/venv/irls (requirements-irls.txt), each with openpyxl 3.1.5 |
the requirements files |
| Second layer: Sequence and Calc (12 October 2026) | /opt/venv/sequence (pipeline/sequence-toolkit/requirements.txt: Biopython 1.88 with Bio.Restriction, primer3-py 2.3.1); /opt/venv/calc (pipeline/lab-calculators/requirements.txt: Biopython 1.85, RDKit 2026.3.6, NumPy 2.4.6); MAFFT 7.526, the BSD core built from the source tarball checked by SHA-256; EMBOSS 6.6.0 (GPL; for cai) from the distribution |
ARG MAFFT_VERSION, MAFFT_SHA256; the requirements files; the build fails on another EMBOSS or MAFFT |
run-oracle <script> (from run-oracle.sh) runs a script under pipeline/ with its interpreter: Rscript for .R; for .py the environment mapped by path (generate_curves.py numpy, relative-irls-reference.py irls, oracles/ and walk/ scipy, pipeline/sequence-toolkit/peptides.py calc because it needs RDKit only, the rest of pipeline/sequence-toolkit/ sequence, pipeline/lab-calculators/ calc), or the one named by ORACLE_VENV; anything else is refused. The last build step, record-environment.sh, writes R-packages.csv, python-packages.txt and environment.json into /opt/oracle-env/; the first two are committed in pipeline/oracle-env/ once the image has built (placeholders until then). The image is never shipped and runs its GPL tools offline (Rule G); the licence position is in pipeline/oracle-env/README.md.
.github/workflows/oracles.yml builds or pulls the image, runs one named script in it on demand (script, version) and uploads what it wrote for a builder to commit, and reruns every Assay oracle weekly to detect drift; see Build, test and CI. On 12 October 2026 the Python half was checked outside the image (the same three environments under CPython 3.11.15): every Assay Python oracle the manifests name reproduced its committed fixture byte for byte, except that the three 0.4.0 fixtures record oracle.python as 3.11.17. The same day the Sequence oracles the Sequence manifest names were run under 3.11.15 in the two new environments: every fixture reproduced with only recordedOn changed (the run's date), and reference.py rewrote manifest.json without the entries of the other Sequence scripts, which a Sequence drift check will have to allow for. The first image build (Oracles run 38263006728) is recorded in docs/progress/oracle-env.md. Its drift job ran the 13 Assay scripts in the image: 11 completed, of which seven reproduced their files byte for byte apart from recordedOn, the three 0.4.0 fixtures differed in oracle.python and the plate walk's row in its recorded MASS version (and possibly further: the comparison then reported only the first difference, and now lists all); zprime_degenerate.R lacked the digest package (added since), and reference.R stopped in limiting_model (Known issues, SCI-37). A second drift run on the final image (Oracles run 38266906174) landed on an AVX2-only runner: reference.R completed, but the SciPy fixtures moved at about 1e-9 relative and inhibition_global.py failed its own 1e-9 metamorphic check. NumPy's SIMD dispatch and OpenBLAS pick kernels by CPU, so an oracle's last digits depend on the runner's CPU class, image or not (Known issues, REL-33); the run and drift jobs record it, and every oracle run is now pinned to x86-64-v3 and checked before it starts (pipeline/oracle-env/cpu_check.py; pipeline/oracle-env/README.md, "The CPU class").
The about.environment and about.crossCheck blocks
The CTO's ruling 3.3 (memo of 12 October 2026, docs/board/memos/2026-10-12-cto-assay-stage.md) sets two blocks in every new fixture's about, so it can be rerun in the same environment. Inside the image ORACLE_ENV_JSON points to environment.json (these keys, with image "local"); the workflow that runs the script sets ORACLE_ENV_IMAGE to the image's tag and digest.
| Key | Meaning |
|---|---|
about.environment.image |
ghcr.io/biotechmat/drybench/oracle-env:<tag>@sha256:… from the workflow, or "local" |
about.environment.r, packages |
R.version.string; the R packages the script loaded, with versions |
about.environment.python, scipy, numpy |
The CPython version and the environment's SciPy and NumPy |
about.environment.recordedOn |
The date of the run |
about.environment.cpu, cpuClass, simd, openblasCore, threads |
The runner's CPU model; the pinned class "x86-64-v3"; NumPy's enabled dispatch targets (Python only); OpenBLAS's core (Haswell); threads (1). Copied from ORACLE_CPU_JSON, which run-oracle writes after cpu_check.py has verified the pin (CTO ruling of 12 October 2026; Known issues, REL-33) |
about.crossCheck.tool, version |
The cross-checking tool and its version |
about.crossCheck.ran |
true or false; the badge sentence derives from it, and "R not run" is permitted only with ran: false and reason |
about.crossCheck.reason, result |
Why it did not run; what agreed when it did |
The five -v2 Assay fixtures published as 0.5.0 (12 October 2026) are the first to carry these blocks; the 0.4.0 files stay without them under write-once, and adding them to any other fixture means rerunning its script in the image and publishing a new version, which the CSO rules on (Known issues, SCI-35). The CTO's fallback, if the image is not green within two focused days, is the CSO's: SciPy alone with the badge "R drc not run", per fixture, until it is.
Calc (lab-calculators)
Script. pipeline/lab-calculators/reference.py (787 lines), sub-commands dilution, formulas, buffers, extinction, publish and all.
Oracles.
| Sub-command | Method | Oracle |
|---|---|---|
dilution |
Molarity, C1V1, "weigh what you weighed", purity and salt corrections, serial plans and their bench flags, unit chains (ng/µL to nM, percent w/v to M) | Python fractions.Fraction and decimal.Decimal (40-digit context), exact |
formulas |
Formula weights of salts, hydrates and parents | The formula from RDKit CalcMolFormula on PubChem's SMILES for organics (CID and InChIKey recorded), hand-typed formulas for inorganics, and the weight as a CIAAW 2021 sum in exact decimals. The CIAAW table is transcribed a second time inside the script, apart from src/lib/units/atomic-weights.ts. RDKit's own MolWt (which uses S 32.067 and Cl 35.453) is recorded beside, never as the expected value |
buffers |
pK at 4, 25 and 37 °C | Goldberg, Kishore and Lennen 2002, equation 12 (Clarke and Glew form with the ΔrCp° term) in NumPy, with pK, ΔrH° and ΔrCp° transcribed a second time from the paper's section 8 |
extinction |
ε280 by the Pace coefficients, both cysteine conventions | Biopython Bio.SeqUtils.ProtParam on 20 UniProt sequences fetched on the day plus constructs; sequences with U or X refused |
Pins. pipeline/lab-calculators/requirements.txt: rdkit==2026.3.6, numpy==2.4.6, biopython==1.85, "Python 3.11". The script records the versions it ran with but does not refuse other versions.
Fixtures. Unlike the other apps, Calc's fixtures are written under src/. The 0.1.0 set: src/lib/dilution/fixtures/dilution.json (31 cases), src/lib/units/fixtures/formula-mass.json (58), src/lib/dilution/fixtures/buffers.json (15), src/lib/seq/fixtures/extinction.json (22). Each file carries an oracle block (name, version, script, command), recordedOn and count.
Consumers and tolerances.
| Test | Compares | Tolerance |
|---|---|---|
src/lib/dilution/fixtures.test.ts |
Every dilution case; display strings | Nine significant figures (1e-9 of scale); display values exact |
src/lib/dilution/fixtures.test.ts |
pK at three temperatures | Within 0.01 (the test title also states 0.0001 for the same equation) |
src/lib/units/formula-mass-fixtures.test.ts |
Formula weights, Hill formulas, hydrate arithmetic, parent fractions; the oracle's second CIAAW transcription against the module's table element by element | Exact (both are decimal sums of the same table) |
src/lib/seq/extinction-fixtures.test.ts |
ε280 in both conventions; cross-checks the nine sequences also in src/lib/seq/fixtures/protparam.json |
Exact (integer coefficients) |
src/lib/fixture-hashes.test.ts |
File integrity (see Fixture integrity checks) | Byte-identical |
Regenerating. python pipeline/lab-calculators/reference.py all --version 0.1.0 (the script's documented example). formulas and extinction fetch PubChem PUG REST and rest.uniprot.org live and do not save the responses, so a rerun depends on what those services return that day.
Wave 1 and Wave 3 scripts. One standard-library script per method, each written and committed before the engine ran on it (pipeline/lab-calculators/):
| Script | Fixture (version) | Cases | Oracle |
|---|---|---|---|
master_mix_reference.py |
src/lib/dilution/fixtures/master-mix.json (0.2.0) |
14 | Python fractions: the Thermo Fisher MAN0025706 B pp. 12–13 compositions in both plate formats, a user dead volume, warnings, six refusals |
intermediate_reference.py |
src/lib/dilution/fixtures/intermediate-dilution.json (0.2.0) |
13 | An exact brute-force grid over stock draw and final transfer; each infeasible case carries an exact certificate |
serial_reference_2.py |
serial-2.json (0.3.0) |
10 | Python fractions, serial-dilution 2.0.0 (constant transfer) |
plate_sheet_reference.py, plate_sheet_reference_2.py |
plate-sheet.json, plate-sheet-2.json (0.3.0) |
22 each | Python fractions; the second on constant transfer is the one the engine is checked against |
preparation_reference.py, _2.py, _3.py |
src/lib/preparation/fixtures/preparation.json, -2.json, -3.json (0.3.0) |
41 each | Python fractions and decimal for cells, centrifuge, percent and antibody; -2 names the antibody intermediate's carrier, -3 checks that case feasible on Wave 1's grid first; -3 is the one tested |
antibody_batch_reference.py |
antibody-batch.json (0.3.0) |
10 | Python fractions, antibody 1.1.0 with allowances |
od600_reference.py |
od600.json (0.3.0) |
14 (9 refusals) | Python fractions; every factor a stated example of a person's own calibration |
buffer_recipe_reference.py |
src/lib/dilution/fixtures/buffer-recipe.json (0.3.0) |
18 (8 refusals) | CIAAW 2021 sum in Python decimal; Goldberg equation 12 in NumPy 2.5.3 on the second transcription |
unit_converter_reference.py |
src/lib/units/fixtures/unit-converter.json (0.3.0) |
44 (16 refusals) | Python fractions with BIPM SI Brochure factors |
a260_reference.py |
src/lib/seq/fixtures/a260.json (0.4.0, 17 October 2026; SCI-147) |
14 (2 refusals) | Python fractions 3.13.16: the A260 rules of thumb (dsDNA 50, RNA 40, ssDNA 33 µg/mL per unit at 1 cm), transcribed a second time from the Thermo Fisher pages and Cavaluzzi and Borer 2004 (the CSO's ruling of 17 October 2026 on SCI-145, ruling 5); dilution and path-length cases; path ≤ 0 refused. The factors are conventions, so the set checks the arithmetic and the transcription, not the factors' accuracy |
a260_reference_2.py |
src/lib/seq/fixtures/a260-2.json (0.5.0, 17 October 2026; SCI-148) |
19 (7 refusals) | Python fractions 3.13.16: the 13 cases of a260.json other than zero-reading, carried forward unchanged (the script asserts it), and the refusals of method 1.1.0 by problem code: zero and negative readings (reading-not-positive), NaN and ±∞ (reading-not-finite, encoded as strings because JSON holds neither), path ≤ 0; plus the smallest accepted reading. about.supersedes names the 0.4.0 case it replaces (the CSO's ruling of 17 October 2026, docs/board/memos/2026-10-17-cso-a260-refusals.md); a260.json stays published |
a280_reference.py |
src/lib/seq/fixtures/absorbance-a280.json and absorbance-ratio.json (0.6.0, 17 October 2026; SCI-149) |
14 (10 refusals) and 10 (8 refusals) | Python fractions 3.13.16: A280 = A/(ε·l) and c·M with ε280 and the mass as inputs, so the set is independent of the ε280 engine; the ratio A260/A280; the refusals of a280-beer-lambert 1.1.0 and a260-a280-ratio 1.0.0 by problem code and in the CSO's order (docs/board/memos/2026-10-17-cso-a280-ratio-refusals.md). The ratio set has three refusals beyond the CSO's seven so it meets the hash check's minimum of ten |
cell_counting_reference.py |
src/lib/preparation/fixtures/cell-counting-v2.json (0.7.0, 18 October 2026; Contract 10) |
25 (14 refusals) | Python fractions and decimal 3.9.6: volume-weighted chamber count pooling, viability from paired counts, safe-integer checks, 14 refusals in the CSO's order |
dose_plan_reference.py |
src/lib/preparation/fixtures/dose-plan-v2.json (0.7.0, 18 October 2026; Contract 10) |
24 (13 refusals) | Python fractions 3.9.6: neutral quantity calculation from dose per cell and titre, five quantity bases, volume scaling across L, mL, µL; allocation not assessed; 13 refusals in the CSO's order |
cell_counting_reference_3.py |
src/lib/preparation/fixtures/cell-counting-v3.json (0.7.0, 18 October 2026; Contract 10) |
28 (15 refusals), plus 6 receipt-integrity cases | Python fractions and decimal 3.9.6: volume-weighted chamber count pooling, viability from paired counts, safe-integer checks, numeric-range overflow refusal, seeding-bridge equality, 6 receipt integrity corruptions |
dose_plan_reference_3.py |
src/lib/preparation/fixtures/dose-plan-v3.json (0.7.0, 18 October 2026; Contract 10) |
35 (23 refusals), plus 7 receipt-integrity cases | Python fractions 3.9.6: neutral quantity calculation from dose per cell and titre, five quantity bases, volume scaling across L, mL, µL; all 10 unordered cross-basis pairs refusing dose-basis-mismatch, numeric-range overflow refusal, 7 receipt integrity corruptions (case 08a sets an invalid unit string, not a swap of two valid bases) |
receipt_integrity_reference.py |
src/lib/preparation/fixtures/receipt-integrity-v1.json (0.9.0, 20 October 2026; the CSO's memo of 20 October 2026, section 3) |
49: 26 that must read, 23 that must read as null | Python fractions 3.13.16, reading the published 0.7.0 v3 fixtures (their hashes in about.baseFixtures): every calculated v3 case's retained receipt rebuilt in full (the everyday-preparation@1.0.0 envelope, rows and qualification flags) after its rows are re-derived in exact fractions from the case input, then mutated copies, each stored in full. Refused: a doubled viable density (live-only, paired), a viable density off by relative 1e-6, an arbitrary total density (total-only, paired) (density-identity); a density in cells/mL, the counted volume in mL, µL or nL, and a self-consistent receipt in mL and cells/mL (canonical-unit); a PFU quantity beside a TU per viable (and per total) cell dose and each of the ten unordered pairs of the five bases swapped between the quantity and dose rows (basis-agreement). Read: all 25 calculated v3 receipts and one density off by relative 1e-12. about.supersedes says 0.7.0's case 08a tests an invalid string and the true swap is tested here; about.notCovered says the dose receipt carries no titre, so the stock volume cannot be rechecked on read |
publish_preparation.py <version> [fixture…] adds each fixture to fixture-hashes.json with its own version, script and scriptSha256, copies it to public/verification/lab-calculators/<version>/ and writes that directory's manifest.json, listing only the fixtures recorded for that version; it refuses to replace a published file or entry with different bytes. The Wave 3 fixtures were recorded with Python 3.13.16 (NumPy 2.5.3 for the buffer recipe), the contract 10 fixtures of 0.7.0 with Python 3.9.6 (each file's oracle.version and about.environment); no requirements file pins those versions. The case and refusal counts the registry's verificationSets and the method notes state are read from the published files by src/apps/lab-calculators/calculators.test.ts (audit of 20 October 2026, counting-dose M5: five places had stated 27, 31, 21 and 26).
| Test | Compares | Tolerance |
|---|---|---|
src/lib/dilution/preparation-oracle.test.ts |
Master mix; intermediate dilution (feasibility, plan within one grid step of the grid's, every constraint re-checked) | 1e-9 relative |
src/lib/dilution/serial-2-oracle.test.ts |
Serial 2.0.0 steps, volumes and flags | 1e-9 relative; flags and refusals exact |
src/lib/dilution/plate-sheet-oracle.test.ts |
Prepare and dispense rows, totals, wells, roles, sources, refusals | 1e-9 relative; the rest exact |
src/lib/preparation/preparation-oracle.test.ts, antibody-batch-oracle.test.ts, od600-oracle.test.ts |
Result rows by key, qualifications and refusals | 1e-9 relative; refusals, flags and text rows exact |
src/lib/dilution/buffer-recipe-oracle.test.ts |
Mass, pK at both temperatures, setpoint, refusals | Mass 1e-9 relative; pK 1e-3; setpoint within 0.01 and equal to two decimals |
src/lib/units/unit-converter-oracle.test.ts |
Exact strings and molar bridges | Exact strings; 1e-9 relative |
src/lib/seq/absorbance-fixtures.test.ts |
Against absorbance-a280.json and absorbance-ratio.json: concentration (mol/L, µM, g/L) and ratio; every refusal code and the order |
1e-9 relative; refusals exact by problem code |
src/lib/seq/a260-fixtures.test.ts |
Against a260-2.json: concentration read and stock (the engine has no dilution input, so the stock is the engine on the dilution-corrected reading), the factors, coverage of the three kinds and the three problem codes; that every a260.json case is carried forward unless about.supersedes names it |
1e-9 relative; refusals exact by problem code |
src/lib/preparation/cell-counting-oracle.test.ts |
Against cell-counting-v2.json and cell-counting-v3.json, every case through readCellCountingInput: total counted volume, sum of counts, viable and total densities, viability percentage and status, every refusal by problem code; the seeding bridge (case 14) and its two refusals; the v3 receipt-integrity cases |
Each fixture's declared about.tolerances (1e-9 relative, asserted to be that value); refusals, statuses and units exact |
src/lib/preparation/receipt-integrity-oracle.test.ts |
Against receipt-integrity-v1.json (0.9.0): each complete cell-counting and dose-plan receipt through the retained reader; the base fixtures' hashes |
Exact: reads unchanged or reads as null |
src/lib/preparation/dose-plan-oracle.test.ts |
Against dose-plan-v2.json and dose-plan-v3.json, every case through readDosePlanInput: required quantity, required stock volume, entered dose, allocation status, every refusal by problem code; the v3 receipt-integrity cases |
Each fixture's declared about.tolerances (1e-9 relative, asserted to be that value); refusals, units and statuses exact |
Related, not an oracle. pipeline/lab-calculators/verify-reagents.test.ts recomputes every formula weight in the shipped reagent table from its formula with the module's CIAAW table, checks hydrates, PubChem identifiers and the Goldberg source of each buffer pKa, and checks the shipped file against its manifest. It runs in npm test.
Protein mass and pI
src/lib/seq/fixtures/protparam.json holds ExPASy ProtParam outputs for nine UniProt sequences (source: "ExPASy ProtParam ... whole UniProtKB sequence, retrieved 2026-10-05"), plus fragment pI values. There is no generating script and no hash: the values were recorded by hand. src/lib/seq/protein-mass.test.ts checks molecular weight within 0.02 Da, pI equal at two decimals and both ε280 values exactly. Calc's proteinMass method cites these as "the protein app's own fixtures".
Assay
Detailed method descriptions are in Assay. The oracle material is the most extensive in the suite and has two layers: the version-1 fixtures written by R on 6 October 2026, and later additions for version 2 of the fit.
Scripts and oracles
| Script | Role | Oracle or tool |
|---|---|---|
pipeline/assay-analyser/generate_curves.py |
Seeded generator: 33 synthetic curves with known parameters, the worked MTT plate and four Z′ control sets. Fits nothing | NumPy legacy RandomState (MT19937), seeds 20261007 to 20261037 (curve n uses 20261006 + n), C32 reuses C31's seed, C33 uses 20261308; refuses NumPy other than 2.x |
pipeline/assay-analyser/make-prism-example.mts |
Writes the guided walk's real plate readings (PRISM Repurposing 19Q4 HTS002, HCC827, three plates) as fixtures/prism-erlotinib-hcc827.json and .csv, and the app's copy src/components/tools/assay-analyser/examples/prism-erlotinib-hcc827.data.ts |
Copies by allow-list; no oracle |
pipeline/assay-analyser/reference.R |
The version-1 oracle: nls fits (unweighted, or 1/Y² by the fitted curve), root-found profile intervals, the flat-line F-test, rules (i) to (vi), Z′ and nested F-tests by hand; drc LL.4 and LL2.4, ED(type = "absolute"), anova() and MASS confint as cross-checks; Nelder–Mead and BFGS polishing to confirm each minimum; writes fixtures/manifest.json |
R 4.3.3, drc 4.0-0 (GPL-2; the script stops on any other drc), MASS 7.3-60.0.1, jsonlite 1.8.8 |
pipeline/assay-analyser/relative-irls-reference.py |
Version-2 oracle for fitted-response 1/Y² weighting: fixed-weight IRLS with SciPy least_squares on C24 and C25, writing fixtures/relative-irls-reference.json with the input's and its own SHA-256 |
SciPy 1.17.0 |
pipeline/assay-analyser/make-plate-fixtures.mts |
Synthetic plate exports (SoftMax Pro and Gen5 style, bare grids, UTF-16, BOM, decimal comma, long and wide tables), four kinetic read blocks (a table, SoftMax Pro- and Gen5-style blocks, a melt; 12 October 2026) and plates/expected.json |
A seeded generator (mulberry32); a parser round-trip, not an independent oracle |
pipeline/assay-analyser/make-xlsx-fixtures.py |
Five synthetic .xlsx workbooks (a 96- and a 384-well grid, a plate on a second sheet, a Gen5-like layout, one of every cell kind) and plates/xlsx/expected.json: every stored cell's type and text, every plate's wells as stored text (12 October 2026). Checked by src/lib/table/xlsx-oracle.test.ts |
openpyxl 3.1.5 writes the files; Python zipfile and xml.etree read them back, never the reader under test |
(removed) docs/reviews/2026-10-09-scientific-continuity/calibration-oracle.py |
Wrote the original src/lib/fit/fixtures/calibration-scipy.json (two cases), which stays write-once; the script moved to oracles/calibration_4pl_scipy.py (below) |
SciPy curve_fit (lm, three starting midpoints, lowest RSS) |
src/lib/assay/qpcr/oracle.py |
Prints two Pfaffl ratios evaluated with 60-digit decimal |
Python standard library; prints only, writes no file |
pipeline/assay-analyser/oracles/growth_gompertz.py, r/growth_gompertz_nls.R (A6) |
The modified Gompertz full fit on 9 synthetic growth courses, every status of the CSO's ruling; writes oracles/fixtures/growth-gompertz.json through stage_a_oracle.py |
SciPy 1.13.1 env; R 4.4.3 nls port cross-check |
pipeline/assay-analyser/oracles/qpcr_welch.py, r/qpcr_welch.R (A6) |
The qPCR fold-change Welch interval from synthetic Cq tables: 10 cases, 8 intervals, every withheld reason; writes oracles/fixtures/qpcr-welch.json through stage_a_oracle.py (refuses outside the pinned image) |
SciPy 1.13.1 env; R 4.4.3 t.test cross-check |
pipeline/assay-analyser/oracles/column_statistics.py, r/column_statistics.R (A-2 family 1) |
Column statistics: 50 cases (the CSO memo of 13 October 2026's twenty and its addendum's comma rule, read by the script independently of the grid), every number from base R (t.test, wilcox.test, shapiro.test, type-7 quantiles, the median interval by pbinom), the memo's withholding policy applied in the script; writes oracles/fixtures/column-statistics.json through stage_a_oracle.py. Recorded 13 October 2026 in image oracle-env:11b8a5e4456df7eb (run 38419861435; SciPy agrees on 179 of 179), published as 0.11.0 |
R 4.4.3 primary; SciPy 1.13.1 cross-checks (ttest_*, mannwhitneyu, wilcoxon, shapiro) |
pipeline/assay-analyser/oracles/column_outliers.py, r/column_outliers.R (A-2 family 7) |
Outlier candidates in columns: 21 cases (the CSO memo of 13 October 2026's eighteen, with grid cell states and an exclusion), G from outliers::grubbs.test, p from the Bonferroni bound gated by car::outlierTest (the package's folded two-sided p recorded as a divergence where 2·res > 1), studentised residuals from rstudent(lm(value ~ group)); writes column-outliers.json. Recorded 13 October 2026 in the pinned image (outliers 0.15, car 3.1-5), published as 0.11.0; cross-check 80 of 84 (car's unbounded n·p and its lm digits on the 1e9 offset, recorded) |
R 4.4.3 with outliers and car primary; SciPy stats.t and NumPy cross-checks |
pipeline/assay-analyser/oracles/power_sample_size.py, r/power_sample_size.R (A-2 family 7) |
Power and sample size: 26 cases (the CSO memo of 13 October 2026's twenty, with R's help-page examples), power from power.t.test, power.anova.test, power.prop.test (strict = FALSE; TRUE recorded), n by uniroot at tol = 1e-10 on R's own power function and the smallest integer reaching the target; writes power-sample-size.json. Recorded 13 October 2026 in the pinned image, published as 0.11.0; SciPy agrees on 19 of 19 |
R 4.4.3 stats primary; SciPy nct.sf, ncf.sf, norm cross-checks |
pipeline/assay-analyser/oracles/fdr.py, r/fdr.R (A-2 family 6) |
False discovery rate: 10 cases from the CSO memo of 13 October 2026 (2026-10-13-cso-fdr.md), every adjusted p and count from base R p.adjust (BH, BY, Holm, Bonferroni); the large cases generated by a formula the test repeats exactly; writes oracles/fixtures/family-6-fdr/fdr.json; committed at oracles/fixtures/fdr.json, where publish.py reads it. Recorded 13 October 2026 (run 38419868024), published as 0.11.0 |
R 4.4.3 |
pipeline/assay-analyser/oracles/groups.py, r/groups.R (A-2 family 2) |
Several groups: 22 cases from the CSO memo of 13 October 2026 (2026-10-13-cso-groups.md): one-way (Welch, classical), Tukey–Kramer, Games–Howell, Dunnett (R's nested integrate() of the product-correlation integral, glht under set.seed(20261013) as a cross-check), Holm, Šidák, Bonferroni, Kruskal–Wallis with Dunn by hand, two-way Type III and II (car::Anova), simple effects, repeated measures (aov Error strata, car::Anova with idata for GG and HF), the mixed model (nlme::lme REML), Friedman with pairwise signed-rank; Tukey and Games–Howell p from R's integrate() of the studentized range (ptukey is inaccurate far in the tail) with ptukey and TukeyHSD as cross-checks; writes oracles/fixtures/family-2-groups/groups.json; committed at oracles/fixtures/groups.json, where publish.py reads it. Recorded 13 October 2026 (run 38419873682; 18 R jobs), published as 0.11.0 |
R 4.4.3: stats, car 3.1-5, multcomp 1.4-32, mvtnorm, nlme 3.1-167 |
pipeline/assay-analyser/oracles/survival.py, r/survival.R (A-2 family 4) |
Time to event: 34 cases (the CSO memo of 13 October 2026's twenty-five, with survival::aml, survival::lung by sex and MASS::gehan pasted as text from a first R pass), every number from R's survival 3.8-3 (survfit with the log-log band, quantile.survfit, summary(times =, extend = TRUE), survdiff(rho = 0)), Gehan–Breslow–Wilcoxon by hand in R; the script refuses to write unless unit weights reproduce survdiff, the Gehan score equals n_A·n_B − 2W without censoring (the memo's sign corrected, its addendum) and the risk table equals sum(time >= t); coxph recorded beside the hazard ratio, never compared; writes oracles/fixtures/survival.json, published at 0.11.0 (Oracles run 38423667888, SHA-256 c9170aa6…; the 2 × 5000 case's table is built from a seed by the script and the test alike, and Kaplan–Meier rows are listed up to 40 per group). The engine agrees on all 34 cases (level-99, a band that is not monotone, after the CSO's ruling on SCI-100), and kaplanMeier, medianSurvival, survivalTests and hazardRatio are verified against it (13 October 2026); the identities inside R agree on 88 of 88 |
R 4.4.3 with survival and MASS; internal identities |
pipeline/assay-analyser/oracles/counts.py, r/counts.R (A-2 family 5) |
Counts and proportions: 42 cases (the CSO memo of 13 October 2026's eighteen, at and over the 10⁷-table Fisher bound counted exactly in the script), every number from base R (chisq.test with and without Yates, fisher.test(workspace = 2e8) and a transcription of its 2 × 2 internals at uniroot(tol = 1e-12), prop.test Wilson limits, binom.test, mcnemar.test, prop.trend.test; Woolf, Katz and Newcombe by hand with qnorm); writes oracles/fixtures/counts.json, published at 0.11.0 (Oracles run 38421922825, SHA-256 5aa45cbf…); the engine agrees on all 42 cases, and contingencyTests, twoByTwoMeasures, mcnemar and trendTest are verified against it (13 October 2026); the checks inside R agree on 37 of 37. The first run's large-count odds ratios were NA from R integer overflow, fixed in the script before this run |
R 4.4.3 base; Wilson by hand against prop.test; fisher.test against the tight transcription |
pipeline/assay-analyser/oracles/relationships.py, r/relationships.R (A-2 family 3) |
Relationships: 115 cases from the five CSO memos of 13 October 2026 (correlation, linear and multiple regression, nonlinear families, model comparison, Deming and Bland–Altman), every number from R (cor.test, lm, predict.lm, car::vif, nls port, anova, AIC, the Deming eigen route and its jackknife) or exact fractions (polynomials, the near-aliased and offset regressions), nonlinear profiles by SciPy's direct profile, the memos' withholding policy applied in the script; the Spearman interval's coverage simulation (48 cells, 20,000 replicates) recorded in about.coverage. A --dry-run <path> mode writes outside fixtures/ for the builder's check (115 of 115 cases agreed with the engines under R 4.3.3 and SciPy 1.18 on 13 October 2026, not a fixture). Recorded 13 October 2026 in the pinned image (run 38422173515; SciPy agrees on 105 of 105), published as 0.11.0 (SHA-256 99ff6967…); the engines agree on all 115 cases, and correlation, linearRegression, nonlinearFamilies, modelComparison, deming and blandAltman are verified against it |
R 4.4.3 primary; SciPy 1.13.1 cross-checks (pearsonr, spearmanr, linregress, least_squares, odr on centred data) |
pipeline/assay-analyser/oracles/nonlinear_coverage.py (A-2 family 3, SCI-90) |
Coverage of the nonlinear relationship fits' profile intervals (the CSO nonlinear memo, ruling 17): one design per family (growth, one-phase, two-phase, power, Boltzmann), 2,000 seeded replicates at 95 percent, SciPy estimates with the engine's withholding rules, coverage of every reported coordinate by the profile set's membership test at the true value, unresolved sides from full profiles on the first 100 replicates, Wilson bands; writes oracles/fixtures/nonlinear-coverage.json, whose about.cleared sets the 0.95 key of NONLINEAR_COVERAGE_CLEARED (nonlinear-coverage-oracle.test.ts); 90 and 99 percent are not simulated and stay uncleared (SCI-90, the CSO's Stage A exit memo of 13 October 2026, section 4). Its own script so relationships.py's published hash is untouched. Recorded 13 October 2026 in the pinned image (run 38426565308), published as 0.11.0 (SHA-256 90c21bc4…): all five families cleared, every one of the 2,000 replicates per family estimable, the lowest coverage 0.945 and no unresolved side |
SciPy 1.13.1 (least_squares, stats.f) in the pinned image |
pipeline/assay-analyser/oracles/calibration_4pl_scipy.py |
The standard-curve 4PL fixture, moved here from docs/reviews/ on 10 October 2026; reproduces the 9 October fixture to about 1e-10 and adds an unknown beyond each end of the standards |
SciPy 1.13.1 curve_fit (lm, three starts, lowest RSS) |
oracles/calibration_linear_exact.py |
Linear standard curves (Bradford- and BCA-style series, blank, excluded well, falling series, exact boundary, three refusals) | Exact rational OLS in Python fractions |
pipeline/assay-analyser/figure/tiff_expected.py |
The TIFF writer's expected structure (Stage A, A-0), written before src/lib/figure/tiff.ts: pixel sizes and strip plans at 150, 300 and 600 dpi for the MTT figure's millimetres, the baseline tags with their spec values in ascending order, the 100 MB refusal, and a 3 by 2 pixel case composed on white and kept transparent; writes fixtures/figure/tiff-expected.json. Checked by src/lib/figure/tiff.test.ts (its own IFD reader); figure/readback.py opens the writer's samples with Pillow and compares every pixel (run by hand, not in CI; Pillow 12.3.0 passed on 12 October 2026) |
Python standard library, from the TIFF 6.0 specification (Adobe, 1992) sections 2, 8 and 18 |
pipeline/assay-analyser/figure/pdf_expected.py |
The PDF writer's expected structure (Stage A, A-0), written before src/lib/figure/pdf.ts: the MediaBox in points, one page, the three standard fonts and their encodings, the Info keys, provenance.json, FlateDecode with the zlib header, every MTT text as a reader decodes it, the painted paths, and seven strings beyond the figure with their substitutions; writes fixtures/figure/pdf-expected.json. Checked by src/lib/figure/pdf.test.ts (its own xref and content reader, node:zlib to inflate); figure/readback.py reads the writer's MTT sample with pypdf (by hand; pypdf 6.17.0 passed on 12 October 2026, and Poppler's pdftoppm rendered it) |
Python standard library, from PDF 1.4 (Adobe, 2001) sections 3.3.3, 3.4, 3.10.3, 5.5.1 and Annex D, and RFC 1950 |
pipeline/assay-analyser/oracles/figure_summaries.py, oracles/r/figure_summaries.R |
The figure's chart-type summaries (Stage A, A-0, the figure editor's item 4; the CSO's chart-types memo of 13 October 2026), run in the oracle image before src/lib/figure/summaries.ts and published at 0.11.0: on six original sets (a far value, n = 4, n = 12, constant, zeros, negatives) the type 7 quartiles in exact rationals, the IQR, the whisker ends within 1.5 IQR and the values beyond, the mean and sample SD, R's bw.nrd0 with its fallbacks, the Gaussian density at 21 points over the data's range; and viridis at t = 0, 0.1 … 1 by matplotlib's ListedColormap rule from fixtures/figure/viridis-table.json; cross-checked with R's quantile(type = 7), bw.nrd0 and the density as the same exact sum with dnorm (1e-12 relative; the density 1e-10); writes oracles/fixtures/figure-summaries.json. Checked by src/lib/figure/summaries.test.ts |
Python standard library (fractions), R stats; Hyndman and Fan 1996 (type 7) |
oracles/replicate_summaries.py |
Replicate means and sample SDs on the MTT plate with and without A9 | Python fractions and decimal |
oracles/zprime_degenerate.R |
Six degenerate and adverse Z′ sets | R 4.3.3, by the definition |
oracles/growth_window.py |
Growth window: five exact whole-doubling cases, three noisy, two withheld, the hours-to-minutes pair and a log-before-blank contrast | Python fractions; SciPy linregress |
oracles/michaelis_menten.py |
Initial-rate Michaelis–Menten, eight cases | SciPy curve_fit (trf, the engine's box as bounds, log parameters, three starts, ftol = xtol = gtol = 1e-14) |
oracles/michaelis_menten_profile.py |
Profile bounds on Km and Vmax, nine cases, plus the coverage simulation | SciPy least_squares (trf) on the nuisance parameter, f.ppf, brentq |
oracles/ki_cheng_prusoff.py |
Ki from IC50, six values and five refusals, with the four-figure display | Python fractions |
oracles/thermal_shift.py |
13 synthetic melts with refusals and the disagreement flag | SciPy curve_fit; hand central differences |
oracles/inhibition_global.py |
18 inhibition cases: reductions, recoveries, adverse model choices, the noisy matrix by each model, metamorphic rows, refusals | SciPy least_squares (trf, nine starts) |
oracles/lineage.py |
Not an oracle: re-records an oracle fixture as the next version of its lineage in the oracle image under the CPU-class pin (CTO rulings and the CSO's lineage-tolerance and ill-conditioned-case rulings of 12 October 2026). It refuses to write outside the image (an @sha256: image, ORACLE_ENV_JSON, run-oracle's ORACLE_CPU_JSON); runs the base script in a temporary copy after checking its SHA-256; runs the R cross-check with R's OpenBLAS checked against the pin; compares the result with the predecessor field by field (1e-8 × max(|old|, scale) in the span, one, conc and rss classes; the profile threshold 1e-10 relative; the rest exact; for the two cases the CSO names, adverse-post-decline and unsaturated-noisy, the named fields within 1e-6 and the case's RSS within 1e-12 relative) and stops on anything beyond; records the classes, the largest difference per class and the ill-conditioned entries in about.basedOn; writes <fixture>-v2.json and prints it into the job log |
Standard library; NumPy and SciPy versions read from the running environment |
oracles/inhibition_global_v2.py, thermal_shift_v2.py, michaelis_menten_profile_v2.py, calibration_4pl_scipy_v2.py, michaelis_menten_v2.py |
The wrappers: each calls lineage.main for its lineage. The first three add the R cross-checks; the last two re-record under the pin only, so every drift baseline comes from the drift environment (their engine tests and badges stay on 0.4.0) |
As lineage.py |
oracles/inhibition_global_v2_base.py |
inhibition_global.py with line 210 alone changed to the CSO's relative assertion at 1e-8 (the 1e-9 metamorphic check fails under the pin); lineage.py refuses any other difference |
SciPy, as inhibition_global.py |
oracles/r/inhibition_nls.R, r/michaelis_menten_confint.R, r/thermal_ssfpl.R |
The R cross-checks the CSO named: nls on every fitted inhibition case; nls on the Michaelis–Menten profile cases, then (CSO ruling of 12 October 2026, docs/board/memos/2026-10-12-cso-michaelis-menten-profile-r-cross-check.md) the likelihood-ratio profile computed directly from R's own fit: the threshold RSSmin (1 + F(level; 1, df)/df), the Km profile with Vmax by closed-form least squares and the Vmax profile by optimize over ln Km inside the fixture's box, each crossing by uniroot (tol 1e-10) bracketed by R's own outward search; agreement is the threshold within 1e-8 relative and every bound SciPy resolves within 1e-3 ln, and any miss stops the run. confint() is kept and recorded beside it with r = (RSS − RSSmin)/(threshold − RSSmin) from R's profile at each of its bounds, but does not decide agreement: its spline interpolation falls inside the region on the curved lower sides of the two unsaturated cases (r 0.71 and 0.04) and stops at its iteration cap on the nM/min case; nls with SSfpl on every fitted melt (A = F_lo, B = F_hi, xmid = Tm, scal = k = exp(lnK); a negative scal is converted). All start from the SciPy estimates with scaleOffset = 1; refusals and bound contacts are SciPy-only and listed; a non-convergence is recorded with the case |
R 4.4.3 and MASS in the oracle image |
oracles/curve_compare.py, r/curve_compare_drc.R |
A4's curve comparison: separate, constrained and shared-midpoint global 4PL fits over two or three curves, the midpoint F and each named constraint's own F, δ = log10 C50_B − log10 C50_A profiled (0.02, 0.04, … log10 outward, brentq) with Wald beside it, the ratio withheld on an unbounded or poorly determined own interval, a reduced model at its search limit withholding its own F (CSO ruling of 12 October 2026), on eleven synthetic cases and the worked MTT plate's Compounds A and B; a coverage simulation of δ's profile and Wald intervals (2,000 seeded replicates per design) and the midpoint test's size. The R cross-check: drc drm with curveid and pmodels (LL2.4), anova(), compParm(, "e", "-") and EDcomp (delta and Fieller, recorded as information) |
SciPy least_squares (trf, ftol = xtol = gtol = 1e-14), f.sf, f.ppf, t.ppf, a central-difference Jacobian; R 4.4.3 with drc in the oracle image |
oracles/screening.py (A-1, the screening campaign) |
The CSO's cases a to k (memo of 13 October 2026): known controls with percent inhibition 0, 50 and 100, inverted controls, exactly 16 and 15 sample wells, odd and even n, MAD = 0, a one-well control and equal control means, values at the threshold and 1e-12 below it, excluded, blank, text and NaN wells, a 1,536-well text with AA1 and AF48, three plates with a ranking tie, one withheld and one with Z′ < 0.5 still called. Writes oracles/fixtures/screening.json through stage_a_oracle.py; --local PATH writes a builder's check outside the image, never a fixture. Recorded 13 October 2026 in the oracle image (run 38419872312), published as 0.11.0 |
NumPy median, scipy.stats.median_abs_deviation (raw and scale="normal"); percent inhibition, Z′, median and MAD to 1e-9 relative, z to 1e-12 |
oracles/curve_compare_v2.py |
Re-records curve-compare.json as curve-compare-v2.json with the drc cross-check's two agreement rules stated: p compared at 1e-4 where the SciPy p exceeds 1e-12 (drc's anova is 1 − pf, at its precision limit below that), the Wald SE at 1e-2 (drc's Hessian against the engine's JᵀJ). Imports curve_compare.py unchanged (the 0.10.0 fixture pins its SHA-256 and relative_potency.py records it as its basis) and replaces only agree(); every case and number is the base script's |
As curve_compare.py |
oracles/negative_result.py |
A-4 item 12's negative result: the permutation rank-trend p (exact as fractions, the seeded Monte Carlo stream ported, and a band against exact enumeration), the feasibility p (y = x), the eligibility verdict, the range, the end means and the experiments clause, on 16 cases including the CSO's addendum refusals. Written 13 October 2026; the fixture waits on the Oracles workflow (Actions stopped by the budget) | Python standard library |
oracles/experiment_summary.py, r/experiment_summary_ttest.R |
A4's experiment-level summary of log10 C50: eleven cases (three and five experiments, k of n with a withheld member, the pair, the single fit, none, the carried poorly-determined flag, mixed kinds refused, identical values, the units shift, absolute C50s). The R cross-check: t.test one-sample and sd on every case with an interval (it refuses constant data, recorded) |
Python fractions and decimal (50 digits), scipy.stats.t.ppf; R 4.4.3 in the oracle image |
oracles/stage_a_oracle.py |
Not an oracle: the Stage A scripts' shared plumbing (CTO ruling 11(a) of 12 October 2026). write_fixture refuses to write outside the oracle image under the CPU-class pin (as lineage.py does), stamps about.recordedOn, script, scriptSha256, helper (its own path and SHA-256) and about.environment, and prints the fixture (gzip, base64, SHA-256) and its run record into the job log; run_r runs an R cross-check with R's OpenBLAS checked against the pin. It repeats lineage.py's helpers because lineage.py's hash is recorded in the published 0.5.0 fixtures |
Standard library, NumPy and SciPy versions read from the running environment |
oracles/calibration_5pl_scipy.py, r/calibration_ll5.R |
The 5PL standard curves (A5; CSO memo of 11 October 2026, item 3): three fitted curves (asymmetric rising and falling, symmetric noiseless) and two refusals (s at its box, six standards), each fitted curve with the 4PL of the same means, the F of 4PL within 5PL and AICc as information, interpolated and out-of-range unknowns; drc LL.5 from the SciPy estimates as the cross-check |
SciPy curve_fit (trf, ln s boxed to [ln 0.1, ln 10], nine starts); R 4.4.3 drc 4.0-0 in the image |
oracles/outliers_scipy.py, r/outlier_checks.R |
Outlier candidates (A5; CSO memo of 11 October 2026, item 7): externally studentised residuals with the Bonferroni threshold on straight-line and 4PL fits (the 4PL hat matrix from a central-difference Jacobian), Grubbs within control groups, and the withheld states (n − p below six, zero residual, a group below six, no spread) | NumPy, SciPy curve_fit and stats.t; R 4.4.3 rstudent, hatvalues, car outlierTest and outliers grubbs.test in the image as the cross-check |
oracles/calibration_inverse_scipy.py, r/calibration_invest.R |
Inverse-prediction intervals on calibration unknowns (A5; CSO memo of 11 October 2026, item 4): the linear Fieller interval, the 4PL and 5PL delta method, and every withheld state; investr invest as the cross-check on unknowns read once (it pools an unknown's replicate scatter for m > 1, a different estimator, so those are recorded as not compared). Superseded from Assay 0.13.0 by calibration_inverse_v2.py (below); its fixture stays the record for 0.6.0 to 0.12.0 |
Python fractions; SciPy curve_fit; R 4.4.3 investr 1.4.2 in the image |
oracles/calibration_inverse_v2.py, r/calibration_invest_v2.R |
Supersedes calibration_inverse_scipy.py from Assay 0.13.0 (SCI-123; the CSO's Stage A exit memo of 13 October 2026): the first oracle, like the engine, took s² from the standard means, which estimates σ²/r with standards read r times, so intervals were too narrow whenever standards were replicated. Version 2 pools s² over the individual readings on N − p degrees of freedom, takes the curve term as (s²/r)(JᵀJ)⁻¹ (r/m in the linear Fieller radicand) and withholds unequal replicate counts (unequal-replicates); the standard-count rule stays on concentration levels. Fifteen cases: the first oracle's six recomputed (its linear case has duplicate standards), linear standards read once (asserted equal to the first method), triplicate linear, duplicate and triplicate 4PL and 5PL, four linear levels in duplicate (standard-count) and two with one standard read an extra time; each unknown also records the first method's bounds (uncorrected), which document the defect and are not a target. investr invest on lm or nls fitted to the individual readings (not the means) for unknowns read once, comparing R's σ² and residual degrees of freedom too. A coverage simulation: 2,000 seeded replicates for each of linear and 4PL × r = 2, 3 × m = 1, 3, gates 0.93 (linear) and 0.90 (4PL), the CSO's judgements; the script exits non-zero before writing if a gate fails. Writes oracles/fixtures/calibration-inverse-v2.json through stage_a_oracle.py, with about.basedOn naming the first fixture; --local [PATH] checks it outside the image (no fixture; R only if present) |
Python fractions; SciPy curve_fit; R 4.4.3 investr 1.4.2 in the image |
oracles/two_phase_dose_response.py, r/two_phase_drm.R |
Bell-shaped and biphasic curves (A-1 second pass; CSO memo of 13 October 2026): two reported cases, a noiseless bell (parameters recovered, the comparison withheld at the exact fit), the bend, separation and minor-phase withholdings, and the two refusals, the rule written independently from the memo; the R cross-check is drc drm with a user-defined fct for each form from the SciPy estimates, and records whether drc exports twophase() with its source for the CSO. Checked by src/lib/fit/two-phase-oracle.test.ts (the engine's RSS at most the oracle's × (1 + 1e-6), parameters within 1e-4, states exact). Not yet published: waiting on the Oracles workflow; the test is a named todo until then |
SciPy least_squares (trf; the CSO's grid of starts plus 300 seeded uniform starts), curve_fit (lm) for the 4PL, f.sf; R 4.4.3 drc in the image |
oracles/agonist_antagonist.py, r/agonist_antagonist.R |
Agonist and antagonist analyses (A-1 second pass; CSO memo of 13 October 2026): competitive (pKB reported), a slope of 0.6 (pKB withheld), receptor removal under the operational model (insurmountable; the Gaddum KB within 1e-3 of the truth), too few levels, a span under tenfold, two controls, a Furchgott set (reported) and one whose maxima do not differ; profiles by refitting on an outward grid and brentq. The R cross-check: drc drm(LL.4, curveid) for the own C50s, nls (port) for the global Gaddum/Schild and operational fits, lm on the classic Schild plot (recorded only). Checked by src/lib/fit/antagonism-oracle.test.ts (estimates within 1e-4, profile bounds within 1e-3, states exact). Published at 0.11.0. For SCI-121, agonist_antagonist_v2.py and r/agonist_antagonist_v2.R re-record the same cases as agonist-antagonist-v2.json, published at 0.13.0 (the original script and fixture stay as recorded). The v2 run also compares every fit R and SciPy both make on one objective, the residual sum of squares in float64, at SciPy's estimate, R's, SciPy refitted from R's and R refitted from SciPy's (nls port for each own 4PL), and records which is lowest (crossCheck.minima; SciPy holds it within 1e-9 relative or 1e-12 absolute). The first run found the R port's control rows carried log[B] = NA, which nls's default na.action dropped, so R's global fits had left out the control curve; with the control carried as lb = 0, has = 0 R agrees on 6 of 7 fitted cases (pA2, s and pKB within 2e-9), and the seventh (Furchgott, full maxima, own log EC50 by 1.9e-4) is drc stopping at a higher residual sum of squares. SciPy holds the lowest on every fit (13 October 2026) |
SciPy least_squares (trf) and curve_fit (lm), f.sf, f.ppf, brentq; R 4.4.3 drc in the image |
oracles/gr_metrics.py |
GR metrics (A-1 second pass; CSO memo of 13 October 2026), from the published definition, not a port: thirteen cases from counts generated by the GR model with a fixed perturbation (cytostatic, cytotoxic at hGR 1 and 2, partial, flat, few doublings, three no-growth refusals, a reading below the blank, a separate time-zero plate, a GEC50 near the top of the range, a step steeper than the hGR box); the CSO's hand cases (per-well GR, GR50 from GRinf, GR_AOC 0.75) asserted in the script; Spearman stands in for the engine's permutation gate only in the flat case. No R: the GRmetrics package is not in the image (SCI-120). Checked by src/lib/growth/gr-oracle.test.ts (arithmetic within 1e-9, fitted values within 1e-6, states exact). Not yet published: waiting on the Oracles workflow; a dry run agreed on 13 of 13 cases (13 October 2026) |
SciPy curve_fit (trf with the bounds), spearmanr |
pipeline/assay-analyser/oracles/combination_hsa_bliss.py |
Combination analysis: Highest Single Agent (HSA) and Bliss Independence models (Contract 13; CSO and CTO memos of 17 October 2026). Standard library only (fractions, json, hashlib, pathlib, platform, datetime). 31 cases covering all 13 CSO mandatory cases: hand anchors asserted before writing, empty-margin combinations withheld with margin named, vehicle missing refused (missing-vehicle-control), non-zero vehicle disclosed unconditionally, multiple biological replicates refused, Bliss undeclared withheld, out-of-range Bliss operand withheld, raw path controls and normalisations, metamorphic transposes (A/B swap, direction and scale equivalence); every expected value computed in exact fractions; writes oracles/fixtures/combination-analysis-v2.json, published as 0.14.0. The planned subprocess cross-check with synergy==1.0.0 (combination_synergy_crosscheck.py) was never run: both fixtures record about.crossCheck.ran: false (SYNERGY_PYTHON unset), and the script computes min and the product with its own arithmetic rather than calling the package's HSA and Bliss models, so running it would not cross-check the package (audit of 20 October 2026, combination C-1). The fixtures' about.method names the contract's proposed ids combination-hsa-observed and combination-bliss-observed; the engine and the registry pin combination-hsa and combination-bliss (the CSO's ruling of 20 October 2026); the published bytes stay as they are |
Python standard library (fractions); synergy==1.0.0 cross-check adapter |
pipeline/assay-analyser/oracles/combination_hsa_bliss_3.py |
Combination analysis: Highest Single Agent (HSA) and Bliss Independence models v3 (Contract 13; CSO and CTO memos of 17-18 October 2026). Adds CSO cases v2 lacked: non-finite response refused at import, negative dose, NaN dose, underflow dose, duplicate axis level as conflict, off-axis observation, maximum 4,096 rows and 4,097 rows beyond. Writes oracles/fixtures/combination-analysis-v3.json, published as 0.15.0 |
Python standard library (fractions); synergy==1.0.0 cross-check adapter |
pipeline/assay-analyser/oracles/degenerate_reference.py |
Degenerate spread and separation across fourteen Assay engines (the CSO's audit-fix memo of 20 October 2026, item 1; audit findings C2 to C5, C8, M5 and M6). Inputs are decimal strings; whether a case refuses is decided in exact decimal arithmetic (fractions.Fraction(str)); the cases that keep calculating carry numpy and scipy numbers at the existing sets' tolerances. 60 cases: the memo's sixteen, split by engine and control set, plus one guard (a clearly falling slow-binding set stays falling). R was not available where it was recorded: the log-rank cases are exact fractions (χ² 5 on disjoint risk sets) and the two-way case numpy least squares with an exact-fraction cross-check, where the memo names R; the fixture's about.crossCheck says so for the CSO. Writes oracles/fixtures/degenerate-v1.json, published as 0.16.0 |
numpy 2.5.4, scipy 1.18.1, Python 3.13.16 (fractions, decimal at 60 digits for the thermal melt's generating curve) |
oracles/publish.py <version> |
Writes oracles/fixtures/manifest.json (each fixture's SHA-256, script and script SHA-256, date, method) and copies every fixture, with its manifest entry, to public/verification/assay-analyser/<version>/; refuses to replace a published file or entry |
Standard library |
walk_plate_reference.R, walk/publish.py |
The plate walk's refit row (Compound A without A9) from reference.R's own definitions, with Z′ and a Monte Carlo permutation p (100,000 permutations, set.seed(20261010)); published under 0.3.0 |
R 4.3.3, drc 4.0-0, MASS 7.3-60.0.1 |
Pins. pipeline/assay-analyser/requirements.txt: numpy==2.4.6 for the generator. pipeline/assay-analyser/requirements-irls.txt: scipy==1.17.0, numpy==2.3.5 (Python 3.12.14 recorded). pipeline/assay-analyser/requirements-scipy.txt: scipy==1.13.1, numpy==2.0.2, Python 3.11.15 named in its header, for every script in oracles/. The three NumPy pins differ, so each needs its own environment. The Wave 1 continuation's three fixtures were run under Python 3.11.17 (what the container's uv supplied), not the 3.11.15 the pin names. R and its packages are installed from Ubuntu 24.04 packages and CRAN; the README records the exact versions. From 12 October 2026 the oracle image (pipeline/oracle-env/) installs all three files in separate environments under CPython 3.11.15, with R 4.4.3 on a dated CRAN snapshot; see The oracle environment. The qPCR script has no requirements file.
Mapping. The engine's 4PL is y = Bottom + (Top − Bottom) / (1 + 10^((logIC50 − x)·Hill)), x = log10(concentration in M). drc's LL.4 maps by e = IC50 = 10^logIC50, b = −Hill, c = Bottom, d = Top; LL2.4 uses e = ln(IC50). src/lib/fit/fixtures.test.ts asserts the mapping on every curve that has one.
Fixtures
All in pipeline/assay-analyser/fixtures/, read in place by the tests; nothing is copied under src/ except the generated example module and the calibration fixture.
| File | Content |
|---|---|
synthetic-curves.json |
33 curves: every well, the true parameters (in the brief's and drc's parameterisations), design, noise model, seed and fit settings |
mtt-example.json, mtt-example.csv |
One 96-well A570 plate with plate map and generative model |
zprime-controls.json |
Four control-well sets (Z04 has one negative well and must be refused) |
drc-reference.json |
Per curve and MTT compound: the expected block, the oracle fits and cross-checks, ruleChecks, minimumCheck; Z′ for the plate and the four sets; six F-tests; tool versions, script and input SHA-256s, tolerances and date |
prism-erlotinib-hcc827.json, .csv, .reference.json |
The walk's published plate readings and R's normalisation, Z′ per plate and fit |
relative-irls-reference.json |
IRLS parameters, iteration count and relative RSS for C24 and C25 |
manifest.json |
SHA-256 of eight files and of reference.R (referenceRSha256) |
The oracle fixtures written by oracles/ live in pipeline/assay-analyser/oracles/fixtures/ with their own manifest.json (version 0.5.0): calibration-4pl-scipy.json, calibration-linear-exact.json, replicate-summaries.json, zprime-degenerate.json, growth-window.json, michaelis-menten.json, ki-cheng-prusoff.json (all first published under 0.2.0), michaelis-menten-profile.json, thermal-shift.json and inhibition-global.json (0.4.0). Each fixture's about block records its script and the script's SHA-256, the date, the method and, for the last three, crossCheck: the R cross-checks named by the CSO (confint, nls(SSfpl), nls) were not run because R was not installed in the build container. The oracle image now carries R; the three stay as published, and their re-run is a new version (The oracle environment; Known issues, SCI-35). Version 0.5.0 (12 October 2026) adds five -v2 fixtures recorded by lineage.py in the image under the CPU-class pin (Oracles runs 38299359746, 38299362365, 38299366209, 38299369733 and 38299373320 at 9cb3909, image oracle-env:11b8a5e4456df7eb@sha256:7e69891c…): inhibition-global-v2.json (R nls agrees on 11 of 11 fitted cases), thermal-shift-v2.json (R nls(SSfpl) 7 of 7), michaelis-menten-profile-v2.json (R's direct profile on 7 of 7 cases with an interval; R confint 4 of 7, recorded; the zero-residual case has none), calibration-4pl-scipy-v2.json and michaelis-menten-v2.json (the pin only). Each records about.environment, about.crossCheck where R runs and about.basedOn with the 0.4.0 predecessor's hash and the measured agreement in the CSO's tolerance classes; src/lib/fit/oracle-lineage.test.ts recomputes it. The 0.4.0 files and rows are unchanged; the engine tests and badges move to the -v2 files with the registry change. The plate walk's row is pipeline/assay-analyser/walk/plate-walk-refit.json with its own manifest.
Version 0.10.0 (12 and 13 October 2026, Stage A, A4; recorded as 0.6.0 on claude/assay-a4 and renumbered on 13 October by the coordinator's allocation, the bytes unchanged) adds two fixtures recorded in the oracle image under the CPU-class pin (image oracle-env:11b8a5e4456df7eb@sha256:7e69891c…, runs from claude/assay-a4): experiment-summary.json (Oracles run 38323805939; R t.test agrees on the 6 cases with an interval within 1e-9 log10, and refuses the constant case, recorded) and curve-compare.json (Oracles run 38324572552 at 0dd2df76; R drc 4.0-0 agrees on RSS and F to about 1e-15 and on the C50 difference to 4e-8 on the 11 compared cases; the fixture's crossCheck reads "differs" on two definitions, p below 1e-12, where drc's anova reports 1 − pf, and compParm's Wald SE, from drc's Hessian where the engine uses JᵀJ, up to 0.7 percent apart. A re-recording with those two comparison rules stated could not start on 13 October 2026: GitHub's Actions budget stopped the image job, runs 38327779963 and 38328924439). Each was taken from the job log, where the script prints it gzipped and base64-encoded with its SHA-256, because the artifact host is not reachable from every container.
Version 0.11.0 (13 October 2026, A4) adds curve-compare-v2.json from curve_compare_v2.py (Oracles run 38420188008 at 36db7c2e, the same image under the CPU-class pin, AMD EPYC 9V45; SHA-256 1a47fe75…09b, recovered from the job log by the Assay coordinator): its cases and coverage are byte-for-byte those of curve-compare.json, and its crossCheck reads "every RSS, F, p and delta within tolerance; the Wald SE within 1e-2", agrees: R drc 4.0-0 on RSS 3.0e-15, F 7.0e-15, p 2.4e-7 (where above 1e-12), δ 3.6e-8 and the Wald SE 0.0071 relative. The engine test (src/lib/fit/global.test.ts) reads it.
Version 0.16.0 (20 October 2026; the CSO's audit-fix memo of that date, item 1 (c)) adds degenerate-v1.json from degenerate_reference.py, recorded outside the oracle image with numpy 2.5.4 and scipy 1.18.1 on Python 3.13.16 (no CPU-class pin; every refusal is exact, and the numbers are closed forms, least squares or distribution functions, not iterative fits). The other 66 fixtures are byte copies of 0.15.0. Its cases name the memo case they come from (memoCase), the engine (engine), the inputs as decimal strings, the exact quantity that decided the case (decision, a fraction written as a string) and the expected status, code and values. The engine test is src/lib/stats/degenerate-oracle.test.ts.
Three curves have no finite least-squares minimum (C30, C33 and MTT Compound C). For them the fixture gates only the outcome, the absence of estimates and that the engine's objective is not below objectiveInfimum.
Tolerances and comparison rules
From the CSO's rulings of 6 October 2026, recorded in drc-reference.json about.tolerances and in TOLERANCE in src/lib/fit/fixtures.test.ts:
| Quantity | Tolerance |
|---|---|
| logIC50 (and shown relative and absolute IC50) | 0.00043 log10 units, absolute |
| Hill | 0.1 percent relative |
| Top, Bottom | 0.1 percent of |Top − Bottom| |
| Objective | 1e-6 relative; an engine objective below the oracle's fails, meaning the oracle must be rerun |
| Wald bounds | 0.0022 log10 units; residual df exact; t quantile 1e-9 relative |
| Profile bounds | 0.0043 log10 units on every finite bound; unbounded flags exact (with an exception for the engine's refit-unresolved upper status) |
| Flat-line F | 1e-6 relative; p 1e-4 relative where p > 1e-12; df exact |
| Rule counts (iii), (iv), (v), (vi) | Exact; bend points 0.001 log10 units |
| Z′ and SDs | 1e-9 relative; refusal below two wells exact |
| Nested F-tests | RSS 1e-6 relative, F 1e-5 relative, p 1e-4 relative |
| MTT normalisation | Means 1e-12 relative; per-well percent viability 1e-9 relative |
IRLS (src/lib/fit/irls.test.ts) |
Each parameter within 1e-5 × max(1, |value|); the frozen-weight score equations sum to within 2e-5 of the sum of their absolute terms |
Calibration (src/lib/fit/calibration-oracle.test.ts) |
Parameters and fitted standards 2e-5 absolute; interpolated concentrations max(1e-6, 5e-5 relative) |
Linear calibration, replicates, degenerate Z′ (src/lib/fit/oracle-fixtures.test.ts) |
1e-9 relative; range flags, n, wells and refusals exact |
Growth window (src/lib/growth/window-oracle.test.ts) |
μ, doubling time and residuals 1e-9; withheld states, rows and n exact; hours and minutes 1e-12 |
Michaelis–Menten (src/lib/kinetics/michaelis-menten-oracle.test.ts) |
Km, Vmax, kcat 1e-5 relative (hand values 1e-9); RSS 1e-6 relative or both below 1e-20·Σv²; correlation 1e-4 |
Profile bounds (michaelis-menten-profile-oracle.test.ts) |
Resolved bounds 1e-3 ln units; threshold 1e-8 relative; states, reasons and flag exact; kcat bounds 1e-3 relative |
Inhibition (inhibition-oracle.test.ts) |
Constants 1e-5 relative (hand 1e-9); RSS 1e-6; correlations 1e-4; Ki′/Ki 1e-5 |
Ki (ki-oracle.test.ts) |
1e-12 relative |
Thermal shift (src/lib/thermal/apparent-tm-oracle.test.ts) |
Tm 1e-3 °C; ln k 1e-4; plateaus 1e-4 of the span; RSS 1e-6; derivative values 1e-9 relative |
Plate walk refit (src/lib/fit/fixtures.test.ts) |
The estimate and interval tolerances above; Z′ unchanged; the gate's decision and p floor |
How version 2 reads the version-1 fixtures, from the comments in src/lib/fit/fixtures.test.ts:
- Weighted curves (C24, C25) are fitted with
historicalRelativeLoss: true, the archived direct-loss estimator, and compared on logIC50 and the objective only. The production estimator is IRLS, checked bysrc/lib/fit/irls.test.tsagainst the SciPy fixture. Minimising Σ((y − f)/f)² and IRLS differ on C24 by 0.00067 log10 units, outside the 0.00043 tolerance (README finding 2), which is why the two are kept apart. - Reporting outcomes. Version 2 replaced the flat-line F reporting gate with a within-plate rank permutation test. The test no longer asserts the fixture's
shownandwithheldByoutcomes for fitted curves; it asserts the counts behind the rules and, where both show a value, the value. The refusal of C29 (three concentrations) is still asserted. - F-tests. Of the six F-tests in the fixture, the four with a fixed parameter are recomputed from the engine's own refits; the two flat-line comparisons are skipped there (
fixtures.test.ts:357) and covered per curve.
Other assay consumers: src/components/tools/assay-analyser/assay.test.ts (MTT grid through the app's layout), src/components/tools/assay-analyser/published-example.test.ts and examples/prism-example.test.ts (the walk's plate), src/lib/table/table.test.ts (plate files), e2e/objects/j3.spec.ts (the walk in a browser against prism-erlotinib-hcc827.reference.json) and scripts/prebuild/suite-walk-values.mjs, which writes content/walks/j3.values.ts from the fixtures before each build. src/lib/assay/qpcr/qpcr.test.ts hard-codes the two Decimal values oracle.py prints.
Regenerating
From pipeline/assay-analyser/README.md and the manifest's use text:
apt-get install -y r-base-core r-cran-car r-cran-gtools r-cran-mass r-cran-multcomp r-cran-plotrix r-cran-scales r-cran-jsonlite
Rscript -e 'install.packages("drc", repos = "https://cloud.r-project.org")'
python3 -m venv /tmp/assay-oracle && /tmp/assay-oracle/bin/pip install -r pipeline/assay-analyser/requirements.txt
/tmp/assay-oracle/bin/python pipeline/assay-analyser/generate_curves.py
node pipeline/assay-analyser/make-prism-example.mts
Rscript pipeline/assay-analyser/reference.R
The Wave 1 oracles, from requirements-scipy.txt and publish.py: uv venv --python 3.11 /tmp/oracle && VIRTUAL_ENV=/tmp/oracle uv pip install -r pipeline/assay-analyser/requirements-scipy.txt, run each script in oracles/, then python pipeline/assay-analyser/oracles/publish.py <version> into a new version directory (write-once refuses changed bytes in an existing one).
The IRLS oracle, from docs/science/2026-10-scientific-audit-implementation.md: python -m pip install -r pipeline/assay-analyser/requirements-irls.txt then python pipeline/assay-analyser/relative-irls-reference.py. The README predates the IRLS oracle, the PRISM fixture and make-plate-fixtures.mts and does not describe them. reference.R stops if drc is not 4.0-0, if polishing finds a point below a no-finite-minimum limit, or if the rules would decide differently along such a valley.
Open items the README records: the Hafner GR-metrics supplement (CC0) is not committed and adding it is the CEO's call; a Prism fit of C24 has not been done; Sebaugh and McCray 2003 (the bend-point constant k = 4.6805) has not been read, the constant being corroborated by arithmetic only; the independent reviewer sign-off (Q-19) is pending.
Bioreactor
Model and solver details are in Bioreactor.
Script. pipeline/bioreactor-simulator/reference.py (1,330 lines), sub-commands scenarios, analytic, transfer, scaleup, step, manifest and all. The script restates the model's equations in its docstring and implements them independently of src/lib/hosts.
| Sub-command | Writes | Oracle |
|---|---|---|
scenarios |
scenarios.json: 33 runs (26 test scenarios and 7 published preset runs), each solved with dissolved oxygen dynamic and quasi-steady, with carbon and electron closure and event times |
scipy.integrate.solve_ivp, Radau, rtol 1e-9, per-state atol, piecewise between time events, state events located by the event finder, sampled every half hour from dense output |
analytic |
analytic.json: chemostat steady states and washout (Pirt as formulated here), Haldane steady states, batch Monod growth by its implicit solution inverted by root-finding |
Closed forms worked by hand, root-finding in SciPy |
transfer |
transfer.json: air saturation (Benson and Krause, USGS OWQ memo 2011.03), Sander's constant as cross-check, exhaust-gas balance, Van 't Riet coalescing kLa (as quoted by Yawalkar 2002) with Hughmark gassed power at ten points, two outside range |
SciPy and NumPy arithmetic |
scaleup |
scaleup.json: three scale-up rules |
Worked by hand in the script |
step |
step.json: one Rosenbrock (Shampine and Reichelt) step on a 3×3 system and at a stiff model state; LU solves that swap rows after the first column; a singular matrix; the right-hand side at 13 stated states |
NumPy and numpy.linalg.solve (LAPACK) |
manifest |
manifest.json: SHA-256 of each file and of the script, with versions |
fixtures/usgs-dotables-2026-10-06.csv is the USGS DOTABLES program's own output, fetched on 6 October 2026, not written by the script. fixtures/preset-specs.json is written first by pipeline/bioreactor-simulator/export-presets.mts from src/lib/hosts/preset-specs.ts: inputs only, so each published preset run is also an oracle scenario.
Pins. pipeline/bioreactor-simulator/requirements.txt: scipy==1.17.0, numpy==2.3.5. The manifest records Python 3.12.14. The script records versions but has no version guard; the test asserts manifest.versions.scipy === "1.17.0".
Consumers and tolerances.
| Test | Compares | Tolerance |
|---|---|---|
src/lib/hosts/fixtures.test.ts, per scenario |
Every output series at the oracle's output times | Within 0.5 percent of the series' peak; balances within 0.5 percent; growth switches, trigger intervals, oxygen-out and DO-below-1-percent within 1e-3 h; resolved feed 1e-5 relative; S and C never below minus their atol; the shipped rtol 1e-7; the oracle's rtol and atol at least 100 times tighter; step counts within 0.5 to 1.25 times the recorded count |
src/lib/hosts/fixtures.test.ts, accuracy check |
Every point (agreement.test-util.ts) |
The larger of 1e-5 of the value, a time shift of max(1e-4 h, 1e-5 of elapsed time) times the slope, and a floor; no time shift for oxygen-driven series inside a time event's window |
src/lib/hosts/fixtures.test.ts, quasi-steady |
Quasi-steady engine runs against the oracle's dynamic runs | 2 percent of peak outside the oxygen trigger, oxygen-out within 0.01 h |
src/lib/hosts/fixtures.test.ts, second route |
Dormand–Prince (src/lib/fit/dormand-prince.test-util.ts, test-only) through the same equations |
As stated in the test |
src/lib/hosts/fixtures.test.ts, closed forms |
Chemostat steady states, observed yield, washout either side of the closed-form D, batch Monod | 1e-3 relative (0.1 percent) |
src/lib/hosts/fixtures.test.ts, transfer |
Benson and Krause, exhaust gas, kLa and its range flags | 1e-9 relative; DOTABLES to its printed rounding (0.005); Sander 1e-6 |
src/lib/hosts/fixtures.test.ts, scale-up |
Large vessel and rpm per rule | 1e-9 relative, 1e-6 for the constant-kLa rule |
src/lib/fit/ode.test.ts |
LU on matrices that pivot after column one; one step | Backward error 1e-12·‖A‖‖x‖; forward error max(1e-12, 8ε·cond); one step 1e-13 |
src/lib/hosts/model.test.ts |
Right-hand side at 13 states in all regimes; Jacobian at the stiff state against the oracle's central differences | 1e-12 relative |
The same test file asserts that FIXTURE_COUNTS and the version constants in content/apps/bioreactor-simulator.verification.ts equal the fixture files' counts and the manifest's versions, and that preset-specs.json is what the presets give now.
Golden runs are not oracles. pipeline/bioreactor-simulator/write-golden.mts writes src/lib/hosts/golden/<id>.json from the engine itself, once, with a manifest of SHA-256s; --freeze marks them published. src/lib/hosts/golden.test.ts re-runs the engine against them at the golden tolerances in content/apps/bioreactor-simulator.verification.ts. They detect change, not error; correctness comes from the SciPy scenarios. A flags-only supersession (FLAG_REVISIONS, first used for setup 2.2.0 on 24 October 2026) keeps the superseded record under test: it is replayed against today's engine with only the added flag code masked, so its curves stay checked.
Exact-time investigation (Wave 8; run once as a diagnostic on 24 October 2026).
pipeline/bioreactor-simulator/exact_time_reference.py prepares an independent
check of the early rescued E. coli feed-cap transient. It reuses the Python
equations above and integrates the published fixed rescue schedule separately
from a schedule retained in a failed golden-test diagnostic. Radau refinement
and a BDF cross-check evaluate common binary64 sample times without the old
fixture's compact rounding. The script pins SciPy/NumPy, records source hashes,
and writes only to a new investigation directory outside the repository.
It cannot overwrite a published fixture. It has no production consumer or
acceptance rule. Syntax has been checked; numerical convergence, event evidence,
review and a new write-once publication remain outstanding. The audit's
rescue-root comparison repair does not substitute for this independent result.
On 24 October 2026 (Stage H BR3.5, the CSO's ruling 1 of that day) it was run
once under the pins, without the retained diagnostic log (not in the
repository), and filed as evidence with a summary under
docs/science/evidence/bioreactor/exact-time-2026-10-24/: Radau refinements
agree at the disputed time within 3e-10 mmol/L/h OUR, Radau and BDF within
1.1e-6, and the published golden differs from the finest Radau by 6e-8 mg/L
DO and 1.2e-5 mmol/L/h OUR. It takes no verification version, backs no
displayed number and gates nothing; B2 no longer waits on it.
Measured-run comparison (Contract 11A). Three further sets check the comparison's elapsed-time adapter (method measured-comparison-time-alignment 1.0.0, Bioreactor). They are separate from the process-model fixtures, use the Python standard library only, and import no TypeScript.
| Script | Writes (published) | Cases | Oracle |
|---|---|---|---|
reference-measured-comparison.py (612 lines) |
fixtures/measured-comparison-0.3.0.json (0.3.0) |
8 projection cases: 40 rows, 36 with an elapsed time, 31 of them exact in binary64 and 5 repeating fractions of an hour; plus 12 refusal entries that name a code and its fault but carry no inputs | fractions.Fraction, Python 3.9.6, 19 October 2026 |
reference-measured-comparison-2.py |
fixtures/measured-comparison-0.4.0.json (0.4.0) |
3 cases that carry their inputs: two measured sources and a recorded simulation reference that fails its hash, its output reader (an RQ value missing where OUR is not zero) or its schema. Each expects the pair compared, the reference dropped (no panel, no part of the elapsed-time union, no simulated qualification), the qualification simulation-reference-unavailable with the failing code, and the hashes the reference claimed |
fractions.Fraction and hashlib, Python 3.13.16, 20 October 2026 (the CSO's ruling of that day, section 5) |
reference-measured-comparison-3.py |
fixtures/measured-comparison-0.5.0.json (0.5.0) |
25 cases, each carrying the complete stored comparison (both imported growth-review payloads with their source and payload hashes, the arrival contexts, the alignments and any recorded reference with its snapshots), so the test constructs nothing (SCI-186). 15 are refused: every validation and verification code from its own input (two inputs each for malformed-comparison-payload, invalid-growth-review-payload, source-input-hash-mismatch and missing-time-alignment-basis) and a reference alignment in minutes or with an empty basis. 3 make the reference unavailable by other rules than 0.4.0's (input schema version 3, the result naming another input, t starting at 1 h) while the pair compares. 7 project a verified reference: anchored at 0, 1.5 and 0.1 h, unresolved, beside an unaligned source, with RQ declared undefined at t = 0, and beside a source carrying its Assay reference (its payload hash is taken with analysisRef null, so the whole-payload hash is not needed). The two largest inputs (2,000,000 bytes; 4,001 readings) are stored without their generated readings, with an expansion rule and the SHA-256 of the expanded record |
fractions.Fraction and hashlib, Python 3.12.14 (built from the signed python.org source), 24 October 2026 (the CSO's ruling of that day, section 6) |
The second script builds the reference snapshots itself, restates the reader's rules from the contract, and recomputes the workspace snapshot hash (SHA-256 of the record without key and contentHash, sorted keys, no whitespace). Before it was committed, the same hash was recomputed in Python for the five snapshots of the engine-written legacy fixture below, and matched. Both scripts write their published copy and manifest themselves; the second refuses to replace an existing file.
The third script restates the comparison's whole validation, verification and projection from the contract and the transport, in the engine's order, and asserts that each refusal input fails exactly its intended rule and each other input verifies. Its canonical JSON lays numbers out as ECMAScript does (Number.prototype.toString), so the source, payload and snapshot hashes it writes are the strings the engine computes. Elapsed tolerances are per point: 0 where the entered time, the anchor and the result are binary64 numbers (IEEE subtraction and division are then exact), otherwise (ulp(time) + ulp(anchor)) / divisor + 2 ulp(result), which replaces the fixed 1e-15 h that is smaller than one ulp above 8 h. Two observations from writing it: the combined 4,000-reading bound is twice the transport's 2,000 per source, so two readable sources never exceed it and aggregate-reading-limit-exceeded is reached only when one source is itself over 2,000 (the combined count is taken before either source is read, and the message names both bounds); and nonfinite-time-anchor follows binary64, so a time and anchor whose difference overflows are refused although the exact elapsed time exists.
Consumers: src/apps/bioreactor-simulator/measured-comparison.test.ts (0.3.0, 0.4.0) and measured-comparison-completion.test.ts (0.5.0, which expands the two large inputs, checks the expansion against the oracle's hash and compares elapsed times at each point's own tolerance). The first asserts that each published copy is byte-identical to its pipeline fixture and that the fixture's about.generatorSha256 and the manifest's scriptSha256 equal the script's SHA-256. It drives every case through the engine: elapsed hours at tolerance 0 where exactBinary64 holds and 1e-15 h otherwise, every other compared field exactly. For the 0.3.0 refusal entries the test constructs each fault itself, so the oracle does not check those constructions (audit of 20 October 2026, measured-overlay m4).
kLa from a value or readings, and the feed (Stage H, 24 October 2026). Two further sets close the CSO's additions A1 and A2 (docs/board/memos/2026-10-24-cso-bioreactor-stage.md): the two methods the inventory listed as unit-tested only. Each script restates its method from the method statement, imports no TypeScript, refuses to run off its pins and refuses to replace a published file.
| Script | Writes (published) | Cases | Oracle |
|---|---|---|---|
reference-kla-scaling.py |
fixtures/kla-scaling-0.6.0.json (0.6.0) |
18 settings of klaAt: under klasrc=value the setting itself (returned exactly), agitation up and down, gas flow, a later volume, all three together (with one and two impellers: the count cancels), another vessel, a gas flow on a reference basis, the non-coalescing exponents (arithmetic only; unread, D-0025) and a setting where Hughmark's ratio is capped at 1; under klasrc=table a reading as read, log-log interpolation, extrapolation above and below, a gas flow other than the readings', unsorted readings with a gap, and a single reading |
NumPy 2.3.5 float64 scalars, Python 3.12.14: Van 't Riet's form between two settings, kLa₂ = kLa₁ ((N₂/N₁)³ (r₂/r₁)(V₁/V₂))^b (Q₂/Q₁)^c with Hughmark's ratio r (Yawalkar et al. 2002, eq. 2) capped at 1; relative 1e-12, 0 where returned as entered. The engine agreed within 2e-16 when the set was written |
reference-feed.py |
fixtures/feed-0.7.0.json (0.7.0) |
12 feeds: constant (to tEnd, to Vmax, to the horizon), exponential (to tEnd, held at the pump limit, Vmax reached while growing and while at the limit, starting above the limit), the rescue's cap (resolved, unresolved, timed before the feed's start) and a vessel already full; for each the pieces, the stop and its reason, rates and fed volumes at probe times, every pump-sheet row and the total. 4 design rates F0, one undefined (Sf ≤ S0) | Python decimal at 40 significant digits, Python 3.12.14; relative 1e-12 with an absolute floor of 1e-15 L or L/h; times within 1e-12 of the feed's length; row counts and stop reasons exact. The engine agreed within 7e-16 when the set was written |
reference-feed-handoff.py |
fixtures/feed-handoff-0.12.0.json (0.12.0) |
14 schedules over A2's published pump sheets (feed-0.7.0.json, its SHA-256 recorded; every feed, Sf from 250.5 to 790 g/L): each row's time in h and min, rate in L/h and mL/min, mass flow in g/h, volume and substrate mass fed in L, mL and g, and the totals. 15 programmes: each A2 feed's keys as H28 sends them, its stop (A2's, or the run's end) and total fed (A2's closed-form total), and the refused incomplete programmes (capNotReached, noFeed, and three synthetic cases: no starting rate, a cap with no starting rate, a feed that starts after the run). 57 substrate matches on the shipped table src/lib/hosts/substrate-identity@1.json (bioreactor-substrate-identity@1, its contentHash recorded as tableHash): every glucose form and the methanol key, a kept compound resolved, unfolded and resolving to another kept ref, molarities at the model's molar masses, both monohydrates by molarity, mass and %w/v (times 180.156/198.171), methanol by mass, molarity and volume, %w/w and a bare %, a mass, zero, negative, Sf's limit and above it, a name only, a CAS, lower-case, bare, trailing-space and first-block refs, a protein and a recipe ref beside a match (ref-not-canonical), a ChEMBL ref and a stereo-less key beside a match (context), a changed protonation character, L-glucose, hexose, the stereo-less and other-configuration hydrates, sucrose, glycerol, glucose for a methanol run, two forms, glucose with its hydrate, a feed recipe in batch and chemostat runs, a medium recipe with and without a chosen target, and the refusal order. The keys: RDKit 2026.3.6 (pipeline/lab-calculators/requirements.txt, its SHA-256 recorded) recomputes every InChIKey of the committed PubChem snapshot fixtures/pubchem-substrate-inchi-2026-10-24.csv (13 records: the seven ruled and six refused; read 24 October 2026, its SHA-256 recorded) from its InChI, and the generator refuses to write unless the table is exactly the snapshot's ruled rows; it never fetches |
Python fractions (exact), Python 3.12.14 in a venv from pipeline/lab-calculators/requirements.txt (RDKit for the key check only); relative 1e-12 with an absolute floor of 1e-15; keys, stops, refusal codes and row counts exact (the CSO's addition A3, H28 rulings and substrate-identity memos; the CTO's of 24 October 2026; Stage H BR5) |
Consumers: src/lib/hosts/kla-scaling-oracle.test.ts and src/lib/hosts/feed-oracle.test.ts, each asserting the published copy, the script's SHA-256 in the fixture and the manifest, and the pins. FEED_VERSION and SETUP_VERSION are unchanged: both engines agreed.
B2, the μmax-only batch fit: fixture before any engine (Stage H BR7.1, 24 October 2026). pipeline/bioreactor-simulator/wave8/batch-fit-reference.py writes fixtures/batch-fit-0.9.0.json (verification 0.9.0) under the CSO's ruling 2: reference.py's model 2.0.0 equations (imported, unchanged), solve_ivp Radau segmented at the regime switches and evaluated at the exact observation times, refined at rtol 1e-9, 1e-10 and 1e-11 with a BDF 1e-11 cross-check; a deterministic 33/65/129 grid in u = (μmax − lower)/(upper − lower), bounded scalar minimisation in every basin, a least_squares(method="trf") cross-check. 20 cases on the proposal's seven: noiseless recoveries (glucose 0.61, slow glucose with product 0.28, methanol Haldane 0.146, oxygen-limited, growth into starvation; all recovered within 3 × the achieved resolution, about 1e-9 per hour), seeded noise, unit-equivalent forms (g/L and h against mg/L and min give the same estimate to the last digit), replicates and an exclusion, OD only and near-coincident times (refused), no growth and bounds excluding the truth (not resolved), two misspecified fixed inputs (one detectable, one confounded), two contradictory-channel cases with two basins, and two resource-cap adapter contract cases. The manifest freezes the three rules (agreement within 3 × the oracle's plus the engine's spread; objective within Σ(2|rᵢ|eᵢ + eᵢ²); resolved only when interior, single-basin within the envelope and the profile rising by more than 10 × the envelope on both sides) and each case's oracle spread, achieved resolution and envelope. Its notes state where those numerical rules and practical identifiability part: the noiseless poorly-excited case and both contradictory-channel cases (one a near-tie, 0.03 percent apart) come out resolved, and the redesign of the contradictory case after a trial run is recorded with the trial's case and result. src/lib/hosts/batch-fit-fixture.test.ts holds the publication (copies, hashes, pins, rules, verdicts); since BR7.2 the engine is checked against it (below). The CSO read it on 24 October 2026 (docs/board/memos/2026-10-24-cso-bioreactor-b2-fixture-read.md): compliant with the contract; "resolved" means numerically resolved only, and a fourth field, practical, carries flag codes with thresholds frozen in that memo (weak-profile and near-competing-minimum withhold application at ΔJ = 4; second-minimum and misfit, RMS scaled residual above 2, are shown), derivable from 0.9.0's stored values, so 0.9.0 stays as published and a derived set follows at a version the CTO names. Two oracle findings for any later version: the least_squares trf status is not checked (all 16 fitted cases have status 1 to 3), and XATOL_U = 1e-10 as the floor of achievedResolutionU understates the bounded search's resolution (SciPy stops near 2 × (√ε·|u| + xatol/3)). The trial run before the contradictory-channels redesign is filed as evidence under docs/science/evidence/bioreactor/b2-trial-2026-10-24/. The CEO took the new-engine decision on the CSO's condition (D-0131).
B2's practical flags (verification 0.10.0) and the engine checked against 0.9.0 (Stage H BR7.2, 24 October 2026). pipeline/bioreactor-simulator/wave8/batch-fit-flags.py (standard library, Python 3.12.14) reads fixtures/batch-fit-0.9.0.json by path and SHA-256 (64572644…63ba, byte-identical to the published copy) and writes fixtures/batch-fit-flags-0.10.0.json, a derived set with no new SciPy result: for each numerically resolved case the CSO's practical flags with the thresholds frozen in the CSO's read (weak-profile, the weaker profile rise below ΔJ = 4, and near-competing-minimum, another basin less than 4 above the lowest, withhold application; second-minimum, another basin 4 or more above, and misfit, RMS scaled residual above 2, are shown); "another basin" is 0.9.0's own (more than 2 × the achieved resolution in u away). The script refuses to publish unless its flags equal the CSO's table: poorly-excited weak-profile (withheld), contradictory-channels-near-tie near-competing-minimum and misfit (withheld), contradictory-channels second-minimum and misfit, misspecified-yield-detectable misfit, every other resolved case none. 0.9.0 is unchanged. Under the CSO's engine-acceptance ruling 3 (docs/board/memos/2026-10-24-cso-bioreactor-b2-engine-acceptance.md) the same flags are republished as 0.11.0 by wave8/batch-fit-flags-2.py, which computes every flag first and compares the set with the CSO's table as a separate assertion, reads 0.9.0 and 0.10.0 by SHA-256, and gives the manifest what ruling 3 asks: the thresholds, the value behind each flag (every other basin's ΔJ, both profile rises, the RMS scaled residual and n), 0.9.0's SHA-256, the citation of the fixture read §2, "resolved means numerically resolved", applicability per case (not-resolved and refused cases "not applicable" by verdict, with no flags) and "adds no new oracle computation". 0.10.0 stays as published, superseded; the engine names 0.9.0 and 0.11.0. The engine, bioreactor-batch-fit@1.0.0 (src/lib/hosts/fit/; engines.md), is held to the CSO's acceptance items 1 to 10 by src/lib/hosts/fit/batch-fit-oracle.test.ts (35 tests) and batch-fit-contract.test.ts (8): admission outcomes, codes and canonical rows equal exactly, both unit forms bit-identical, the −4.84e-16 g/L reading admitted unchanged; one prediction per distinct time, a time-map failure refusing the fit; the engine's predictions at the oracle's μmax within (oracle eᵢ + engine eᵢ) × scale at every observation; every estimate within 3 × (oracleSpreadMu + engineSpreadMu), the engine's spread (shipped against tightened 100× in rtol and atol) at most 1e-4 μmax; objectives within Σ(2|rᵢ|eᵢ + eᵢ²); the oracle's lowest basin, and every basin of both contradictory cases (four and two); verdicts and reasons equal with the engine's envelope added to the oracle's; the practical flags equal to 0.11.0 (read from its file); at each resolved optimum a stable halving with max |∂r/∂u| within 1e-3 of the oracle's and closures at most 1e-9; and, in engine tests, the two resource caps, an injected solver failure (profile-incomplete), a failed refinement (integration-failed), sensitivity-unavailable and deadline exhaustion, each with no estimate. The engine's estimates differ from the oracle's by 2e-8 to 7.4e-7 per hour, about the engine's own spread. One measure differs from the oracle's (SCI-262): the engine's prediction envelope adds the state's shipped absolute tolerance to the level differences, because four near-zero substrate readings after exhaustion fell outside the level differences alone; the CSO amended item 3 to allow it on conditions, and the test asserts that a reading passing only with the term lies below the shipped atol ÷ rtol. At the tightened level two of those readings (s-9, s-10) are not yet within the oracle's envelope plus that level's atol (1(b) as first worded), and all four reach it by rtol 1e-12. The per-case figures for items 3 to 9 are filed in docs/science/evidence/bioreactor/b2-acceptance-2026-10-24/. B2's shipped level is rtol 1e-8 with each state's atol scaled from the simulator's (S 1e-10 g/L in these cases), not the simulator's own 1e-7; the term only widens the envelope, which makes "resolved" harder to reach. The CSO replaced the convergence check by 1(b′) after seeing 1(b) fail (2026-10-24-cso-bioreactor-b2-engine-acceptance-2.md); its monotone clause fails at maintenance-transition s-2 between rtol 1e-7 and 1e-8, a reading three orders below the shipped atol, so the CSO withheld engine acceptance (2026-10-24-cso-bioreactor-b2-engine-acceptance-3.md). The CEO then amended clause (i), after the data were seen, to the CSO's replacement text (D-0137): between adjacent levels the difference does not increase where the looser level's difference exceeds that level's S atol, and where it does not, the difference is at most that level's S atol. Under the amended condition all 36 records pass and the engine is accepted on items 1 to 10 for bioreactor-batch-fit@1.0.0; item 11 stays with the receipt (BR7.3), and any later failure of the amended condition fails item 3 outright (SCI-262). The test is src/lib/hosts/fit/b2-acceptance.check.ts, run as its own CI step ("B2 acceptance", reporting, continue-on-error) by the CTO's ruling; batch-fit-oracle.test.ts holds the tripwire, which expects no failure (Build, test and CI). The Levenberg–Marquardt cross-check's stop is recorded per case and not acted on; a test holds its optimum to Brent's within the resolution measured without it.
B2 1.1.0 against verification 0.13.0 (D-0139, 25 October 2026). pipeline/bioreactor-simulator/wave8/batch-fit-reference-2.py imports the 0.9.0 reference by path and SHA-256 and replaces only admit (the 1.1.0 rule: the initial value is the mean of the kept t = 0 readings in g/L, summed in row order from 0 then divided by k, never a compensated sum; then initial-state-not-physical; every refusal with its detail; the initial-state basis on an admitted case). It writes fixtures/batch-fit-0.13.0.json (SHA-256 e45d9c0078315d81…): the 20 cases of 0.9.0 re-run through the 0.9.0 reference's own run_case and asserted byte-equal (not copied), and 13 new cases (the replicate means, a negative replicate, excluded rows at t = 0, the non-physical refusals and their order, S0 = 0 and a +0 mean), each with its 1.0.0 expectation computed by the unchanged 0.9.0 admit and labelled. The generator asserts that every S0 = 0 case ends not resolved on an exactly flat objective (every profile J equal to the lowest, every scaled residual 0) and flags it flatObjective; that case (d)'s μmax from the median, first or last reading differs beyond 3 × the resolution; and that (h)'s mean is +0. It is resumable through per-case checkpoints keyed by its own SHA-256 (the session host restarts), and the fixture lists any case it resumed. batch-fit-oracle.test.ts holds 1.1.0 to 0.9.0 and 0.11.0 unchanged but for the edits D-0139 names (condition 5(a)); batch-fit-oracle-0-13-0.test.ts holds it to 0.13.0's new cases (5(b)). On a flagged case the reasons are compared with every bound-contact removed from both lists, with status, both profile rises within the envelope and the objective rule checked: where a bounded minimiser stops on a constant function is a property of the implementation, not the data (the CSO's ruling of 25 October 2026); the exemption is keyed on the flag, never on a case id. The run record is docs/science/evidence/bioreactor/b2-1-1-0-oracle-2026-10-25/.
Numbers in prose. src/lib/hosts/preset-prose.test.ts recomputes the hand-typed numbers in the preset and calculator pages' words (content/apps/bioreactor-simulator.pages.ts) from the preset values and range constants they describe: the Pichia page's growth optimum (0.08 per hour at 3.6 g/L, from μ = (μmax + Y·ms)·f(S) − Y·ms at S* = √(Ks·Ki)) and feed rate, the kLa calculator's Van 't Riet and Hughmark range sentences, the OTR page's Benson–Krause range and the scale-up page's search limit (the CSO's addition A5). Source-quoted numbers not held in a preset are not asserted.
Supplementary transport: feed dissolved oxygen and a reference-basis gas flow. pipeline/bioreactor-simulator/reference-transport-20261011.py wrote fixtures/transport-20261011.json on 11 October 2026 with Python 3.9.6, SciPy 1.13.1 and NumPy 2.0.2: four chemostat cases (a baseline, feed DO 20 mg/L, a gas flow on a reference basis at 0 °C and 1 atm, and both) solved by reference.py's solve. It sits in no manifest and is read by src/lib/hosts/transport-oracle-20261011.test.ts, which pins its hash. Under the CSO's ruling 5 of 24 October 2026 it stays as it is, and reference-transport-pinned.py reruns the same four specifications (it checks them equal to the earlier file's) with the same, unchanged reference.py under the pins, publishing fixtures/transport-0.8.0.json as verification 0.8.0. Its crossRuntime block and the manifest's supersedes record the difference from the earlier file: every sample time identical; every output, peak, event time and switch identical as written (reference.py rounds outputs to 12 significant figures and event times to 10, so this is agreement to that rounding); the unrounded slopes within 5.9e-11 of their largest magnitude; the closures at 1e-16 against 0. src/lib/hosts/transport-oracle-0.8.0.test.ts holds the engine to the same gate as the earlier test (0.5 percent of peak per output, the accuracy agreement with its time-shift allowance, events within 1e-3 h, closures within 0.5 percent).
src/apps/bioreactor-simulator/fixtures/measured-comparison-settings-legacy-v1.json is engine-written, not an oracle: two recorded comparisons captured by the engine before the settings snapshot received its own schema id (the CTO's ruling of 20 October 2026, item 7), kept so that the legacy settings reader is tested against what the old writer actually wrote. pipeline/bioreactor-simulator/write-legacy-measured-comparison-settings.mts wrote it and now refuses to run.
The preset pages' runs file is not a fixture. public/data/bioreactor-simulator/preset-runs.json is written by the prebuild from the engine itself and checked by preset-runs.test.ts, which recomputes it; no oracle is involved. Since 24 October 2026 (REL-09) its closure residuals are rounded to an absolute 1e-9 before six significant figures, so its bytes do not depend on the machine: the session host (4 cores, Node 22.22.0) and the Fast chain build of 36929bb4 (run 40086036121, GitHub's ubuntu-24.04 runner, prebuild line at 10:43:28 UTC) both wrote SHA-256 ac44732a4cf5399a925526c28eee6e03b14b4a9f433cb65498b9777ba06ce70b.
Regenerating. From docs/science/2026-10-scientific-audit-implementation.md and the requirements file:
node pipeline/bioreactor-simulator/export-presets.mts
python3 -m venv /tmp/bioreactor-venv && /tmp/bioreactor-venv/bin/pip install -r pipeline/bioreactor-simulator/requirements.txt
/tmp/bioreactor-venv/bin/python pipeline/bioreactor-simulator/reference.py all
node pipeline/bioreactor-simulator/write-golden.mts --freeze
node pipeline/bioreactor-simulator/record-presets.mts records preset version hashes in src/lib/hosts/presets/manifest.ts. The process-model fixtures have no publish script: the copies under public/verification/bioreactor-simulator/0.2.0/ were made by hand, and fixtures.test.ts:199 hard-codes that directory. The measured-comparison scripts publish their own 0.3.0 and 0.4.0 copies (above).
Sequence
Methods are described in Sequence.
Scripts and oracles
| Script | Writes | Oracle |
|---|---|---|
pipeline/sequence-toolkit/reference.py fixtures |
oligo-tm.json (45 cases under two concentration models, 7 refusals, 3 acceptance cases), hairpin-dimer.json (20), hairpin-dimer-probes.json (11 salt probes), hairpin-dimer-extra.json (5 loop-class probes, reported not gated), translation.json (108 frame translations and 5 CDS), orf.json (18), dna-mw.json (5), protein.json (22) |
Biopython 1.88 MeltingTemp.Tm_NN (DNA_NN3; salt corrections 5, 6 and 7), Seq.translate, ProteinAnalysis; primer3-py 2.3.1 calc_hairpin, calc_homodimer, calc_heterodimer (libprimer3 2.6.1 thal), each rerun through the bundled ntthal program; an independent ORF scan cross-checked with Biopython and NCBI ORFfinder 0.4.3; saved Expasy ProtParam responses; exact rational arithmetic for DNA mass |
pipeline/sequence-toolkit/peptides.py |
peptide-smiles.json (101 cases: 89 drawn, 12 refusals) |
Python RDKit 2026.03.6 Chem.MolFromSequence (flavor 0); version-checked |
pipeline/sequence-toolkit/exact_matches.py |
pcr-product.json (30 cases, 13 refusals), fragment-search.json (26, 9 refusals) |
Independent derivation: every window tested directly, circular case by explicit doubling, reverse complements from Biopython 1.88; one case worked by hand base by base, and the script refuses to write unless its derivation agrees |
pipeline/sequence-toolkit/revision-transforms.py |
revision-transforms.json (10) |
Hand-worked coordinate maps and an independent unit-cost edit distance (dynamic programme written in the script); reverse complements by Biopython 1.88 |
pipeline/sequence-toolkit/sdm_primers.py |
sdm-primers-v1.json (10 design cases over the two schemes, 8 refusals) |
Biopython 1.88 Tm_NN for every melting temperature (the oligo Tm fixtures' oracle and calls) and Bio.Seq.reverse_complement for every reverse complement; the mutant sequence, the candidate windows, the matched flanks, GC, the ranking and the counts by reason derived in the script from the CSO's contract of 17 October 2026; one case per scheme worked by hand with a write-refusal guard; written before the engine existed, so no value in it can have come from the code it checks. The only Tm-decided rule is the back-to-back pair's ΔTm, and a case is refused rather than written when any candidate's ΔTm lies within 1e-6 °C of that limit or the returned order would rest on one step of a ranking term's published precision |
pipeline/sequence-toolkit/primer_design.py |
primer-design.json (9 design cases, 4 refusals) |
Biopython 1.88 Tm_NN for every melting temperature (the oligo Tm fixtures' oracle and calls); candidate windows, GC, penalties, caps and counts derived in the script from the method's conventions; exact sites and products by exact_matches.py; one case worked by hand with a write-refusal guard; a case is written only when no decision turns on the Tm tolerance (margins 1e-6 °C, with the Tm bounds, ΔTm limit and preferred Tm moved to the nearest clearing values and recorded). primer3-py 2.3.1 run offline on the real-template cases as a recorded comparison (docs/science/evidence/primer-design-2026-10-11/), not as the oracle |
pipeline/sequence-toolkit/align_reference.py |
dna-align-v1.json (60 cases, 12 refusals; published in 0.8.0, regenerated under the CSO's fixture ruling and the architecture rulings of 12 October 2026) |
Biopython 1.88 PairwiseAligner with a 15-letter IUPAC matrix (scoring set dna-default-1: +2, −3, ambiguity 0) and gap scores −5 and −2 (the gap cost 5 + 2(k − 1) without conversion), in global, semiglobal-read (the four end "deletion" scores 0) and local, both strands, a circular reference on the CSO's certified text (the reference and its first m − 1 + D bases, D certified): optimum, co-optimal count and up to 20,000 co-optimal alignments enumerated for checking (1,000 listed). Independent of the script's programme: the tie-order alignment is the first of Biopython's set sorted by the published order, the shared nodes are the set's intersection (on a circle, a node and its copy n letters on as one), each difference's common is read again off the set (its run and both bounding columns in every alignment), and the placement groups are the set's partition by shared pairs of the reference. The programme (a consistency check) gives the alignment, the shared pairs again by forward and backward path counts, the groups and the wrap by one forward pass over its optimal-path graph (exact above the 20,000 enumerated too), the diagonals and the reported numbers, and must agree with all of these; one case worked by hand; each of the CSO's cases 7a to 7j, the first readings' cases (an unlisted wrap, two placements across an origin, the strand example, the regions) and the final reading's (the folded circle, no paired letter, the two insertion positions, the periodic circle, one undetermined group) is checked for what it says, all with write-refusal guards |
| By hand | selection-hand.json (8: 3 GC, 5 position cases) |
The working is in the file |
pipeline/sequence-toolkit/make-abif-fixtures.py |
reads/abif-v1.json (60 cases, regenerated 20 October 2026 for Q43 as amended and the specification's reading rules) and 60 abif-*.ab1 files: 20 synthetic reads from J01636.1 and L09137.2 with synthetic peaks, traces and confidence values (the CSO's seven Q43 files, the seventh as four range files, 99 and 94 read and 100 and 0 not, the CTO's set-1-only and set-2-only files, no traces, no confidence values, no peaks, mixed-base calls, and six base-caller cases on the CTO's SPAC2/SVER2 bounds), nine files the ABIF specification says a reader must accept (a directory data size larger or smaller than 28 × count, its name, type or element size differing, a user-type and an unsupported legacy entry whose sizes disagree, version 199; one reads as the unaltered base file where Biopython walks it differently, readsAs) and a 19-file corruption corpus (versions 200 and 99, an undefined element type and a user-type entry past the end among them); each case's expected kind, every quality reason, base caller and edited-calls state written by hand from the rulings |
Biopython 1.88 SeqIO abi and its abif_raw dictionary, or Biopython's own directory walk (AbiIO._abi_parse_header) where its reader refuses a file (the edited-set-only file); Biopython reads seven corrupt files the reader refuses, recorded side by side. The same understanding of the format wrote the files and the reader, so instrument files are still needed (held for the General Counsel). Stage B reads module, not yet merged |
pipeline/sequence-toolkit/read_check_reference.py |
readcheck/readcheck-v1.json (write-once from main; its public copy, with its manifest and read files, goes in Sequence 0.9.0 with SCI-09; 44 reviews, RC-1 to RC-20b with RC-14c, one a line: settings, the reads' files, expected position arrays as run-lengths, rows with each counted read's per-position summary in the CTO's columnar shape, listings, totals, junctions, ratios and refusals) and the 44 read files beside it (FASTQ, FASTA and the writer's .ab1), with its own manifest readcheck/manifest.json; generated 20 October 2026 after the verifier confirmed the plan's new values; the independent third check agreed and the CSO read the set; regenerated under Q43 as amended twice with RC-14c, a second run byte for byte. Published 20 October 2026 as generated, write-once from here (the CSO's six conditions met, the aligner's brute-force run without a disagreement); it lands on main after the aligner, through the merge queue; publication also waits on the aligner's regenerated head passing the CSO's re-reading and on the reader fixtures under the ABIF fixes (docs/progress/sequence-readcheck-fixture-plan.md) |
Biopython 1.88 PairwiseAligner in semiglobal-read (the reference's flanks free, every read base aligned, read bases past a linear end penalised), with only align_reference.py's scoring constants imported; both strands; a circle on the certified text (D = m) with the drop rule; every equally scoring alignment enumerated (refusing above 20,000). The CSO's ruling 2 counting rule is written fresh from its text: common pairs, the counted stretch, fixed, deleted and variable positions, common insertions (the same flanking columns in every alignment), varying ends and overhang; trimming in integers; the junction rule; Q43 as amended twice applied to the four .ab1 files (RC-14b (i) and (ii), RC-14c, RC-18) written by make-abif-fixtures.py's writer and read back with Biopython's abif_raw, with the KB documentation a named input ("held"); the peak ratio as an exact fraction. It refuses on any difference from the plan's hand values. B-4 read check, not merged |
pipeline/sequence-toolkit/make-fastq-fixtures.py |
reads/fastq-v1.json (51 cases and 5 read sets, regenerated 20 October 2026 for the CSO's ruling 3 of 14 October and on 21 October for the readers review's read names) and the .fastq files beside it, including codes 63 and 32 at offset 64, every case of ruling 3's table (blank lines before, between and after records and in CRLF, lines of only spaces or of a tab in each place, a blank line at each of a record's three inner positions, an empty read first, mid-file and last, a file of blank lines only) and, for the read sets, N = cap + 1, 2·cap − 1, a prime N, two files below the cap and four files given out of code-point order with one that gives no read (indices from the script's own integer arithmetic), cut by hand from J01636.1 (already in pipeline/plasmid-map/records/, never fetched); its own manifest reads/manifest.json |
Biopython 1.88 SeqIO fastq-sanger and fastq-illumina, whichever offset the case declares; the outcome Drybench must give (reads, or a refusal code and line) is written by hand from the rulings and recorded beside Biopython's, with every case where the two differ naming the difference and the rule behind ours (differs): Biopython reads the wrapped, empty-read, non-IUPAC, inner-blank and between- or trailing-whitespace cases that the four-line reader refuses (the CSO's Q44, ruling 3, architecture §6.3), and refuses the leading-blank case ours reads, so that case also records Biopython's parse without its leading blank lines. Stage B reads module, not yet merged |
The fixtures step also asserts that Biopython's DNA_NN3 equals SantaLucia 1998 Table 2, that its genetic code tables 1, 2 and 11 equal ncbi/gc.prt, that each Owczarzy 2008 case took the regime its R selects, and that every sodium-equivalent rerun of a hairpin or dimer is bit-identical; it stops on any ORF disagreement with Biopython or ORFfinder.
Saved inputs, so the fixtures step runs offline: inputs/nucleotide/*.gb (NCBI efetch), inputs/uniprot/*.json (UniProt release 2026_03), inputs/sources.json (URL, time, status, SHA-256 and release of each), protparam/P01.html to P22.html with index.json, orffinder/*.fa (36 runs) with index.json, ncbi/gc.prt.
Pins. pipeline/sequence-toolkit/requirements.txt: biopython==1.88, primer3-py==2.3.1 (GPL-2.0-or-later, build-time only; its primer3_config files are never copied) and, since 12 October 2026 (MNT-17), rdkit==2026.03.6 for peptides.py, which checks EXPECTED_RDKIT = "2026.03.6". reference.py refuses other versions. reference.py still rewrites fixtures/manifest.json from its own entries only (MNT-17). The ORFfinder binary (linux-i64, SHA-256 recorded in the README) is downloaded and not committed. Python 3.11.15 is recorded.
Fixture record shape
Each case follows { id, input, expected, method, oracle, oracleVersion, recordedOn, source } plus call and notes where useful; each file's about block records the generating script, its SHA-256, run date, oracle, licence and the ruling that shaped it. pipeline/sequence-toolkit/fixtures/manifest.json holds each file's SHA-256 and case counts; files written by the other scripts carry their own generatedBy, and some their own scriptSha256.
Consumers and tolerances
| Test | Fixture | Tolerance |
|---|---|---|
src/lib/seq/tm.test.ts |
oligo-tm.json |
Tm under both models within 1e-6 °C of Biopython (the badge states 0.1 °C); ΔH and ΔS 1e-6; R 1e-9 relative; regime, near-boundary flag, refusals and acceptance exact |
src/lib/seq/structure.test.ts |
hairpin-dimer*.json |
Gate per engine: ntthal's own structure rescored by the engine's rules within STRUCTURE_GATE.<engine>.tolerance (hairpin 0.4, dimer 0.1 kcal/mol, each the measured maximum rounded up to 0.1, ceiling 0.5); the engine's own structure at or below that rescore; no-structure cases report nothing below −0.1 kcal/mol; the dynamic programme's minimum equals exhaustive enumeration on 200 seeded random oligos per engine. Loop-class probes reported, not gated |
src/lib/seq/translate.test.ts, src/lib/seq/orf.test.ts |
translation.json, orf.json |
Exact, every field of every list |
src/lib/seq/protein.test.ts |
protein.json |
MW 0.01 Da, pI 0.01, ε280 exact |
src/lib/seq/dna-mw.test.ts |
dna-mw.json |
0.01 g/mol; formula exact |
src/lib/seq/peptide.test.ts |
peptide-smiles.json |
The engine's SMILES canonicalised by RDKit.js 2026.03.6 identical to Python RDKit's canonical SMILES; refusals exact |
src/apps/sequence-toolkit/exact-match-fixtures.test.ts |
pcr-product.json, fragment-search.json |
Exact, refusals included |
src/lib/seq/revision-transforms.test.ts |
revision-transforms.json |
Kind, map, hashes, withheld state and annotation finding exact; edit list length equals the independent edit distance and reconstructs the target |
src/lib/seq/sdm-primers-fixtures.test.ts |
sdm-primers-v1.json |
Candidates in order; primer sequences, lengths, matched flanks, matched 3′ segments, coordinates, origin crossings, the counts by reason, the cap and the ranking terms exact; GC within 1e-9; every melting temperature and the pair ΔTm within 1e-6 °C; refusals by field and message pattern. Two invariants over every returned primer: its matched 3′ segment is its own 3′ matched flank, and a primer reported as fully matched carries one Tm. One test asserts that no result mentions a cycle count, polymerase, DpnI, phosphorylation or ligation, or carries a mismatched-duplex Tm field. The Tm qualification the result carries (tmLimits: the salt correction's fitted range and any out-of-range note) is the Tm engine's own and is gated by oligo-tm.json and the primer-qualification sets, not by this one |
src/lib/seq/primer-design-fixtures.test.ts |
primer-design.json |
Pairs in order; annealing parts, tails, coordinates, products, exact site and product counts, counts and the capped flag exact; GC within 1e-9; Tm, penalties and ranking terms within 1e-6 °C; refusals by message pattern |
src/components/tools/sequence-toolkit/model-selection.test.ts |
selection-hand.json, oligo-tm.json |
Exact |
src/lib/seq/align/align.test.ts (branch claude/sequence-dna-aligner) |
dna-align-v1.json |
Score and co-optimal count exact on both strands; the selected alignment equal to the oracle's tie-order alignment and among Biopython's enumerated set; every value the fixture records exact, under a test-only column bound: listed alignments in the published tie order, numbers, notFixed anchors, placement groups with their sizes and bands, hull, the placement and strand flags, variation, tuples, the band block and diagonals; a listing cut short by the bound, in the CTO's unit (columns less free end gaps), is the oracle's prefix on linear and circular strands alike (the oracle does not encode the bound); under the 20,000-cell test limit band-limited with banded acceptance off (shared words in the CTO's unit equal to the fixture's), and with it on equal to the whole table or band-limited; refusals exact |
src/lib/seq/align/property.test.ts (branch claude/sequence-dna-aligner) |
None: 170 generated pairs, the banded search against the whole table; 44 n-spaced pairs for the count filter's lemma |
Zero tolerance on outcome, score, capped count, columns and shared pairs; every first table inside 20,001 to 100,000 cells; every route at least ten times; the lemma's count, diagonal span and seeds on every listed alignment, 50 strands a mode with a positive bound, n-spaced at the bound exactly. 170 pairs (17 a generator, the CTO's ruling 5 of 20 October 2026); the 10 s ceiling read as the median of three CI readings for one head, then one pair a generator at a time to a floor of 140. Readings on 2aea997c (Fast chain run 39499120831, attempts 1 to 3, 20 October 2026): 6,008, 6,051 and 6,015 ms; median 6,015 ms. A fourth, not part of the median: 10,278 ms (CPU 14,753 ms) on 71823eb6, same src/ tree (run 39503684384). On 37cb7c00 after the D-0108 renumbering in method.ts (run 39515988212, attempts 1 to 3, 21 October 2026): 10,404, 7,992 and 6,309 ms; median 7,992 ms |
src/lib/seq/align/positions-off.test.ts (branch claude/sequence-dna-aligner) |
None: a sha256 taken from 6cf9cbf5's engine | With the per-position summary off, every result (at the defaults and with an exact count) on the fixture's 60 cases in each mode and 300 seeded pairs equals 6cf9cbf5's, the head the CSO read (her ruling 6 of 20 October 2026, condition 2) |
src/lib/seq/align/positions.test.ts (branch claude/sequence-dna-aligner; the summary is not read by the CSO and not used) |
dna-align-v1.json (Biopython's own alignments, for every strand it lists whole), and generated pairs |
The read check's per-position summary against states computed afresh from every enumerated alignment by the CSO's read-check ruling 2 (fixed and variable positions, common insertions with the same two flanking pairs, the varying end), zero tolerance: (a) 360 seeded tie-dense pairs, every mode, both strands, linear and circular across the origin, local starts, 1,874 strands; (b) the fixture's whole listings; (c) one strand of 1,296 alignments enumerated whole (the CTO's ruling 4 of 20 October 2026; the CSO's ruling 6.2). 6.3 s on CI (fast chain, run 39454787442, 20 October 2026), 5.0 s of it (b)'s 7j |
pipeline/sequence-toolkit/fixtures.test.ts |
All | Hashes, script hashes and counts (see below) |
src/lib/seq/readcheck/fixture.test.ts |
readcheck/readcheck-v1.json |
Every file's sha256 against the manifest; references A, B, C and A12 rebuilt from J01749.1 and their stated constructions, each equal to its recorded hash; for every review not refused, each read rebuilt from the fixture's own files, given the oracle's kept range, outcome and per-position summary, and countReads giving the oracle's aligned, supporting and disagreeing arrays, listings and totals exactly. It checks the counting rule; the placement (the aligner's positions.ts) is checked when it exists |
src/lib/seq/reads/fastq.test.ts |
reads/fastq-v1.json |
Reads, titles, ids, upper-cased bases and every quality value equal to Biopython's at the indices the set takes (floor(j × N / 1,000) above 1,000 reads); each read set's file order, receipt (N and taken per file, hash, size) and the reads taken, by digest, equal to the script's; refusal code and line exact with zero allocations; four allocations a read taken; the blank lines skipped equal the fixture's; a case carries differs exactly when ours and Biopython's outcomes differ |
src/lib/seq/reads/abif.test.ts |
reads/abif-v1.json |
Bases, peaks, values, channel order and trace digests equal to Biopython's abif_raw for the set chosen; the base caller equal to Biopython's SPAC2 and SVER2 strings; the kind, every reason and the edited-calls state exact; the tags not read equal Biopython's directory order less those read; refusal codes exact with zero allocations; one allocation a tag read; truncation at every seventh byte and single-byte corruption of the header and directory never throw and never allocate on a refusal |
src/lib/seq/reads/module.test.ts |
reads/manifest.json |
Every file in fixtures/reads/ listed with its SHA-256, every script hashed, every file some script's (the CTO's Q31) |
pipeline/sequence-toolkit/codes.test.ts |
ncbi/gc.prt |
src/lib/seq/codes-data.ts equals what build-codes.mts writes from it, byte for byte |
src/apps/sequence-toolkit/sequence.test.ts holds the badge constants in content/apps/sequence-toolkit.verification.ts equal to STRUCTURE_GATE in src/lib/seq/structure.ts, and makes the structure method "verified" only while both engines pass their gate.
Regenerating
From pipeline/sequence-toolkit/README.md and the scripts' headers:
python3 -m venv /tmp/seq-oracle && /tmp/seq-oracle/bin/pip install -r pipeline/sequence-toolkit/requirements.txt
/tmp/seq-oracle/bin/python pipeline/sequence-toolkit/reference.py fetch-inputs
/tmp/seq-oracle/bin/python pipeline/sequence-toolkit/reference.py fetch-protparam
curl -O https://ftp.ncbi.nlm.nih.gov/genomes/TOOLS/ORFfinder/linux-i64/ORFfinder.gz && gunzip -k ORFfinder.gz && chmod +x ORFfinder
/tmp/seq-oracle/bin/python pipeline/sequence-toolkit/reference.py run-orffinder --exe ./ORFfinder --gz ./ORFfinder.gz
/tmp/seq-oracle/bin/python pipeline/sequence-toolkit/reference.py fixtures
python3 pipeline/sequence-toolkit/peptides.py
python3 -I pipeline/sequence-toolkit/exact_matches.py
python3 -I pipeline/sequence-toolkit/revision-transforms.py
<venv>/bin/python -I pipeline/sequence-toolkit/align_reference.py # Biopython 1.88; refuses to overwrite different bytes
<venv>/bin/python -I pipeline/sequence-toolkit/primer_design.py # needs primer3-py for the comparison
<venv>/bin/python -I pipeline/sequence-toolkit/sdm_primers.py # --out PATH writes a draft without touching the manifest
python3 -I pipeline/sequence-toolkit/publish.py <version>
The first four steps fetch only what is not already saved (--refresh and --only P21 are documented). The README warns that a shared site-packages once shadowed Biopython 1.88 with 1.85 and the version check stopped the run; use a virtual environment. The README covers reference.py only; peptides.py, exact_matches.py, revision-transforms.py, align_reference.py and publish.py are documented in their own headers.
Plasmid
Methods are described in Plasmid. There is no README in pipeline/plasmid-map/; each script's header is the documentation.
Scripts and oracles
| Script | Writes | Oracle |
|---|---|---|
reference.py enzymes |
enzyme-check.json (223 rows), digests.json (446 cases); trio.json (4,157 judged digests, trio v1 under gel rules 1, written by enzymes until 17 October 2026 and kept byte for byte) |
Biopython 1.88 Bio.Restriction (REBASE emboss files release 404, offline only, never copied into the app's table), with an independent site scan; the golden set compared with REBASE pages read by hand |
reference.py gel |
gel-bands-v1.json (11 hand cases), digest-causes-v1.json (9 lanes: why a lane has no cut, from the script's own motif scan and stated methylation rules), trio-v2.json (4,157 judged digests under gel rules 2, with againstV1: 43 digests' bands changed, 0 verdicts changed) |
The CSO's grouping cases worked by hand with Python fractions (the script refuses to write unless its rule agrees with each hand answer); the trio from Biopython 1.88 Bio.Restriction as for enzymes |
reference.py features |
features.json (26 alignments), boundaries.json (12) |
Biopython Align.PairwiseAligner with the engine's stated scoring, against other NCBI records carrying the same parts |
reference.py display |
specificity-display.json (10 locus cases, 4 lac cases, the offset, repeat, suppression and tie cases) |
The same aligner, applying the two-step display rule per locus to sequences built from the library's own entries; it reports every call a feature makes, not only its best, because the rule is per locus |
reference.py absences |
methylation-absence-rule.json (223 enzymes, 82 whose site can carry a dam or dcm methylated base, 57 of them with a Table I row, 25 owed a dated line) |
A second, independent implementation of the rule deciding which enzymes are owed a dated Table I absence line, agreeing row by row with the generated list. It calls no Biopython, and its header and the registry name it as its own oracle ("Independent second implementation (reference.py can_carry)"); until 17 October the generic header stamp named Biopython (REL-456) |
reference.py specificity |
specificity-v2.json (12 controls, 30 short-feature cases, 47 family cases, 4,005 pairs compared and 66 reported; published with 0.14.0 together with the display and absence-rule fixtures) |
The same aligner against the library-2 manifest: controls that must call nothing, every feature under the 30-base floor five ways, and every family member against every other |
reference.py amounts |
amounts.json (11) |
Python fractions, exact |
reference.py methylation |
methylation.json (19) |
McClelland et al. 1994, Table I, each case worked by hand |
reference.py dna with make-dna-fixtures.py |
fixtures/dna/*.dna (6 synthetic, 4 refusal files), dna-files.json (10) |
Biopython's SnapGene reader; refusal reasons stated in the script, not taken from Biopython |
reference.py genbank |
genbank.json (2) |
Biopython's GenBank parser and its cds=True translation on the app's own exports in fixtures/genbank-samples/ (written by genbank-samples.mts) |
reference.py examples |
fixtures/examples/<id>@<n>.json and their manifest: puc19@1, pbr322@1 (library 1) and puc19@2, pbr322@2 (library 2, 20 October 2026) |
Every library call (Biopython PairwiseAligner), UniProt check, primer site (a brute-force scan of both strands in the script) and enzyme cut count on the pinned example sequence, with the library version the pin names (its ids read from that version's data file, each part's bases cut from its NCBI record by the script); the @2 files also list each call's mismatched bases. Write-once: a file on main is left as written; not part of all |
genbank_roundtrip.py |
src/lib/seq/features/fixtures/genbank-roundtrip-v2.json (write-once; genbank-roundtrip.json keeps the first fixed writer's reading) (1,200 values; not published, no manifest: it pins the shared GenBank writer, src/lib/seq/features/genbank-write.ts) |
Biopython 1.88's GenBank parser on the writer's round-trip corpus (genbank-roundtrip-corpus.ts, seeded): whether it parses, and the SHA-256 of each feature's qualifiers as read (batch 1 review, 14 October 2026); it refuses an empty or headless corpus rather than recording a refusal |
ds-ends.py |
ds-ends.json (5 ends, 8 compatibility, 3 flips) |
Overhangs worked by hand and re-derived from a two-strand drawing; refuses to write if they disagree |
restriction-cloning.py |
restriction-cloning.json (3 clones, 2 refusals) |
Products worked by hand and round-tripped by Bio.Restriction |
golden-gate.py |
golden-gate.json (3 assemblies, 4 refusals) |
BsaI and BsmBI assemblies built around a chosen scarless product; Bio.Restriction confirms overhangs and that no site survives |
gibson.py |
gibson.json (2 assemblies, 5 refusals) |
Products re-derived by independent concatenation; homology floor declared from Gibson et al. 2009 |
segment-map.py |
segment-map.json (7 assemblies) |
The correspondence, base by base, between an assembly product and the inputs it came from. Each product is built here from chosen inputs by the script's own construction, never from a junction record, so every segment is known from the coordinates that construction used; cut positions and overhangs come from Bio.Restriction and the enzymes' own published offsets. The script checks each segment against the product it wrote before it writes the fixture |
feature-transfer.py |
feature-transfer.json (9 assemblies, 43 carried features) |
Products written from chosen segment maps, every segment covering a base checked to agree about it; the carried features worked out by Biopython SeqFeature arithmetic with extract() equality asserted against the parent for every piece, and each product written as GenBank and read back. Two cases are metamorphic: the same assembly read from another origin, and with both inputs supplied on the other strand — the script refuses to write unless what is carried is unchanged |
restriction-from-map.py |
restriction-from-map.json (9 cases, 5 withheld) |
Cloning from two chosen sites: site positions, cut coordinates and overhangs from Bio.Restriction and the enzymes' published offsets, products written by the script's own slicing, dam and dcm marks found from McClelland et al. 1994. The uncertain-site case uses the one enzyme in the shipped table whose site must carry a dam mark and has no row in that table's hand transcription (AsiSI) |
golden-gate-multi.py |
golden-gate-multi.json (10 cases, 6 withheld) |
Each circle designed first, then the parts written with the flanks that release it, each piece checked to be released with its designed overhangs; the part cap covered on both sides of the boundary. No fidelity, mis-ligation or efficiency figure appears, and no datum of Potapov et al. 2018 |
crispr.py |
crispr.json (5 sequences, 21 guides) |
One regular-expression enumeration of NGG protospacers on both strands; one case worked by hand base by base, and the script refuses to write unless its search agrees |
Pins. pipeline/plasmid-map/requirements.txt: biopython==1.88. Every script exits if Biopython is not 1.88. Python 3.11.15 is recorded.
Consumers and tolerances
| Test | Fixture | Tolerance |
|---|---|---|
src/lib/seq/digest.test.ts |
Manifest hashes; enzyme-check.json, digests.json, trio-v2.json (and trio.json's pinned hash) |
Cut positions and fragment lengths exact; the picker's pool, every band and every verdict exact under gel rules 2; trio v2's count of verdicts changed against v1, the same distinguishing set, FIXTURE_COUNTS equal to the fixtures |
src/lib/seq/gel.test.ts |
gel-bands-v1.json |
Every hand case's bands, hidden sizes and may-not-resolve marks exact; the rules' version and fraction as the fixture states them |
src/lib/seq/digest-causes.test.ts |
digest-causes-v1.json |
Every lane's census per enzyme exact (DpnI on unmethylated pUC19 and pBR322: 15 and 22 sites needing methylation, none blocked), the census summing to the sites, off-end first, the drawing, the uncut flag for circles only |
src/lib/seq/ladders.test.ts |
pipeline/plasmid-map/ladders.csv |
The shipped ladders equal the hand-typed data file; every row sourced, dated and held ("To be checked"); no supplier or product word ships |
src/components/tools/plasmid-map/gel-lanes.test.ts |
pUC19 digests from the engine | Several digests and a ladder on one scale, the host per lane, the band table, the refusals (duplicate, full) |
src/components/tools/plasmid-map/figure/png-phys.test.ts |
A PNG built in the test | pHYs after IHDR at 11,811 pixels per metre for 300 dpi, replaced not duplicated, read back |
src/lib/seq/methylation.test.ts |
methylation.json |
Outcomes exact |
src/lib/seq/features/annotate.test.ts |
features.json, boundaries.json |
Identity and coverage within 0.1 percentage points; place, strand and call exact |
src/lib/seq/features/specificity-v2.test.ts |
specificity-display.json |
The per-locus display: which call is drawn where two meet, which are named beside it, and which are drawn inside a container. Includes the suppression case (a longer partial call drawn over a shorter exact one, both scores hand-computed from the scoring constants), the tie (a partial call inside an exact one, equal scores, the exact call drawn), the tandem-repeat case (one call drawn, five entries named once each) and the four lac cases |
src/lib/seq/features/specificity-v2.test.ts |
specificity-v2.json |
No match call on any control and at most two partial calls in all; every short feature exact, reverse-complemented, across the origin, mutated and truncated; one best call shown at a locus with the others named |
src/lib/seq/dna-amounts.test.ts |
amounts.json |
1e-12 relative |
src/lib/seq/features/snapgene.test.ts |
dna-files.json |
Sequence and topology identical, every feature recovered, refusals for the stated reason |
src/components/tools/plasmid-map/genbank-export.test.ts |
genbank.json |
Biopython's verdicts on this writer's output |
src/lib/seq/features/genbank-write.test.ts |
genbank-roundtrip-v2.json |
The corpus's bytes equal the recorded hash; Biopython parsed it; every value read back exactly by Biopython (hash) and by readGenBank |
src/lib/seq/plasmid-examples.test.ts |
fixtures/examples/ |
Exact, each file with its own library version; mismatch counts against mismatchesAt; the ColE1-type origin's spans on both plasmids and pUC19's mismatch at 1308 pinned |
src/lib/seq/ds-fragment.test.ts, restriction-cloning.test.ts, golden-gate.test.ts, gibson.test.ts, crispr.test.ts |
Their fixtures | Exact (products up to rotation and strand) |
src/lib/seq/segment-map.test.ts |
segment-map.json |
Segments exact; every segment also re-checked against the engine's own product base by base, and the covered-once and covered-twice counts exact |
src/lib/seq/feature-transfer.test.ts |
feature-transfer.json |
Carried features exact, qualifiers included; each parent's accounting exact |
src/lib/seq/restriction-from-map.test.ts |
restriction-from-map.json |
Refusal kinds and the positions they name exact; products up to rotation and strand; the qualifications exact |
src/lib/seq/golden-gate-multi.test.ts |
golden-gate-multi.json |
Refusal kinds exact, products up to rotation and strand, internal sites and their codons exact, the cap as the fixture records it; overhang pairs matched up to strand and compared on how many positions differ |
src/lib/seq/plasmid-data.test.ts |
Published copies | Byte-identical to the pipeline fixtures for the current version, including every golden file examples/manifest.json lists |
Regenerating
pip install -r pipeline/plasmid-map/requirements.txt
node pipeline/plasmid-map/genbank-samples.mts
node src/lib/seq/features/genbank-roundtrip-corpus.ts > /tmp/corpus.gb && python -I pipeline/plasmid-map/genbank_roundtrip.py /tmp/corpus.gb
python pipeline/plasmid-map/reference.py # all: enzymes, features, amounts, methylation, dna, genbank
python pipeline/plasmid-map/reference.py examples
python3 -I pipeline/plasmid-map/ds-ends.py
python3 -I pipeline/plasmid-map/restriction-cloning.py
python3 -I pipeline/plasmid-map/golden-gate.py
python3 -I pipeline/plasmid-map/gibson.py
python3 -I pipeline/plasmid-map/segment-map.py
python3 -I pipeline/plasmid-map/feature-transfer.py
python3 -I pipeline/plasmid-map/restriction-from-map.py
python3 -I pipeline/plasmid-map/golden-gate-multi.py
python3 -I pipeline/plasmid-map/crispr.py
python3 -I pipeline/plasmid-map/publish.py <version>
reference.py always rewrites the manifest's top-level scriptSha256 and recordedOn, whichever sub-command ran. The per-file about.scriptSha256 values show that the committed fixtures were written by four different revisions of the script (only genbank.json matches the current one), so the manifest's script hash does not mean every file came from that revision.
Each fixture keeps the library it was written against. The 0.12.0 annotation cases (features.json), the GenBank writer samples (genbank.json, from pipeline/plasmid-map/genbank-samples.mts, SAMPLE_LIBRARY) and the golden examples (examples/puc19@1.json, examples/pbr322@1.json, whose pins name it) were all written with annotation library 1, and their tests annotate with library 1 by name (loadLibraryFile("1") in src/lib/seq/plasmid-test-data.ts) rather than with the current version. A published verification set keeps its answers when a newer library ships, as a map link pinned to lib=1 does. Library 2's own expectations are the specificity v2 and display fixtures and the library-2 goldens examples/puc19@2.json and examples/pbr322@2.json, new fixtures written on 20 October 2026 rather than a rerun of the @1 files (the CSO's ruling 4, board/memos/2026-10-20-cso-puc-ori-call.md). This was found on 17 October 2026, when building library 2 turned five such tests red because they read whichever library was current; on 20 October the docs page's fixture tables were found checking library-1 goldens against library 2 the same way, and now name the library each set is checked against.
A constructed library, for one branch the shipped library cannot reach
src/lib/seq/features/constructed-geometry.test-util.ts holds a constructed two-entry library used by one test: the different-element branch where two calls overlap at a locus without either containing the other. The shipped library cannot exhibit that geometry — every marked different-element pair in it is strictly nested — so the branch would otherwise be unexercised, and that branch draws both calls whatever the geometry, so a fixture that drifted into containment would still pass a both-shown assertion.
It is a rule fixture, not a verification-dataset one. A verification fixture says what the shipped library does on real sequences and nothing in it may be constructed; a rule fixture says a stated rule's branch behaves as stated, given inputs in the rule's own domain, which here is two spans and a marking. The conditions on it are in the file's own header and are enforced rather than promised: it is not a library version and is never indexed with or substituted for the shipped library, it is excluded from every library count, it carries no accession, source location, retrieval date or licence line, not even placeholders, its entries are named for the roles they play in the rule, and src/lib/test-util-imports.test.ts fails if any shipped module imports it.
The bounded claim it rests on, which is not a general theorem: no partial-overlap different-element case is constructible from this library's three such pairs (sv40Ori/sv40Promoter, rnaI/pucOri, lacO/lacPromoter), as of library 2 on 17 October 2026. reference.py display tries all three, records each attempt and its outcome in specificity-display.json, and exits if one ever becomes constructible. Nested pairs can partially overlap in principle: containment here is of annotated extent rather than always of literal sequence (one of the three is 106 of 107 bases, not an exact substring), and a local alignment with affine gaps can end early inside a container's footprint. When a future library makes the case constructible, the real case joins the verification dataset with the oracle's coordinates and this file is kept or retired on the record — the guard is never relaxed.
Molecule
The Molecule app's engine is RDKit.js (MinimalLib) 2026.03.6 plus TypeScript ports of QED and PAINS, which MinimalLib lacks. See Molecule and Engines.
Script. pipeline/molecule-workbench/rdkit_reference.py, sub-commands pains (writes src/components/tools/molecule-workbench/pains-data.ts and public/tools/molecule-workbench/qed-notice.txt), reference (writes src/components/tools/molecule-workbench/fixtures/rdkit-reference.json, 30 molecules) and library <out.json> (every molecule in public/data/molecule-workbench/approved-drugs.json). It exits unless RDKit is 2026.03.6. There is no requirements file; the README and workflow install rdkit==2026.3.6.
What it checks. The oracle is Python RDKit of the same release as the shipped RDKit.js, so for most descriptors it checks the WebAssembly build and the suite's wrapping, not an independent implementation. It is an independent check of the two ports (QED from rdkit/Chem/QED.py, PAINS with explicit hydrogens merged as FilterCatalog does) and of the suite's formula function.
Consumers and tolerances.
| Check | Tolerance |
|---|---|
src/components/tools/molecule-workbench/chem.test.ts on the 30 reference molecules |
Canonical SMILES, formula, HBD/HBA (generic and historical), rotatable bonds, rings, heavy atoms, charge, PAINS names and Morgan bits exact; masses, cLogP, TPSA, Fsp3 and the eight QED inputs within 1e-6; QED within 1e-9 |
pipeline/molecule-workbench/verify-library.mts on all 2,197 library molecules |
The same, except QED is gated at 0.01 (the largest difference is printed; 2.78e-16 on 10 October 2026) |
The library check runs weekly (see Scheduled re-checks). The PCA map was checked once against an independent NumPy eigendecomposition; only the result is committed (docs/science/evidence/molecule-pca-reference.json), not the script. Molecule publishes the bounded ten-query similarity set below; descriptor and map checks are separate. Since the CSO's Wave 4 condition C3, rdkit_reference.py passes every Morgan generator argument explicitly, as the policy oracle does; no fixture changed.
Fingerprint policy and same-compound rule (Wave 4). pipeline/molecule-workbench/fingerprint_policy.py (Python RDKit, EXPECTED_RDKIT = "2026.03.6", refuses another release) re-implements policy morgan-r2-2048-described-v1 and the CSO's ruling 2 (the same compound: described fragments sharing a full standard InChIKey, else identical canonical SMILES) from their documentation, ports prepareInput's described-fragment rule from its stated behaviour, and imports no TypeScript. Its sub-commands wrote three write-once fixtures in pipeline/molecule-workbench/fixtures/, committed (091523d) before the engine changed: library-multifragment-bits.json (the 75 multi-fragment library rows: described fragment and bits), library-same-compound.json (26 index pairs) and neighbours-under-policy.json (for the default example and the ruling-2 cases, the forms left out and the top ten with shared and union bit counts). A key spike (docs/science/evidence/molecule-fingerprint-policy-2026-10-10/, Python and RDKit.js) showed onlyNonzeroInvariants and includeRingMembership change no bit on 2,205 molecules, so neither is passed.
Published similarity reference (23 October 2026). pipeline/molecule-workbench/publish.py 0.3.0 imports the unchanged Python policy oracle, recalculates all ten queries in memory from the pinned library SMILES, and requires byte-identical serialization before copying the existing neighbour fixture unchanged to public/verification/molecule-workbench/0.3.0/neighbours-under-policy.json. Its manifest records hashes of the original fixture, oracle, publisher, actual library and fixed PubChem-input receipt; the Python version; publication/reproduction dates; library release/retrieval/build receipts; source licences; and an independent Chem.GetMolFrags count. The original generation date was not recorded and is explicitly null. --check repeats the calculation and checks the recorded hashes without writing. Existing destinations are refused. This is an implementation check using the same RDKit release, not independent experimental evidence or verification of every app method.
The registry retains app version 0.2.0 and explicitly publishes verification version 0.3.0 for molecule.morgan-tanimoto, with ten queries, each retaining its full top ten. pipeline/molecule-workbench/publication.test.ts binds the original/public bytes and hashes, all ten generated Erlotinib neighbours (ordered IDs, documented sentence-case/trailing-anhydrous names and six-decimal scores), input identity, dropped HCl, excluded parent, library metadata and fingerprint policy. It also checks actual RDKit.js query bits, described structures, excluded identities and unrounded rank scores for all ten queries (exact identities/ranks, Tanimoto within 1e-12). The six-decimal display values are never used as the full-precision engine reference. The existing policy.test.ts remains unchanged.
Reproduction executed on 23 October 2026 with /workspace/.tools/rdkit-2026.3.6/bin/python pipeline/molecule-workbench/publish.py 0.3.0, then the same command with --check: all ten query payloads reproduced byte for byte. A portable command uses any standalone interpreter with rdkit==2026.3.6; the script refuses another RDKit release. ChEMBL 37/EMBL-EBI CC BY-SA 3.0 share-alike and PubChem depositor terms remain attached. Publication does not resolve the existing held legal questions.
Series (Wave 4). series_reference.py (same pin, reusing the policy oracle's functions) wrote fixtures/series-example.json: a made-up series covering a salt and its parent, a tautomer pair standard InChI merges and a keto-enol pair it does not, stereoisomers, an unreadable SMILES, a duplicate label and censored values, with each row's identities, descriptors, three nearest drugs (forms left out), the within-series matrix, the same-compound pairs and hand pIC50 values.
The walk's values. pipeline/walks/nearest/walk_values.py (same pin) writes content/walks/nearest.values.ts and nearest.series.ts from the policy oracle, the library and the write-once ChEMBL fixture pipeline/walks/nearest/fixtures/chembl1134804-egfr-ic50@ChEMBL_37.json (written by fetch-series.mts --expect-release ChEMBL_37, which stops if the API serves another release). It refuses to run when the compound is a library row as a whole, when it has no same-compound row, or when the series and the library are from different releases. These are generated content, not fixtures: content/walks/nearest.test.ts checks that the words use only them.
| Check (Wave 4) | Tolerance |
|---|---|
policy.test.ts: described fragments and bits of the 75 rows; sameCompound pairs; forms and top ten of the oracle's queries |
Bits, pairs, forms and ranks exact; Tanimoto 1e-12 (the neighbour engine check does not directly assert shared/union integers) |
series.test.ts: each row of series-example.json |
Identities and counts exact; MW, cLogP, TPSA 1e-6; QED 1e-9; pIC50 1e-9; matrix shared and union counts exact |
Protein: AlphaMissense, substitution and alignment
Wave 4 gave Protein its first independent oracles (D-0055 for the aligner). All are Python, import no TypeScript, write under pipeline/protein-explorer/fixtures/ and refuse to overwrite an existing target. Those Wave 4 sets remain unpublished. Feature 9 adds a separate retained-image publication and registry declaration, described below. See Protein.
| Script | Oracle and inputs | Fixtures | Consumer and tolerance |
|---|---|---|---|
alphamissense_reference.py |
Downloaded five AlphaFold DB entries (BRAF P15056, EGFR P00533, SLC3A2 P08195, RAB12 Q6IQ22, POLR2A P24928) with their AlphaMissense CSVs and UniProt sequences, stored gzipped with sha256 recorded (AlphaFold DB refuses a default user agent, so a browser-like one is sent); standard library only | alphamissense/<acc>@2026-10-11/expected.json (rows, mismatches, placement offset, per residue the count and mean) |
alphamissense.test.ts: n exact, mean within 1e-6, POLR2A's withholding (38 mismatching rows; count and first) exact |
substitution_reference.py |
extract streamed the Zenodo bulk file AlphaMissense_aa_substitutions.tsv.gz (record 8208688, 1,207,278,510 bytes) and kept the ten rows the cases need with its header lines and sha256 (alphamissense-source/2026-10-11/); expected reads those, the step 2 fixtures and 25 typed inputs |
substitution/checks@2026-10-11.json |
substitution.test.ts (31 tests): every case's outcome exactly (BRAF V600E in five notations 0.9927; V599E refused; EGFR L858R 0.9968, T790M 0.966; L834R refused; offset models scored at the model's numbering; Q6IQ22 M1V withheld; POLR2A withheld; every refusal kind); each shown score also found in the source rows with its text and class |
blosum62.py |
NCBI's BLOSUM62 file (stored as blosum62/ncbi-BLOSUM62@2026-10-11.txt, sha256 85510d38…efb40) checked against Biopython 1.88's copy: 576 entries, no difference, symmetric |
blosum62/ncbi-BLOSUM62@2026-10-11.check.json; generates src/lib/protein/align/blosum62.ts |
align.test.ts re-reads the stored file: hash, every entry, symmetry |
align_reference.py (--set ties, --set adjacent) |
Biopython 1.88 PairwiseAligner: BLOSUM62 from Biopython, open_gap_score −10, extend_gap_score −0.5, end gaps 0 in the free mode; modes global, global with end gaps penalised, local. UniProt JSON inputs stored write-once (release 2026_03, with sequence version, MD5, CRC64). Where a set of co-optimal alignments is at most 20,000 it is enumerated |
alignment/2026-10-11/ (nine UniProt pairs including two refusals), 2026-10-11-ties/ (three tie-heavy literals, up to 8 co-optimal), 2026-10-11-adjacent/ (the CSO's C2 pairs) |
align.test.ts (45 tests): score exactly (doubled integers); the shown alignment re-scored by the test's own scorer to the optimum; its counts among the enumerated tuples; the co-optimal count equal; the unique residue pairs equal to the intersection of the enumerated alignments as sets (global modes) |
EMBOSS Needle cross-check (CSO condition C1). EMBOSS needle 6.6.0.0 (GPL; build-time only; run locally, not on EBI's service) on four runs with EBLOSUM62, gap open 10.0, extend 0.5: TP53 human against mouse (1558.0; identity 304, similarity 326, gaps 9 of 396), the EGFR construct against P00533 with end gaps free (1711.0) and charged (1292.5), and the P01275 peptides (88.0). Every figure matched the CSO's expectations, Biopython's tuples and the engine. The commands, inputs and outputs are in docs/science/evidence/protein-align-2026-10-11/. Needle reports one alignment, so the co-optimal results rest on Biopython alone. The engine's copy states the convention in words because the licence scan forbids naming EMBOSS in src/ and content/.
align_reference.py records the Biopython version it ran but does not refuse another; alphamissense_reference.py and substitution_reference.py use the standard library only. No unit test re-checks the sha256 values recorded for the gzipped AlphaMissense and UniProt inputs, and those .gz files fall outside the write-once rule (which covers .json and .csv).
Protein loaded experimental correspondence reference (Stage F F2)
pipeline/protein-explorer/experimental-map/ freezes the independent source
reference before live F2 consumers. publish_v1.py --check verifies pinned
inputs, redecodes retained BCIF with Mol*5.11.0, recomputes scientific
expectations from Python3.12/Biopython1.88 text-CIF/XML extraction, runs24
Python and seven resource checks, and compares every publication hash.
The three experimental-*-v1.json fixtures contain46 correspondence cases,
46 strict inner/tamper cases,216 nominal plus35 typed version negatives,
15 UTF-8 wire cases and23 synthetic application relations. Publication
/verification/protein-explorer/0.7.0/ retains all required source bytes,
original metadata, methods and pinned tools; it is a dataset version, not
an app release. Existing publications remain unchanged.
0.10.0 (25 October 2026, MNT-206) republishes that tooling readably from
experimental-map/readable/: Black and Prettier formatting with identical
syntax trees (one docstring's indentation and an unused binding aside) and
the limits named once in limits-v1.json. Its publish_v1.py --check
requires the outputs to equal 0.7.0's bytes; the manifest copies only the
tools and names each 0.7.0 output and input by path and SHA-256 (committed-
growth rule B3). Replay is recorded on CPython 3.12.3 and 3.13.17 with
Biopython 1.88. readable/republication.test.ts checks the publication.
Real anchors include3IKA P00533T790→deposited MET at labelA99/authA790, qualified by the source's exact Engineered mutation annotation; exact I789; unobserved P694; and1PPB Y410→labelB47/authH60A. Labels and authors remain distinct. Related CIF/XML are not independent biological truth. Modified/unknown, current-source mismatch, nonunique/conflicting and unobserved rows withhold. Source terms stay To be checked; no new runtime host, licence clearance or native applied-focus event is inferred. The method file documents conservative encoded-byte/row bounds, all-known author comparisons and the historical/predicted/new experimental receipt matrix. Native viewer, lifecycle, complete capture and live consumer acceptance are subsequent work.
The frozen F2 Node tools use CommonJS. eslint.config.mjs identifies the
three exact pipeline .cjs paths and their three byte-identical public
copies as CommonJS and disables only no-require-imports on those six
files. Every other lint rule stays active. The two unused-crypto
warnings in the frozen resource script and its mirror remain visible;
publication bytes are preserved. This is a dialect correction, not a
directory ignore or a change to product lint rules.
Protein gap-aware correspondence reference (Stage F)
Levels on the methods sheet (25 October 2026, the CSO's ruling, SCI-245). protein.residue-map@1.0.0 is "verified" against this reference (0.5.1, 47 cases) and the counting reference (0.8.0, 8 cases); protein.source-confidence@1.0.0 has its own method record, "verified" against the confidence reference (0.6.0, 102 cases); capture 3.0.0 is "unit-tested". The references were written by the app's authors, separately from the runtime; the method notes say so.
pipeline/protein-explorer/residue-map/reference_v2.py is an offline
Biopython 1.88 PairwiseAligner oracle for method
protein.residue-map@1.0.0, policy standard-invariant-free-ends-v1.
It imports no TypeScript and enumerates every optimum under the existing
D1 global/free-end BLOSUM62 policy before intersecting residue-pair sets.
Above 20,000 optima it errors; a partial intersection is never published.
It independently classifies invariant pairs against the 20 standard amino
acids: exact identity, standard mismatch, or withheld correspondence.
B/Z/X never enter its exact-transfer map. Expected partners, forward/reverse
states, null transfer masks, refusals and doubled scores compare exactly.
The fixture pipeline/protein-explorer/fixtures/residue-map-v2.json is
committed before implementation. Its source inputs are explicit synthetic
strings plus the first 100 residues of the frozen P00533 response already
used by D1 (source hash retained). Cases include equal-length compensating
indels; terminal changes in both directions; the 100/101 insertion after
95; adverse adjacent gaps; B/Z/X against themselves and standard letters;
complete withholding; unavailable/invalid inputs; the individual length and
cell limits; and 4,096 completely enumerated optima. W/D is a hand-checkable
policy control: two free-end opposite-gap alignments with no invariant pair,
versus one charged-end mismatching pair. Non-ASCII inputs are refused before
Unicode case conversion can alter coordinates; a surrogate-pair example
also checks raw UTF-16 stop offsets. These references do not establish
biological equivalence or live annotation-transfer correctness.
pipeline/protein-explorer/residue-map/reference_v3.py (verification 0.8.0,
residue-map-v3.json, CSO ruling of 24 October 2026,
docs/board/memos/2026-10-24-cso-protein-optima-cap.md) is a second,
non-enumerating oracle for the same method. It is written from method-v1.md
in the Python standard library, with Biopython 1.88 only for BLOSUM62 and the
optimal score. It runs a three-state affine-gap recursion with free boundary
gaps on doubled integer scores, keeps forward and backward best scores for
every node, and counts optimal paths with Python integers only for nodes on an
optimal path. A pair is invariant exactly when the optimal paths through its
diagonal step equal the total. generate and check first reproduce all 47
frozen v2 cases (mapped and refused) exactly. Its eight cases are:
- O1 (4^7) and O2 (20,000), where full enumeration must agree;
- U1 (4^8), where enumeration must refuse;
- U2 (4^40, beyond 2^64);
- U3, 3,000 by 3,000 with 5^499 optima;
- three real AlphaFold DB/UniProt pairs (Q13421, Q9NQ90, P43251), whose UniProt sequences were revised after the model.
The real pairs' raw responses are pinned by SHA-256 in
fixtures/residue-map-v3-inputs/pins.jsonand are never downloaded at check time. Counts are stored as decimal strings. Biopython's count is recorded where it can give one: it raisesOverflowErrorbeyond 2^63. The oracle runs offline only, never in CI (CTO memo of 24 October 2026 on the desktop-dark project, section 2).generaterecords its wall time, peak RSS, CPU and versions inabout.run: 113.7 s and 336 MiB on 4 cores. Each case stores its sequences' SHA-256. The runtime keeps no count and no cap.src/lib/protein/residue-map-v3.test.tscompares its partners, states, exact maps and null masks with every v3 case. U3 runs in about 0.4 s against the CTO's Fast-chain budget of 3 s and 512 MB.reference_v3.test.tsproves U3 from stored data without recomputing: - the sequence hashes;
- 8,000,000 to 10,000,000 cells, within
CELL_LIMIT; - a decimal count of at least 10^100, equal to the closed form 5^499;
- Biopython's score equal to the oracle's;
- one-to-one partners, an exact W anchor and an unpaired Q run.
Published files under /verification/protein-explorer/0.5.1/ are the JSON
fixture, frozen reference_v2.py and a manifest of byte lengths, SHA-256s
and source paths. reference_v2.test.ts verifies published/source bytes and
the frozen script and UniProt-input hashes. The offline Python tests
test_reference_v2.py check the hand example, cap refusal, ambiguity,
coordinate preservation, write-once generation/publication, path traversal
refusal and full recomputation before publication. Request/expected envelopes
retain raw and normalised sequences and optional synthetic provenance;
unchanged metadata with different actual sequence text produces a different map.
The publication is not added to the live registry in this foundation.
Consumer src/lib/protein/residue-map.test.ts checks all 47 v2 outcomes
exactly against buildResidueMap: states, both raw partner arrays, both null
transfer masks, raw/normalised envelopes, lengths, resource accounting and
refusals. It separately asserts the historical 96→97 through 100→101 suffix,
reciprocal maps, standard-only transfer, metadata snapshots, the accepted
1,000 × 10,000-cell bound and the absence of runtime consumers. No expected
scientific map is generated from the TypeScript implementation.
The initial residue-map-v1.json, reference_v1.py and publication 0.5.0
were generated before pre-freeze review found the metadata coverage and
publisher-hardening gaps. They remain byte-for-byte chronology evidence,
checked by the v1 integrity test. Their numerical policy is unchanged.
The v1 publish entrypoint is withdrawn: it lacks version-path validation
and case recomputation. Use v2 for reproduction and future publication.
python pipeline/protein-explorer/residue-map/reference_v2.py check
python -m unittest discover -s pipeline/protein-explorer/residue-map -p 'test_*.py'
generate --out NEW_FILE and publish --version NEW_VERSION refuse existing
targets; publication also refuses a version already on origin/main.
Biopython 1.88 is required only for offline tooling, with its environment
versions recorded in the fixture. No npm dependency is added.
Pipeline: coverage oracle (Wave 4)
pipeline/drug-pipeline-explorer/coverage_oracle.py re-implements, from their documentation, the partial-date rule, the phase buckets and the sponsor-alias rules, and counts over a frozen ClinicalTrials.gov search (freeze-ctgov.mts: fixtures/ctgov/cystic-fibrosis@2026-10-11/, 751 trials, API 2.0.5, with a manifest.json of sha256s): coverage, start year × phase × date type, undated, sponsors with and without individual-class lead sponsors. It writes <dir>.expected.json and refuses to overwrite. src/components/tools/drug-pipeline-explorer/coverage.test.ts requires exact equality and asserts the fixture holds no imaging-only trial (the oracle counts all trials; the app's timeline leaves imaging-only trials out). The test does not re-check the manifest's hashes; the files are under fixtures/, so write-once covers them. See Pipeline.
Pipeline: precedent transcription fixtures
The Pipeline app's FDA precedent panel has fixtures that are transcriptions from FDA's own documents rather than computations: pipeline/drug-pipeline-explorer/precedent/fixtures/build_products.py, build_crl.py, build_section14.py, build_examples.py (a second implementation in Python reading FDA's spreadsheet) and build_names.py, pinned by precedent/requirements.txt (openpyxl==3.1.5, pypdf==6.19.0). They write src/components/tools/drug-pipeline-explorer/precedent/fixtures/*.json, read by nme.test.ts, crl.test.ts, pivotal.test.ts and examples.test.ts in the same directory, with exact tolerances. The precedent README marks all of them drafts, "transcribed in the build session on 5 October 2026 and not independently checked", awaiting a regulatory-affairs reviewer under Q-19. content/apps/drug-pipeline-explorer.verification.ts describes the sets. Commands are in pipeline/drug-pipeline-explorer/precedent/README.md; see Pipeline and Data pipeline.
Patent: the rule engine against a Python second implementation
The Patent app's engine (src/lib/exclusivity/: parse.ts, index.ts, trace.ts, dates.ts) has a second implementation that shares no code, pipeline/patent-exclusivity/trace_reference.py (standard-library Python): the same parsers, the same join by application key, the same seven rules and the same summary set. The fixture is pipeline/patent-exclusivity/fixtures/synthetic/ (products, patents, exclusivities, a Purple Book CSV and a Paragraph IV list in the FDA file layouts, written through capture.py's normalisation, with one change-day record for the Changes view). Its rows are invented: the README says so and the store built from it is marked synthetic. The Python writes expected.json (the about block records the day the trace is read for, the code list's date, the code definitions and the SHA-256 of every input file); src/lib/exclusivity/reference.test.ts checks the input hashes and then compares all seven applications field by field: every rule date with its row citations, the summary, NCE-1, the Paragraph IV rows, the flagged patents and the notes. Dates compare exactly (ISO days; nothing is floating point). The fixture exercises the rules the CSO's memo names: one entry per patent, pediatric flag and product set (R1); the *PED row as listed and a product with pediatric exclusivity listed but no *PED row as the negative control (R2); a code outside the FDA's list rendered as unknown (R3); NCE-1 as the only computed date (R4); a Paragraph IV row by application number and one joined by name and flagged (R5); delist-requested and use-code patents kept and flagged (R7); a date the FDA wrote in words kept as text; a biologic's Purple Book dates and a biosimilar whose reference product's date stays on its row. dates.test.ts, parse.test.ts, index.test.ts, trace.test.ts and store.test.ts pin the parts (46 tests). Once on main, expected.json, the CSV and the change-day record are write-once under the fixtures pattern; a changed fixture is a new directory. The real fixtures F1 to F7 (the brief §6) are transcribed from the FDA's files of a stated date when the first capture lands, and a regulatory-affairs reviewer's signature on them is a release gate (Q-19, D-0077).
The second fixture, pipeline/patent-exclusivity/fixtures/ped-inheritance/ (SCI-125, 14 October 2026), is mixed: every row of application N 202155 (Eliquis) and the apixaban Paragraph IV row are verbatim from the FDA's files as the capture kept them on 14 October 2026 (the README gives each captured file's SHA-256 and the data branch's commit), and two invented applications (N 999901, N 999902) carry the edge cases the real rows lack: base rows that disagree, a *PED row whose base row is on another product, a *PED row with no base row, and a product-only inheritance with a use code. trace_reference.py --fixture … --statement … writes its expected.json; src/lib/exclusivity/reference-ped.test.ts compares every entry with its inheritance (the flags taken, the base rows' use codes, the base rows cited), the summary, the notes, the inherited use codes and the Paragraph IV row (its date read from the spreadsheet's day number). Write-once on main like the first.
Cellosaurus: cell line resolution
The cell line passport (E10, D-0099) shows no computed number; what it must get right is which record a name or an accession names. The CSO set the rule and the threshold on 17 October 2026 (docs/board/memos/2026-10-17-cso-enterprise-review.md, "F5"): by accession (CVCL_), exact, 100 per cent on the fixture; by name, exact on the primary name or a listed synonym after case, whitespace and hyphen normalisation only, with no fuzzy matching; a normalised name shared by more than one record returns the candidates and chooses none; zero wrong assignments, unresolved permitted and counted; a misspelling comes back unresolved.
- Oracle.
pipeline/cellosaurus/oracle.py, standard-library Python, reads the Cellosaurus release file itself with its own parser and its own normalisation. It imports nothing fromsrc/or frombuild-snapshot.pyand never reads the served snapshot, so the snapshot builder, the shards and the engine are all under test. It refuses a release file that differs from its fetch record, refuses to overwrite a fixture, checks seven hand-worked cases (HAND_WORKED: HeLa, HEK293, MCF-7/ADR to NCI-ADR-RES, HeLa S3, a secondary accession, HT-29 and U87MG as shared names) and refuses to write unless the three misspellings resolve to nothing. - Fixture.
pipeline/cellosaurus/fixtures/resolution-56.0.jsonwithfixtures/manifest.json(SHA-256, script hash, counts), written on 17 October 2026 with CPython 3.13.16 from release 56.0 (the file's SHA-256 is in itsabout.release). 100 cases: 50 named lines (HEK293, HeLa, CHO-K1, HCC827, 31 other common lines and 15 lines Cellosaurus annotates as problematic) and 50 adversarial names (case, space and hyphen variants, Unicode hyphens and a no-break space, near-duplicates such as MCF-7/ADR, HeLa S3 and Vero E6, shared names such as PC-3, HT-29, U87MG and C2, accessions in upper and lower case, a secondary, a deleted, a malformed and an absent accession, and the misspellings HEK239, MFC-7 and Jurcat, which Cellosaurus lists as misspellings and not as synonyms). Expected: 84 resolved, 10 ambiguous, 6 unresolved; 25 of the resolved records carry a "Problematic cell line" comment. Each expected record has its name, species, diseases, sex, age, category, problematic comments verbatim and the count of STR marker lines. - Integrity.
src/lib/cells/snapshot.test.tsasserts the fixture's hash against the manifest,oracle.py's hash against the current entry'sscriptSha256, that the oracle read the same release file as the snapshot, and that the served release has a fixture. The files are underfixtures/, so write-once covers them; a new release gets a new fixture file.
Apps with no numerical oracle
- Protein: since Wave 4 it has the AlphaMissense, substitution and alignment oracles above; the rest of the app (structure ranking, placement, pLDDT, drugs) has no oracle beyond the ProtParam values.
pipeline/protein-explorer/verify.tschecks the live services, not numbers. - Cell:
pipeline/inside-a-cell/verify_sources.pychecks that cited structures and papers still resolve; no computed value is compared. - Education:
pipeline/education/verify.pychecks the rendered media files; the Education tools' model tests have no external oracle (audit M1 lists "Fourteen Education tools" with "27 model tests" and no fixture). See Education. pipeline/walks/j3/holds candidate data sets and sources for the J3 walk; nothing there is an oracle.pipeline/walks/nearest/holds the nearest-drugs walk's ChEMBL series fixture andwalk_values.py, which generates the walk's values from the Molecule policy oracle (above).
Fixture integrity checks
Three mechanisms keep a committed fixture tied to the script that wrote it.
Hash manifests checked by unit tests
src/lib/fixture-hashes.test.ts is the Calc variant. It reads pipeline/lab-calculators/fixture-hashes.json (written by reference.py publish and publish_preparation.py), which maps each fixture path to sha256, published, count, recordedOn, oracle and oracleVersion, and, for the twelve later files, their own version, script and scriptSha256. For each of the sixteen files it asserts:
- the file's SHA-256 equals the recorded one ("was edited after its script wrote it");
publishedispublic/verification/lab-calculators/<version>/<name>, with the entry's ownversionor else the manifest's;- the published copy is byte-identical to the fixture;
countis at least 10 and the oracle, version and ISO date are present;- for an entry with its own version, the script's SHA-256 equals
scriptSha256and that version's publishedmanifest.jsonlists the file with the same hash.
src/lib/fit/oracle-fixtures.test.ts does the same for pipeline/assay-analyser/oracles/fixtures/manifest.json: the manifest lists exactly the fixture files; each file's and each script's SHA-256 match; the fixture's own about.scriptSha256 agrees; the copy under public/verification/assay-analyser/<manifest version>/ is the same bytes and its manifest entry is identical.
The other apps follow the same pattern with a manifest.json beside their fixtures:
| Manifest | Files hashed | Script hash asserted by a test | Published copy asserted |
|---|---|---|---|
pipeline/lab-calculators/fixture-hashes.json |
20 | Yes for the sixteen 0.2.0 to 0.6.0 entries; none recorded for the four 0.1.0 entries | Yes, each in its own version (src/lib/fixture-hashes.test.ts) |
pipeline/assay-analyser/fixtures/manifest.json |
8 (not relative-irls-reference.json) |
reference.R and generate_curves.py (src/lib/fit/fixtures.test.ts); relative-irls-reference.py and its input (src/lib/fit/irls.test.ts) |
No |
pipeline/assay-analyser/oracles/fixtures/manifest.json |
15 | Yes, every script (src/lib/fit/oracle-fixtures.test.ts); the five -v2 lineages also by src/lib/fit/oracle-lineage.test.ts |
Yes, ten under 0.4.0, all fifteen under 0.5.0 |
pipeline/assay-analyser/fixtures/project-files/manifest.json |
6 (the MTT session file, records v1 to v5; engine-written) | n/a | No (pipeline/assay-analyser/project-files.test.ts checks the hashes) |
pipeline/assay-analyser/walk/manifest.json |
1 (plate-walk-refit.json) |
Checked by src/lib/fit/fixtures.test.ts with the row |
Published under 0.3.0 |
pipeline/bioreactor-simulator/fixtures/manifest.json |
7 | No (recorded, not asserted) | Yes, 0.2.0 hard-coded (src/lib/hosts/fixtures.test.ts:199) |
public/verification/bioreactor-simulator/0.3.0/manifest.json, 0.4.0/manifest.json, 0.5.0/manifest.json |
1 each (measured-comparison-0.3.0.json, -0.4.0.json, -0.5.0.json) |
Yes, with the fixture's own about.generatorSha256 (src/apps/bioreactor-simulator/measured-comparison.test.ts; measured-comparison-completion.test.ts for 0.5.0) |
Yes, byte-identical to the pipeline fixture |
public/verification/bioreactor-simulator/0.6.0/manifest.json, 0.7.0/manifest.json, 0.8.0/manifest.json |
1 each (kla-scaling-0.6.0.json, feed-0.7.0.json, transport-0.8.0.json) |
Yes, with the fixture's own about.generatorSha256 (src/lib/hosts/kla-scaling-oracle.test.ts, feed-oracle.test.ts, transport-oracle-0.8.0.test.ts) |
Yes, byte-identical to the pipeline fixture |
public/verification/bioreactor-simulator/0.9.0/manifest.json, 0.10.0/manifest.json, 0.11.0/manifest.json, 0.13.0/manifest.json |
1 each (batch-fit-0.9.0.json, batch-fit-flags-0.10.0.json, batch-fit-flags-0.11.0.json, batch-fit-0.13.0.json) |
Yes, with the fixture's own about.generatorSha256 (src/lib/hosts/batch-fit-fixture.test.ts; src/lib/hosts/fit/batch-fit-oracle.test.ts for 0.10.0 and 0.11.0, with 0.9.0's SHA-256 as derivedFrom and 0.10.0's as 0.11.0's supersedes; batch-fit-oracle-0-13-0.test.ts for 0.13.0, with the 0.9.0 reference's and fixture's SHA-256); the engine names 0.9.0's, 0.11.0's and 0.13.0's manifests' SHA-256 in BATCH_FIT_VERIFICATION |
Yes, byte-identical to the pipeline fixture |
pipeline/sequence-toolkit/fixtures/manifest.json |
13 | reference.py, peptides.py, exact_matches.py (pipeline/sequence-toolkit/fixtures.test.ts); not revision-transforms.py |
Yes, the registry's current version (src/apps/sequence-toolkit/sequence.test.ts) |
pipeline/sequence-toolkit/fixtures/reads/manifest.json |
Every reader fixture, text and binary (Stage B) | Yes, every script (src/lib/seq/reads/module.test.ts) |
No |
pipeline/plasmid-map/fixtures/manifest.json |
14 | No | Yes, the registry's current version (src/lib/seq/plasmid-data.test.ts) |
pipeline/plasmid-map/fixtures/examples/manifest.json |
Golden examples | No | Yes |
src/lib/hosts/golden/manifest.json |
Engine-written golden runs | n/a | n/a |
src/lib/workspace/fixtures/manifest.json |
1 (repeat-setup-1.json) |
Yes, through the fixture's own about.scriptSha256 (src/apps/lab-calculators/repeat-setup.test.ts) |
No |
No hash covers src/lib/seq/fixtures/protparam.json, the original src/lib/fit/fixtures/calibration-scipy.json (its successor calibration-4pl-scipy.json is hashed) or src/components/tools/molecule-workbench/fixtures/rdkit-reference.json.
Write-once
scripts/check-write-once.mjs, run by the CI check-write-once job against the base commit, forbids changing or deleting any .json or .csv under a fixtures/ or golden/ folder, any .json under public/verification/, and any files entry of a manifest.json or fixture-hashes.json except by the documented superseded or withdrawn transitions. The rules are described in Data pipeline. Two consequences for oracle work:
- A rerun of an oracle on another day changes
recordedOninside the fixture, so its bytes change and the write-once check fails. Fixtures onmaincannot be regenerated in place; a rerun can only confirm reproducibility (compare, then discard) or add new files under new names. - Where a test asserts a script's hash (assay, sequence), editing that script fails
npm testuntil its fixtures are rewritten, which write-once then refuses. Changing such an oracle means new fixture files and manifest entries.
Under pipeline/sequence-toolkit/fixtures/ (the CTO's Q31, 12 October 2026) every file is protected whatever its extension, so the reads module's .fastq and .ab1 fixtures are; such a file that is neither JSON nor CSV must be listed in the manifest.json beside it, or the check fails, and it is compared byte for byte (read as latin1) so no change hides behind text decoding. Nothing outside that folder changed protection; scripts/check-write-once.test.mjs records Assay's paths as before.
The manifests' top-level fields (scriptSha256, recordedOn, versions) are not protected, and .py, .txt and .csv files under public/verification/ are not covered by the public/verification/ rule.
Fixtures live once
Since 24 October 2026 (D-0126; the CTO's memo of that day, docs/board/memos/2026-10-24-cto-bioreactor-stage.md §8, rule B, as the CTO amended it after the guard's rollout run), scripts/check-committed-growth.mjs, run in the same CI job and by growth.yml against the merge base with main, refuses a net copy: a new path whose bytes sit on more paths of the tree than they did at the merge base, so a move is never a copy. Allowed is one pair, a file in a pipeline area (pipeline/<area>/…, fixtures and the reference scripts that publish with them, such as pipeline/protein-explorer/residue-map/reference_v3.py) and its first publication under public/verification/<slug>/<version>/. Licence texts required beside what they license (LICENSE*, LICENCE*, COPYING*, NOTICE*, <name>-LICENSE*, <name>-LICENCE*, <name>-OFL.txt), evidence images (bounded by the evidence rule's sizes; an identical before/after pair is the CDO gate's proof) and empty files are exempt; a copied machine summary is not. There is no allowance for a re-copied publication: a later version is to name the earlier file by path and SHA-256 in its manifest (rule B3). That needs a change to the resolver in content/apps/routes.ts that has not been made, so whichever app re-publishes next builds it first, in its own branch (Known issues, MNT-213). The check also refuses any new file under src/**/fixtures/, except a .ts module directly in src/lib/fixtures/ (the server-side readFixture<T> reader of MNT-26): new fixtures go under pipeline/<area>/fixtures/ and tests read them there with fs. The duplicates and src/ fixtures already on main stay (they are write-once and published URLs). The check is described in Build, test and CI.
Counts tied to the registry
The counts quoted on badges live in content/apps/<slug>.verification.ts and are checked against the fixture files by src/lib/hosts/fixtures.test.ts (bioreactor), src/apps/sequence-toolkit/sequence.test.ts and exact-match-fixtures.test.ts (sequence), src/apps/lab-calculators/calculators.test.ts (Calc), src/lib/seq/plasmid-data.test.ts (plasmid), src/lib/fit/fixtures.test.ts (assay curve counts) and src/lib/stats/degenerate-oracle.test.ts (the Assay degenerate-inputs set's 60 cases, file and version).
Published verification datasets
Fixtures are published as static files under public/verification/<slug>/<version>/, served at /verification/<slug>/<version>/<file> and linked from each app's docs page. Old versions are kept.
| App | Versions present | How the files got there | Test that ties them to the pipeline |
|---|---|---|---|
| Calc | 0.1.0 (four files), 0.2.0 (byte copies of those four, plus master mix and intermediate dilution, with a manifest.json listing only the two), 0.3.0 (ten files and a manifest), 0.4.0 (a260.json and a manifest), 0.5.0 (a260-2.json and a manifest), 0.6.0 (absorbance-a280.json, absorbance-ratio.json and a manifest) |
reference.py publish --version <v> (0.1.0); the 0.1.0 copies in 0.2.0 were made in commit 732ac12; publish_preparation.py for the rest |
src/lib/fixture-hashes.test.ts, every version |
| Assay | 0.1.0 (six files), 0.2.0 (the earlier twelve, including relative-irls-reference.py and requirements-irls.txt, plus the seven Wave 1 oracle fixtures), 0.3.0 (plate-walk-refit.json only), 0.4.0 (byte copies of the seven plus the three continuation fixtures), 0.5.0 (byte copies of the ten plus the five -v2 fixtures recorded in the oracle image), then 0.6.0 to 0.15.0 (each directory's manifest.json lists its files), and 0.16.0 (the 66 fixtures of 0.15.0, byte for byte, plus degenerate-v1.json) |
By hand before Wave 1; oracles/publish.py and walk/publish.py since |
src/lib/fit/oracle-fixtures.test.ts (0.5.0) and oracle-lineage.test.ts; none for the hand-published files, which today equal their pipeline sources byte for byte |
| Bioreactor | 0.1.0, 0.2.0 (eight files each), 0.3.0 (measured-comparison-0.3.0.json and a manifest), 0.4.0 (measured-comparison-0.4.0.json and a manifest), 0.5.0 (measured-comparison-0.5.0.json and a manifest), 0.6.0 (kla-scaling-0.6.0.json and a manifest), 0.7.0 (feed-0.7.0.json and a manifest), 0.8.0 (transport-0.8.0.json and a manifest), 0.9.0 (batch-fit-0.9.0.json and a manifest), 0.10.0 (batch-fit-flags-0.10.0.json and a manifest; derived from 0.9.0; superseded), 0.11.0 (batch-fit-flags-0.11.0.json and a manifest; the same flags, the full manifest), 0.12.0 (feed-handoff-0.12.0.json and a manifest), 0.13.0 (batch-fit-0.13.0.json and a manifest; B2 1.1.0's oracle, D-0139) |
By hand for 0.1.0 and 0.2.0; reference-measured-comparison.py, -2.py and -3.py for 0.3.0 to 0.5.0; reference-kla-scaling.py (0.6.0); reference-feed.py (0.7.0); reference-transport-pinned.py (0.8.0); wave8/batch-fit-reference.py (0.9.0); wave8/batch-fit-flags.py (0.10.0); wave8/batch-fit-flags-2.py (0.11.0); reference-feed-handoff.py (0.12.0); wave8/batch-fit-reference-2.py (0.13.0) |
src/lib/hosts/fixtures.test.ts (0.2.0); src/apps/bioreactor-simulator/measured-comparison.test.ts (0.3.0, 0.4.0); measured-comparison-completion.test.ts (0.5.0); src/lib/hosts/kla-scaling-oracle.test.ts (0.6.0); src/lib/hosts/feed-oracle.test.ts (0.7.0); src/lib/hosts/transport-oracle-0.8.0.test.ts (0.8.0); src/lib/hosts/batch-fit-fixture.test.ts (0.9.0, the publication); src/lib/hosts/fit/batch-fit-oracle.test.ts (0.9.0 and 0.11.0, the engine; 0.10.0's publication); src/lib/hosts/feed-handoff-oracle.test.ts (0.12.0); src/lib/hosts/fit/batch-fit-oracle-0-13-0.test.ts (0.13.0) |
| Sequence | 0.1.0 to 0.4.0 | pipeline/sequence-toolkit/publish.py <version> from 0.3.0; 0.2.0 was a byte copy of 0.1.0 (audit M1) |
src/apps/sequence-toolkit/sequence.test.ts, current version |
| Plasmid | 0.1.0 to 0.7.0 | pipeline/plasmid-map/publish.py <version>, which also copies the golden examples as examples-manifest.json, puc19@1.json, pbr322@1.json |
src/lib/seq/plasmid-data.test.ts, current version |
| Protein | 0.3.0 (three evidence references, their three Python scripts and a manifest), 0.4.0 (the construct tag-model reference construct-cropper-v2.json, its oracle script and a manifest) |
pipeline/protein-explorer/evidence/publish.py <version>; pipeline/protein-explorer/construct-cropper/reference.py publish <version> for 0.4.0 |
The manifests give byte lengths and SHA-256 for every copied source; capture assertions are in src/apps/protein-explorer/evidence-record.test.ts, the construct tag model in src/lib/seq/construct-cropper.test.ts |
Both publish.py scripts copy from the pipeline's fixtures (never from an earlier published directory), check with git cat-file whether any known main (origin/main or main) already holds each target file, and refuse to replace one that it does. With no main known they refuse everything.
The assay 0.2.0 manifest does not list relative-irls-reference.json, and the assay 0.1.0 directory's manifest.json lists the three PRISM files, which that directory does not contain.
src/components/suite/marketing/release.ts (verificationFiles, used by the changelog page) looks for files in public/apps/<slug>/verification/<version>/, a path that does not exist, while returning URLs under /verification/<slug>/<version>/. The changelog therefore lists no verification files for any app.
Scheduled re-checks
.github/workflows/scheduled.yml runs weekly (Mondays 05:17 UTC) and on demand. Its verify matrix runs one job per check with fail-fast: false and writes a heartbeat line to the step summary. The checks and their services are listed in Data pipeline. Only one of them re-runs a numerical oracle:
pip install rdkit==2026.3.6 (Python 3.12 from actions/setup-python)
python pipeline/molecule-workbench/rdkit_reference.py library /tmp/lib-ref.json
node pipeline/molecule-workbench/verify-library.mts /tmp/lib-ref.json
The Assay R and Python oracles run weekly in the oracle image (.github/workflows/oracles.yml, job drift, Mondays 06:43 UTC, not gating): every script the three Assay manifests name is rerun (for a lineage with a newer version, only the newest version's script) and its output compared with the committed files, ignoring only recordedOn and two run-dependent keys ($.about.environment.cpu, and $.about.environment.image when both sides are digests), which it reports (CTO ruling of 12 October 2026 on the drift keys); see The oracle environment and Build, test and CI. None of the oracles for Calc, Bioreactor, Sequence or Plasmid runs in CI or on a schedule. What runs on every change is npm test (the CI check job; see Build, test and CI), which compares the engines with the committed fixtures and checks the hashes; it never calls Python, R or the network. The workflow header says it runs "every pipeline//verify script"; pipeline/education/verify.py is not in the matrix.
Evidence records
docs/science/evidence/ holds dated run records. They are evidence of what was executed, not expected-value fixtures, and no test reads them.
| Path | What it records |
|---|---|
molecule-library-2026-10-10/ |
README.md (commands, Python 3.13.16, Node 22.22.0, library and reference SHA-256s), verify-library.txt (2,197 molecules, all match, largest QED difference 2.78e-16, 124 with a PAINS match) and the gzipped Python reference |
molecule-fingerprint-policy-2026-10-10/ |
The Wave 4 key spike: key_spike.py and key-spike.mts with their JSON outputs, showing which Morgan settings RDKit.js and Python RDKit honour (onlyNonzeroInvariants and includeRingMembership change no bit on 2,205 molecules) |
protein-align-2026-10-11/ |
The EMBOSS Needle 6.6.0.0 cross-check of the protein aligner (CSO condition C1): README with commands and results, FASTA inputs written from the oracle's stored UniProt responses, and four .needle outputs |
molecule-pca-reference.json |
NumPy eigendecomposition of the 2,197 molecules' descriptors against the shipped PCA: loadings within 4.22e-15, explained fractions within 4.45e-16 |
implementation-verification/ |
Logs and JSON ledgers of the 7 to 10 October correction work: checks.json (commands, versions, exit codes, a science-patch SHA-256 reproducible with the git diff in its README), build, lint, unit, browser, worker-gate and write-once logs |
implementation-verification/r-crosscheck-2026-10-07/ |
The app's exported R scripts for C01, C24, C25, C26 and C27, their outputs and comparison.json: executed with R 4.6.1 and drc 4.0.0, midpoints within 7.93e-5 log10 units of the references (tolerance 0.00043) and unweighted Wald bounds within 2.23e-7 (tolerance 0.0022). Scope as stated: point and unweighted-Wald cross-checks, not inference calibration or profile bounds |
original-audit-results/, original-audit-inventory.json |
Five numerical result files from the original audit and the hashes of the archive they came from; the archive itself was moved outside the repository. They describe the pre-correction engine |
The scientific audits that cite these records are docs/science/2026-10-scientific-audit.md, docs/science/2026-10-scientific-audit-implementation.md, docs/science/2026-10-scientific-audit-followup.md and docs/science/2026-10-10-scientific-audit-of-the-7-to-10-october-changes.md. See Science overview.
Coverage matrix
Method ids are those in content/apps/<slug>.methods.ts. "Badge" is the tolerance the registry quotes; "Test" is what the test asserts where it differs.
| App | Method | Oracle | Test file | Tolerance |
|---|---|---|---|---|
| Calc | formula-mass, stock-salt-purity |
CIAAW 2021 decimal sum, RDKit formula | src/lib/units/formula-mass-fixtures.test.ts |
Exact |
| Calc | c1v1, recipe-table, dmso-carry-over (and serial-dilution 1.0.0) |
Python fractions |
src/lib/dilution/fixtures.test.ts |
9 significant figures; displays exact |
| Calc | serial-dilution 2.0.0, plate-sheet |
Python fractions |
src/lib/dilution/serial-2-oracle.test.ts, plate-sheet-oracle.test.ts |
1e-9 relative |
| Calc | master-mix-preparation, one-intermediate-dilution |
Python fractions; exact grid |
src/lib/dilution/preparation-oracle.test.ts |
1e-9; plan within one grid step |
| Calc | cell-seeding, centrifuge-rcf-rpm, percent-solution, antibody-dilution, od600-cell-estimate |
Python fractions and decimal |
src/lib/preparation/preparation-oracle.test.ts, antibody-batch-oracle.test.ts, od600-oracle.test.ts |
1e-9 relative |
| Calc | buffer-recipe |
Python decimal; NumPy 2.5.3 |
src/lib/dilution/buffer-recipe-oracle.test.ts |
Mass 1e-9; setpoint 0.01 |
| Calc | unit-conversion, temperature-offset |
Python fractions |
src/lib/units/unit-converter-oracle.test.ts |
Exact strings; 1e-9 |
| Calc | pka-temperature-goldberg |
Goldberg 2002 eq. 12 in NumPy | src/lib/dilution/fixtures.test.ts |
0.01 |
| Calc | extinction-280-pace, a280-beer-lambert |
Biopython 1.85 ProtParam | src/lib/seq/extinction-fixtures.test.ts |
Exact |
| Calc, Protein | protein-average-mass |
Recorded ExPASy ProtParam (9 proteins) | src/lib/seq/protein-mass.test.ts |
0.02 Da; pI at 2 dp |
| Assay | four-pl, levenberg-marquardt-bounded (unweighted) |
R nls, drc cross-check |
src/lib/fit/fixtures.test.ts |
logIC50 0.00043; Hill 0.1 %; plateaus 0.1 % of span; objective 1e-6 |
| Assay | weighting-1-over-y2 (IRLS) |
SciPy least_squares IRLS (2 curves) |
src/lib/fit/irls.test.ts |
1e-5 × max(1, |p|) |
| Assay | wald-log-ic50, profile-likelihood-log-ic50 |
R | src/lib/fit/fixtures.test.ts |
0.0022 and 0.0043 log10 units |
| Assay | ic50-rules, absolute-ic50 |
R (version-1 counts and crossing) | src/lib/fit/fixtures.test.ts |
Counts exact; outcomes no longer asserted |
| Assay | z-prime |
Hand-worked in R | src/lib/fit/fixtures.test.ts |
1e-9 relative |
| Assay | extra-sum-of-squares-f |
Hand-worked in R, drc anova() |
src/lib/fit/fixtures.test.ts |
RSS 1e-6, F 1e-5, p 1e-4 (4 of 6 cases asserted) |
| Assay | standard-curve-interpolation |
SciPy curve_fit (2 cases); exact rational OLS (7 cases) |
src/lib/fit/calibration-oracle.test.ts, oracle-fixtures.test.ts |
2e-5 absolute; 5e-5 relative; linear 1e-9 |
| Assay | replicate-summaries |
Python fractions and decimal |
src/lib/fit/oracle-fixtures.test.ts, chart-replicates.test.ts |
1e-9 relative |
| Assay | growth-window-rate |
Python fractions, SciPy linregress |
src/lib/growth/window-oracle.test.ts |
1e-9 relative |
| Assay | michaelis-menten-initial-rate, michaelis-menten-profile-interval |
SciPy curve_fit; SciPy direct profile |
src/lib/kinetics/michaelis-menten-oracle.test.ts, michaelis-menten-profile-oracle.test.ts |
1e-5 relative; bounds 1e-3 ln units |
| Assay | inhibition-global-fit |
SciPy least_squares |
src/lib/kinetics/inhibition-oracle.test.ts |
1e-5 relative |
| Assay | ki-cheng-prusoff-competitive |
Python fractions |
src/lib/kinetics/ki-oracle.test.ts |
1e-12 relative |
| Assay | binding-equilibrium, binding-ki-cheng-prusoff-munson-rodbard |
SciPy least_squares; exact mass balance for Munson–Rodbard; R nls, drc, 20 of 20 (binding.json, 0.11.0) |
src/lib/binding/binding-oracle.test.ts |
1e-5 relative; bounds 1e-3 ln |
| Assay | hill-initial-rate |
SciPy curve_fit; R nls, 7 of 7; coverage on h (hill-kinetics.json, 0.11.0) |
src/lib/kinetics/hill-oracle.test.ts |
1e-5 relative; bounds 1e-3 ln |
| Assay | bisubstrate-initial-rate |
SciPy least_squares; R nls, 15 of 15 (bisubstrate.json, 0.11.0) |
src/lib/kinetics/bisubstrate-oracle.test.ts |
1e-5 relative |
| Assay | morrison-tight-binding |
SciPy least_squares, 50-digit decimal hand rates; R nls, 9 of 9 (morrison.json, 0.11.0) |
src/lib/kinetics/morrison-oracle.test.ts |
1e-5 relative |
| Assay | slow-binding-one-step, slow-binding-two-step |
Python fractions and SciPy; R lm, nls, 15 of 15 (slow-binding.json, 0.11.0) |
src/lib/kinetics/slow-binding-oracle.test.ts |
line 1e-9; hyperbola 1e-5 relative |
| Assay | thermal-shift-apparent-tm |
SciPy curve_fit, hand differences |
src/lib/thermal/apparent-tm-oracle.test.ts |
Tm 1e-3 °C |
| Assay | Figure PDF writer (src/lib/figure/pdf.ts; not a scientific method) |
Python standard library from PDF 1.4 and RFC 1950; pypdf read-back by hand | src/lib/figure/pdf.test.ts |
Structure, texts, substitutions and path count exact; MediaBox 0.001 pt |
| Assay | Figure TIFF writer (src/lib/figure/tiff.ts; not a scientific method) |
Python standard library from the TIFF 6.0 specification; Pillow read-back by hand | src/lib/figure/tiff.test.ts |
Tags, strips and the 3 by 2 pixels exact |
| Assay | qpcr-relative-expression |
Python decimal (2 values, hard-coded) |
src/lib/assay/qpcr/qpcr.test.ts |
2e-12 × max(1, |value|) |
| Assay | growth-gompertz-fit (A6) |
SciPy curve_fit (trf, twenty starts) in the oracle image; R nls port as the cross-check (growth-gompertz.json, 0.7.0) |
src/lib/growth/gompertz-oracle.test.ts |
1e-5 relative; RSS 1e-6 |
| Assay | qpcr-fold-change-welch-interval (A6) |
Hand Welch formula with scipy.stats.t.ppf from the Cq tables, in the oracle image; R t.test and SciPy ttest_ind as cross-checks (qpcr-welch.json, 0.7.0) |
src/lib/assay/qpcr/welch-oracle.test.ts |
Bounds 1e-9; df 1e-9 relative; reasons exact |
| Assay | column-descriptives, column-t-tests, column-rank-tests, median-order-statistic-interval, shapiro-wilk-royston, column-effect-sizes (A-2 family 1) |
R 4.4.3 stats in the oracle image; SciPy 1.13.1 cross-checks, 179 of 179 (column-statistics.json, 0.11.0, 50 cases; set column-statistics) |
src/lib/stats/column-statistics-oracle.test.ts |
Descriptives, quantiles, intervals and estimates 1e-9 relative; t, df and p 1e-8 relative; exact rank p and achieved coverage 1e-10; approximate rank p 1e-8 relative; W 1e-7, its p 1e-5 relative; the rank-biserial 1e-12; withheld states exact |
| Assay | one-way-anova, post-hoc-comparisons, rank-groups, repeated-measures, two-way-anova (A-2 family 2) |
R 4.4.3: stats, car 3.1-5, multcomp 1.4-32, nlme 3.1-167; Tukey and Games–Howell p from R's integrate(), with ptukey and TukeyHSD as cross-checks (groups.json, 0.11.0, 22 cases; set groups) |
src/lib/stats/groups-oracle.test.ts |
Sums of squares, F, η², ω², ε and rank statistics 1e-9 relative; p 1e-8; Tukey–Kramer and Games–Howell p 1e-7; Dunnett p 1e-6; Holm, Šidák and Bonferroni 1e-8; the mixed model's F and p 1e-6, its means, SEs and variances 1e-5 |
| Assay | correlation-pearson-spearman, linear-regression-qr, nonlinear-families, relationship-model-comparison, deming-regression, bland-altman (A-2 family 3) |
R 4.4.3 (cor.test, lm, car 3.1-5 vif, nls port, anova, AIC, the Deming eigen route and jackknife), exact fractions for polynomials and two ill-conditioned regressions, a SciPy direct profile for curve intervals; SciPy 1.13.1 cross-checks, 105 of 105 (relationships.json, 0.11.0, 115 cases; set relationships); the nonlinear coverage simulation, 2,000 replicates per family (nonlinear-coverage.json, 0.11.0; set nonlinear-coverage) |
src/lib/relate/relationships-oracle.test.ts, nonlinear-coverage-oracle.test.ts |
Correlation 1e-10 relative (t-based p 1e-8, exact p 1e-12, bounds 1e-9); regression 1e-9 relative (p 1e-8); curves RSS 1e-8 relative, parameters 1e-6, profile bounds 1e-3, polynomials 1e-9; comparison ΔAICc 1e-8 (linear), 1e-4 (nonlinear); Deming 1e-10; jackknife and Bland–Altman 1e-9; coverage at least 0.90 for every quantity (lowest 0.945) |
| Assay | kaplan-meier, median-survival, survival-tests, log-rank-hazard-ratio (A-2 family 4) |
R 4.4.3 survival 3.8-3 (survfit, quantile.survfit, survdiff), Gehan–Breslow–Wilcoxon by hand in R; identities inside R, 88 of 88 (survival.json, 0.11.0, 34 cases; set survival) |
src/lib/stats/survival-oracle.test.ts |
S(t), std.err, band, medians 1e-10 relative; χ², E and the hazard ratio 1e-9 relative; p 1e-8 relative; counts, the risk table and withheld states exact |
| Assay | contingency-tests, two-by-two-measures, mcnemar, cochran-armitage (A-2 family 5) |
R 4.4.3 base (chisq.test, fisher.test, prop.test, binom.test, mcnemar.test, prop.trend.test; Woolf, Katz and Newcombe by hand); checks inside R, 37 of 37 (counts.json, 0.11.0, 42 cases; set counts) |
src/lib/stats/counts-oracle.test.ts |
Measures, proportions and expected counts 1e-9 relative; X², chi-squared, Fisher 2 × 2, McNemar and trend p 1e-8 relative; Fisher r × c 1e-6 relative; refused cells and withheld states exact |
| Assay | p-value-adjustment (A-2 family 6) |
R 4.4.3 p.adjust (BH, BY, Holm, Bonferroni) (fdr.json, 0.11.0, 10 cases; set fdr) |
src/lib/stats/fdr-oracle.test.ts |
Adjusted p 1e-12 relative; counts, m and withheld states exact |
| Assay | grubbs-two-sided, studentised-residuals-one-way, power-sample-size (A-2 family 7) |
R 4.4.3: outliers 0.15 grubbs.test, the Bonferroni bound with car 3.1-5 outlierTest as its cross-check, rstudent; power.t.test, power.anova.test, power.prop.test with SciPy cross-checks (column-outliers.json, 21 cases, and power-sample-size.json, 26 cases, 0.11.0; sets column-outliers, power-sample-size) |
src/lib/stats/outliers-power-oracle.test.ts |
G, t and critical G 1e-9 relative, p 1e-8 relative; power 1e-8 (t, proportions) and 1e-7 (ANOVA), n 1e-6 relative; rows, flags and withheld states exact |
| Assay | Degenerate spread and separation across z-prime, percent-of-controls, percent-of-vehicle-background, screening-percent-inhibition, screening-robust-z, plate-diagnostics, growth-window-rate, growth-gompertz-fit, gr-metrics, qpcr-fold-change-welch-interval, two-way-anova, repeated-measures (the mixed model), survival-tests, log-rank-hazard-ratio, slow-binding-one-step, slow-binding-two-step, thermal-shift-tm-intervals |
Python fractions on the decimal inputs for every refusal; NumPy 2.5.4 and SciPy 1.18.1 for the cases that keep calculating; exact-fraction log-rank and NumPy two-way where the memo names R, which was not available (degenerate-v1.json, 0.16.0, 60 cases; set degenerate-inputs) |
src/lib/stats/degenerate-oracle.test.ts; the bound itself in src/lib/stats/degenerate.test.ts |
Statuses and codes exact; values at the existing sets' tolerances; no NaN or ±∞ under a value status |
| Bioreactor | unstructured-growth-model, modified-rosenbrock, dissolved-oxygen-balance |
SciPy Radau (33 scenarios) | src/lib/hosts/fixtures.test.ts |
0.5 % of peak; accuracy check 1e-5 or time shift; events 1e-3 h |
| Bioreactor | unstructured-growth-model (closed forms) |
Closed forms | src/lib/hosts/fixtures.test.ts |
Badge 1e-6; test 1e-3 relative |
| Bioreactor | modified-rosenbrock (one step, LU) |
NumPy, LAPACK | src/lib/fit/ode.test.ts |
1e-13; backward error 1e-12 |
| Bioreactor | Right-hand side, Jacobian | SciPy script | src/lib/hosts/model.test.ts |
1e-12 relative |
| Bioreactor | dissolved-oxygen-balance, van-t-riet-kla |
USGS DOTABLES output; SciPy | src/lib/hosts/fixtures.test.ts |
DOTABLES rounding; 1e-9 |
| Bioreactor | scale-up-rules |
Worked in the script | src/lib/hosts/fixtures.test.ts |
Badge 1e-9; test 1e-9, constant kLa 1e-6 |
| Bioreactor | dissolved-oxygen-balance (feed DO, reference-basis gas flow) |
SciPy Radau via reference.py under the pins (transport-0.8.0.json, 4 chemostat cases; the 3.9.6 file of 11 October kept) |
src/lib/hosts/transport-oracle-0.8.0.test.ts |
0.5 % of peak; accuracy check; events 1e-3 h |
| Bioreactor | kla-value-scaled |
NumPy float64 (kla-scaling-0.6.0.json, 18 settings) |
src/lib/hosts/kla-scaling-oracle.test.ts |
Relative 1e-12; exact where returned as entered |
| Bioreactor | feed-law (laws, Vmax stop, pump sheet, design rate) |
Python decimal, 40 digits (feed-0.7.0.json, 12 feeds, 4 design rates) |
src/lib/hosts/feed-oracle.test.ts |
Relative 1e-12, floor 1e-15; times 1e-12 of the feed's length |
| Bioreactor | measured-comparison-time-alignment (Contract 11A) |
Python fractions (measured-comparison-0.3.0.json, 8 cases; measured-comparison-0.4.0.json, 3 cases with a failed simulation reference; measured-comparison-0.5.0.json, 25 cases with complete inputs: every refusal and the reference alignment) |
src/apps/bioreactor-simulator/measured-comparison.test.ts, measured-comparison-completion.test.ts |
Elapsed hours 0 where exact in binary64, else 1e-15 h (0.3.0, 0.4.0) or each point's stated bound (0.5.0); rows, codes, hashes, the dropped reference and its claimed hashes exact |
| Sequence | translation-ncbi-genetic-codes |
Biopython 1.88 | src/lib/seq/translate.test.ts |
Exact |
| Sequence | orf-finder-ncbi-conventions |
Independent scan, Biopython, ORFfinder 0.4.3 | src/lib/seq/orf.test.ts |
Exact |
| Sequence | protein-average-mass, isoelectric-point-bjellqvist, extinction-280-pace |
Saved Expasy ProtParam (22) | src/lib/seq/protein.test.ts |
0.01 Da; 0.01; exact |
| Sequence, Plasmid | oligo-tm-nearest-neighbour and salt corrections |
Biopython 1.88 Tm_NN |
src/lib/seq/tm.test.ts |
Badge 0.1 °C; test 1e-6 °C |
| Sequence | hairpin-dimer-dg37 |
primer3-py 2.3.1 (ntthal) | src/lib/seq/structure.test.ts |
0.4 (hairpin), 0.1 (dimer) kcal/mol on ntthal's structures |
| Sequence, Plasmid | nucleic-acid-average-mass |
Exact arithmetic | src/lib/seq/dna-mw.test.ts, src/lib/seq/dna-amounts.test.ts |
0.01 g/mol; 1e-12 relative |
| Sequence | peptide-smiles-linear |
Python RDKit 2026.03.6 | src/lib/seq/peptide.test.ts |
Identical canonical SMILES |
| Sequence | pcr-product-exact-pair, exact-fragment-search |
exact_matches.py |
src/apps/sequence-toolkit/exact-match-fixtures.test.ts |
Exact |
| Sequence | exact-transform-global-edit-v1 |
revision-transforms.py |
src/lib/seq/revision-transforms.test.ts |
Exact; edit count equals independent distance |
| Plasmid | restriction-sites, restriction-digest, diagnostic-digest-picker |
Biopython Bio.Restriction (the picker against trio v2 under gel rule 2.0.0; trio v1 superseded, kept) |
src/lib/seq/digest.test.ts |
Exact; the set of digests telling all three apart asserted equal under both rules |
| Plasmid | dam-dcm-blocking |
McClelland 1994 Table I by hand | src/lib/seq/methylation.test.ts |
Exact |
| Plasmid | feature-annotation |
Biopython PairwiseAligner |
src/lib/seq/features/annotate.test.ts |
0.1 percentage points |
| Plasmid | plasmid-dna-amounts |
Python fractions |
src/lib/seq/dna-amounts.test.ts |
1e-12 relative |
| Plasmid | snapgene-read |
Biopython SnapGene reader | src/lib/seq/features/snapgene.test.ts |
Exact |
| Plasmid | ds-fragment-ends, restriction-cloning, golden-gate, gibson-style, crispr-spcas9-guides |
Hand-worked scripts with Biopython | src/lib/seq/ds-fragment.test.ts, restriction-cloning.test.ts, golden-gate.test.ts, gibson.test.ts, crispr.test.ts |
Exact |
| Molecule | Descriptors, QED, PAINS, Morgan | Python RDKit 2026.03.6 | src/components/tools/molecule-workbench/chem.test.ts; weekly verify-library.mts |
1e-6; QED 1e-9 (library: 0.01); counts and bits exact |
| Molecule | molecule.morgan-tanimoto under morgan-r2-2048-described-v1; same-compound forms |
fingerprint_policy.py (Python RDKit 2026.03.6) |
src/components/tools/molecule-workbench/policy.test.ts |
Bits and forms exact; Tanimoto 1e-12 |
| Molecule | Series rows, pIC50, within-series matrix | series_reference.py |
src/components/tools/molecule-workbench/series.test.ts |
1e-6; QED and pIC50 1e-9; counts exact |
| Protein | AlphaMissense reference check and per-residue mean (no method id) | alphamissense_reference.py (AlphaFold DB files) |
src/components/tools/protein-explorer/alphamissense.test.ts |
n exact; mean 1e-6 |
| Protein | Substitution check (no method id) | substitution_reference.py |
src/components/tools/protein-explorer/substitution.test.ts |
Exact (score text) |
| Protein | Pairwise alignment (no method id) | Biopython 1.88 PairwiseAligner; EMBOSS Needle 6.6.0.0 cross-check |
src/lib/protein/align/align.test.ts |
Scores exact; counts and unique pairs exact |
| Pipeline | Registry coverage, timeline by phase, sponsor figure (no method id) | coverage_oracle.py on a frozen search |
src/components/tools/drug-pipeline-explorer/coverage.test.ts |
Exact integers |
| Cell lines (no app) | cell-line-resolution |
pipeline/cellosaurus/oracle.py reading the Cellosaurus 56.0 release file (resolution-56.0.json, 100 cases) |
src/lib/cells/resolve.test.ts |
Exact: status, accessions, what matched and each record's fields; zero wrong assignments |
Methods without an independent oracle
Methods whose verification is "unit-tested", or whose check is not independent of the engine:
| App | Method | What exists instead |
|---|---|---|
| Calc | pipette-minimum-2ul, od600-calibration (a260-rules-of-thumb left this table on 17 October 2026, SCI-147) |
Unit tests (the OD600 calibration rests on fitCalibration, itself checked against the Assay calibration fixtures) |
| Assay | aicc |
"Unit-tested on nested fits of the fixture curves; no external oracle" |
| Assay | percent-of-controls |
Unit-tested on the MTT plate against the fixture's formula |
| Assay | ic50-rules (version 2 permutation gate) |
Exhaustive-null and regression tests; version-1 fixtures archival |
| Assay | plate-export-parser |
Round-trip of synthetic files written by make-plate-fixtures.mts; no vendor file |
| Assay | kinetic-read-blocks |
Round-trip of the four synthetic kinetic files written by make-plate-fixtures.mts (every read of every well, markers, missing reads, axis and unit); no vendor file |
| Assay | qpcr-relative-expression |
Independent Decimal values exist but are not published, not hashed and not in verificationSets |
| Assay | Initial-rate fit failed state |
Unit-tested through the stored model, export and record; no oracle case means the same on both sides |
| Assay | two-phase-dose-response |
The SciPy oracle in the image (verification set 0.11.0, 8 cases) and R drc::drm with the seven-parameter forms (6 of 6 fitted cases agreeing); verified. The drc twophase() form and the Prism equation forms are not yet read (CSO memo of 13 October 2026) |
| Assay | gaddum-schild-global, gaddum-noncompetitive, operational-model-furchgott |
The SciPy oracle in the image (verification set 0.13.0, 8 of 8 cases agreeing), with R nls and drc agreeing on 6 of 7 and SciPy holding the lowest residual sum of squares on every fit (SCI-121, closed), so verified; no published raw dataset is yet used as a reference (CSO memo of 13 October 2026, risks) |
| Assay | gr-metrics |
The hand cases of the published definition and a definition-only oracle; checked on 13 cases (verification set 0.11.0); not compared with the GRmetrics package (SCI-120) |
| Assay | Weighted (1/Y²) intervals | Not claimed; the verification note says no weighted interval coverage and no Prism equivalence |
| Bioreactor | oxygen-rescue-cascade |
Unit tests (kla-value-scaled and feed-law have had references since 24 October 2026: verification 0.6.0 and 0.7.0) |
| Bioreactor | Golden runs | Engine-written regression only |
| Plasmid | gel-schematic |
Band grouping against 11 hand cases with exact fractions (gel-bands-v1.json); band positions unit-tested only, "no oracle exists for band positions" |
| Plasmid | primer-sites |
Exact-site check at build and weekly; one-mismatch places unit-tested |
| Sequence | construct-tags-and-sites, primer-pair-annealing-rule |
Unit tests |
| Molecule | molecule.ro5, molecule.veber |
Unit tests |
| Molecule | Descriptors other than QED and PAINS | Same library (RDKit) as the engine: a build and wrapper check, not an independent method |
| Education | The Education tools | Model tests only |
Audit M1 (docs/science/2026-10-10-scientific-audit-of-the-7-to-10-october-changes.md) lists the methods shipped outside roadmap rule 1 and is recorded as open. Since that audit, PCR product and fragment search have gained exact_matches.py fixtures, and Waves 1 and 3 published fixtures for calibration, master mix, intermediate dilution and every new method; the permutation gate, qPCR and the unlisted IRLS fixture remain as described above.
Unread or unchecked sources the oracles depend on, as their records state: Sebaugh and McCray 2003 (assay bend-point constant), Van 't Riet 1979 non-coalescing coefficients ("Unverified until the paper is read"), the Owczarzy 2008 free-Mg²⁺ range ("recalled, not read"), IDT OligoAnalyzer not run, ProtParam's terms "To be checked" (held for the General Counsel).
Risks
- Re-runs are rare and manual. Apart from the weekly Molecule check, no oracle is re-executed after its first run. If a pinned package becomes uninstallable, or an upstream web service (ProtParam, PubChem, UniProt, NCBI) changes, a reproduction may no longer be possible. Calc's
reference.pyfetches PubChem and UniProt live without saving the responses; the sequence and plasmid scripts save their inputs. - Environment reproducibility. Requirements pin only top-level packages, without hashes or a lock file, and none pins the Python version. Recorded interpreters differ: Python 3.11.15 (Calc 0.1.0 and 0.2.0, Sequence, Plasmid, the Wave 1 Assay oracles), 3.11.17 (the Wave 1 continuation's three Assay fixtures, against a 3.11.15 pin), 3.13.16 (Calc 0.3.0 fixtures, Molecule evidence and Bioreactor's measured-comparison 0.4.0, unpinned), 3.12.14 (Bioreactor's process model, measured-comparison 0.5.0, kLa scaling 0.6.0, the feed 0.7.0 and transport 0.8.0, IRLS), 3.9.6 (Bioreactor's measured-comparison 0.3.0 and the transport file of 11 October, republished under the pins as 0.8.0), 3.12 (scheduled workflow). NumPy is pinned at 2.4.6, 2.3.5 and 2.0.2 in three files of the same Assay directory, and the Calc buffer-recipe oracle used NumPy 2.5.3. R and drc come from Ubuntu packages and CRAN source, not a locked environment; the R cross-check evidence ran on R 4.6.1 while the fixtures were written on R 4.3.3. Biopython is 1.85 for Calc and 1.88 elsewhere.
- R cross-checks for the Wave 1 continuation. Not run for the 0.4.0 profile, thermal shift and inhibition fixtures (R was not installed in the build container). Run in the oracle image for their 0.5.0
-v2files, which the engine tests and badges now use: Rnls11 of 11,nls(SSfpl)7 of 7, and R's direct profile on 7 of 7 cases with an interval (the CSO's ruling replacedconfint, recorded beside it at 4 of 7, as the deciding check). - Version guards are uneven. The sequence, plasmid, molecule (including
fingerprint_policy.pyand the scripts built on it) and assay R scripts refuse other versions; Calc'sreference.py, the bioreactorreference.py,relative-irls-reference.py, and Protein'salign_reference.pyandblosum62.pyonly record what ran. Protein's oracles have no requirements file. - Wave 4 inputs outside the write-once rule. The gzipped AlphaMissense, prediction and UniProt inputs, the AlphaMissense source rows (
rows.tsv.gz) and the stored BLOSUM62.txtare not.jsonor.csv, socheck-write-oncedoes not protect them; only the BLOSUM62 file's hash is asserted by a test. Their sha256 values are recorded in the fixtures beside them. - Thin tie coverage for the aligner. The largest fixtured co-optimal set is 8 alignments; the cell-limit and U/O refusals are checked in a different order by the oracle and the engine, and no fixture exercises both at once (CTO review).
- Oracle drift and provenance gaps. Plasmid fixtures were written by four revisions of
reference.py, and nothing asserts the plasmid, bioreactor process-model (reference.py) or Wave 2 script hashes (the two measured-comparison scripts' hashes are asserted), so an oracle script can change without any test noticing. Oracles outsidepipeline/(src/lib/assay/qpcr/oracle.py) and hand-recorded files (src/lib/seq/fixtures/protparam.json) are not hashed. - Write-once against reruns. Fixtures embed
recordedOn, so a correct rerun on a later day is refused bycheck-write-once. Corrections must go into new files and versions; this protects published numbers but makes routine regeneration impossible onmain. - Shared assumptions. Several oracles share inputs with the engine by design: the CIAAW table (transcribed twice), the engine's stated alignment scoring, the SnapGene packet layout (the same understanding wrote the test files and the reader, as
make-dna-fixtures.pysays), RDKit for Molecule (the policy and series oracles use Python RDKit of the same release), BLOSUM62 for the protein aligner (NCBI's file, equal to Biopython's), and the AlphaMissense files themselves for Protein. Agreement there shows consistent implementation, not correctness of the shared assumption. - Build-time-only GPL tools. drc, MASS and R (GPL), and primer3-py with libprimer3 (GPL-2.0-or-later), are run offline only; their numbers are committed and published under
public/verification/. The design briefs' Rule G treats this as permitted. Whether committing and publishing those outputs, and naming GPL oracles in verification copy, is acceptable for a sale is held for the General Counsel. A change that ported any of their code or parameter tables intosrc/would need a rewrite at the pre-live pass;scripts/licences.mjsgrepssrc/andcontent/for such identifiers. - No independent sign-off. Q-19 (
docs/board/open-questions.md) requires fixture sign-off independent of whoever directed the engine before an app leaves preview. It is open; the assay and bioreactor badges carry "Draft until the independent reviewer signs" text. - Badges and tests can disagree. The bioreactor closed-forms badge says 1e-6 relative while the test asserts 1e-3; the scale-up badge says 1e-9 while the constant-kLa rule is tested at 1e-6; the assay F-test badge counts six comparisons while four are asserted. Nothing checks badge tolerance text against test constants, except the sequence structure gate.
Adding an oracle for a new method
Derived from the existing scripts, tests and the CSO's rulings they cite.
- Choose the oracle before writing the engine. Prefer a named external library run offline (Biopython, SciPy, R, RDKit), exact arithmetic (
fractions,decimal), or a published table worked by hand. Do not use anything derived from the engine under test. A GPL tool may be run offline only; record it withpurpose: "oracle"and say so when proposing it, because it will be reviewed at the pre-live pass. Academic-only tools (for example CRISPOR) are not available even as oracles under the briefs' Rule G. - Write the script in
pipeline/<slug>/. Python or R, with a header stating the method, the oracle, the sub-commands and the exact command line (python3 -I pipeline/<slug>/<script>.py). Pin the packages in arequirements*.txtbeside it and refuse any other version at start-up (EXPECTED_BIOPYTHON = "1.88"andsys.exitis the pattern). Never import or readsrc/. Save any downloaded input with its URL, retrieval time and SHA-256 so the fixture step runs offline. - Add a hand-worked guard where the derivation is algorithmic. Work at least one case by hand inside the script and refuse to write unless the script's own derivation agrees (
HAND_WORKEDinexact_matches.pyandcrispr.py). - Write the fixture in
pipeline/<slug>/fixtures/. Give it anaboutblock (generatedBy,scriptSha256,recordedOn, oracle, version, licence, the ruling or source) and cases in the shape{ id, input, expected, method, oracle, oracleVersion, recordedOn, source }. Include refusal and edge cases, and cases chosen to fail a plausible wrong implementation. - Record it in the manifest. Add the file's SHA-256, case counts,
generatedByandscriptSha256to the directory'smanifest.json. A new entry is allowed by the write-once check; changing an existing one is not. - Write the test beside the engine. Read the fixture in place with
readFileSync(new URL(...)), assert the manifest hash and the script hash first, then compare with explicit tolerances stated once as constants. Use absolute tolerances on logarithms and exact equality for counts, sequences and refusals. Assert that the engine's objective is not better than the oracle's where it minimises one. - Declare it in the registry. Add the oracle to the app's
oracles[], aVerificationSettoverificationSets[]with the method ids, count, tolerance text and file name, and the counts and badge sentence tocontent/apps/<slug>.verification.ts. Set the method'sverificationto"verified"with a note in the badge form. Add a test that the badge counts equal the fixture's. - Publish for the release. Run the app's
publish.py <version>where one exists, or copy the fixture and manifest byte for byte intopublic/verification/<slug>/<version>/, and make sure a test compares the published copy with the pipeline file for the registry's current version. - Document and record. Update the pipeline README (or the script header where there is none) with the commands, versions and findings, and add a dated evidence record under
docs/science/evidence/if the run produced results beyond the fixture. Runnpm testand the write-once check before merging; once onmain, the fixture is immutable.
Gibson primer-tail integration fixtures (16 October 2026)
pipeline/plasmid-map/gibson-primers/reference.py uses independent Python
standard-library string algebra and exhaustive exact windows to generate the
write-once pipeline/plasmid-map/fixtures/gibson-primers-v1.json, committed before
the corresponding transformation. Its five positive cases retain raw templates,
explicit orientations, forward/reverse oligos, separate annealing parts and 5′
tails, PCR products, junctions, final sequence/hash and base accounting. Eight
design refusals and three supplied-fragment checker cases exercise short, invalid,
ambiguous, repeated and consumed input. CPython 3.9.6/arm64 is recorded; there are
no floating-point calculations in this oracle. Temperature estimates continue to
use the existing independently checked nearest-neighbour Tm engine and fixtures.
The additive gibson-primers-v2.json and gibson-primers/reference-v2.py
retain v1 unchanged and add a literal hand-worked circle, self-complementary
homology, the six-fragment 50,000 bp boundary, and explicit refusals for a
longer native overlap, repeated closing homology, oversized product and seventh
fragment. The literal cross-check asserts the four exact oligos and both PCR
products, not only a prose algebraic claim.
Fix-round fixtures (17 October 2026)
The 17 October fix round for the 16 and 17 October Gemini commits
(docs/reviews/2026-10-17-recent-work-review/README.md) added these write-once
fixtures. Each was committed before the engine change it checks.
| Fixture | Oracle | Consumer |
|---|---|---|
pipeline/lab-calculators/fixtures/normalization-v1.json |
pipeline/lab-calculators/normalization/reference.py, Python fractions, 12 cases: sample normalisation volumes, the withheld too-dilute rows and the 2 µL flag |
src/lib/dilution/normalization.test.ts |
pipeline/lab-calculators/fixtures/liquid-handler-formats-v1.json |
Transcribed by hand from published documentation, read 17 October 2026: Tecan GWL record layout and examples (TeselaGen, robotools), Echo pick-list columns, nL units and droplet sizes. The exporters were not used to write it. | src/lib/export/liquid-handlers/liquid-handlers.test.ts and the Calc worklist tests |
pipeline/protein-explorer/fixtures/construct-cropper-v1.json |
pipeline/protein-explorer/construct-cropper/reference.py, Biopython ProtParam: tagged construct MW, ε280 (reduced and oxidised) and aliphatic index |
src/lib/seq/construct-cropper.test.ts |
pipeline/protein-explorer/fixtures/pae-v1.json |
pipeline/protein-explorer/pae/reference.py, NumPy: PAE parsing of the current and legacy AlphaFold DB JSON shapes |
src/lib/protein/pae.test.ts |
pipeline/protein-explorer/fixtures/construct-coordinates-v1.json, pae-upload-v1.json |
Written by hand from the F3 acceptance plan; pipeline/protein-explorer/f3/check_v1.py re-derives them (verification 0.9.0) |
src/lib/protein/construct-draft.test.ts, src/lib/protein/pae-decode.test.ts, src/lib/protein/pae-preview.test.ts |
pipeline/molecule-workbench/fixtures/medchem-v1.json |
pipeline/molecule-workbench/medchem/reference.py, RDKit descriptors and the primary definitions of LE (factor 1.3642 kcal/mol at 298.15 K, as Assay's efficiency.ts), LLE and LELP |
src/lib/chem/medchem-sar.test.ts |
pipeline/molecule-workbench/fixtures/seo-evidence-v1.json |
pipeline/molecule-workbench/seo-evidence/reference.py, Python RDKit 2026.03.6 (the shipped library's version), with the described-fragment rule and Morgan settings of fingerprint_policy.py and the build's quantile definition: every computed number on the Molecule reference pages |
content/seo/presets/seo-evidence.test.ts |
The command-palette Tm now calls the suite's oligoMelting, so it is covered by
the existing Tm fixtures. The MCP dose-response tool applies the Assay engine's
own withholding rules, covered by their existing fixtures.
Repeat setup reviewed-input fixture (17 October 2026)
Contract 06 (D-0096 item 1) adds no scientific method: a repeated Calc → Assay setup validates the reviewed replacement inputs, then calls the existing serial-dilution 2.0.0 and plate-sheet 1.0.0 engines. Under D-0090 the mapping still has an independent, write-once fixture, committed before any engine code (8e8fc15): pipeline/workspace/repeat_setup_reference.py (Python fractions, standard library only, CPython 3.13.16) types the rules from the contract and the CSO memo of 16 October 2026 and writes src/lib/workspace/fixtures/repeat-setup-1.json (42 cases) and its line in src/lib/workspace/fixtures/manifest.json.
- 33 reviewed-input cases: eight valid (neat, zero and 0.5% carrier; alias units written back as the table's symbols; a sub-2 µL first transfer flagged, not refused; a 96-well sheet with bulk overage and dead volume; a sheet with absent allowances kept absent; a 384-well sheet with an intermediate made from the replacement stock) with the expected calculator state, carrier fraction, table, every tube's volumes, flags, stock volume, total diluent and, for sheets, the dose wells and totals; and 25 refusals, each with every problem in field order (missing values never defaulted, wrong dimensions, unknown units, non-finite numbers, fractional replicate wells, out-of-range points and carrier, fold 1, zero top, a stock weaker than the top, incomplete or wrong-dimension sheet quantities, out-of-range overage and maximum carrier, and the sheet's own refusals).
- 8 analysis-reconciliation cases: omitted, kept when agreeing (new review day, the newly stated exclusion rule), a difference refused until "use-reviewed-settings" is chosen, a difference omitted, background-subtracted normalisation kept as a v7 setting with no block and refused as a block, and an absolute endpoint kept as reviewed.
Consumers: src/apps/lab-calculators/repeat-setup.test.ts (every reviewed-input case; values within 1e-12 relative, states, units, problems and wells exact; it also asserts the fixture's SHA-256 against the manifest and the script's against about.scriptSha256), src/apps/assay-analyser/repeat-setup.test.ts (every reconciliation case, exactly) and src/lib/workspace/repeat-setup.test.ts (the 0.5% case's table through the whole command). Not published under public/verification/: it verifies a mapping, not a method.
Protein construct and uploaded-PAE coordinate cases (Stage F F3, 24 October 2026)
Verification 0.9.0 (/verification/protein-explorer/0.9.0/, procedure protein.construct-and-pae-upload-coordinates@1.0.0) freezes hand-derived cases written from docs/design/protein-next-acceptance-plan.md before any F3 consumer changed. They test coordinates, admission, display sampling and resource bounds only; no new property, confidence or PAE calculation is made, and the construct numbers stay under construct-cropper-v2.json (0.4.0) and the parser's matrix under pae-v1.json.
pipeline/protein-explorer/fixtures/construct-coordinates-v1.json: on the toy parentP04637-2MACDEFGH, seven tagged species of the existing v2 policy (no tag, N- and C-terminal His6, length-one intervals at either end) with the core, the exact tagged body, whether an initiator Met is added and where the first kept parent residue sits in the body (for 1–8 with N-terminal His6 the parent Met is omitted and A, parent 2, is output 11, so no offset maps one numbering to the other); the C-terminal protease refusal; draft admission (digits only, safe integers, 1 ≤ start ≤ end ≤ length, with the reason that fires first); incoming selected regions; source admission (ASCII letters only:MA CDEFGH,MA-CDEFGH,MACD*EFGH,MAßCDEFGHand the empty sequence are refused, since stripping them would shift indices); the 1,800-residue hand-off limit; the FASTA header's fields.pipeline/protein-explorer/fixtures/pae-upload-v1.json: an asymmetric 3 × 3 matrix (row 1 column 2 is 5 Å, row 2 column 1 is 15 Å) in twelve encodings, its row-major Float64 offsets, fourteen refusals given as raw JSON text (including1e999, a 10⁹ pair index and an unsafe integer), hover on a 300 × 300 box with outside points unavailable, nine colour checkpoints of the existing fixed scale (pae-scale.ts, rounding half up), the nearest-cell preview rule (pixel p shows cell ⌊(p + ½)·N/P⌋ with P = min(N, 1,024); a toy N = 4, P = 2 case whose preview is[5, 7] / [13, 15], not the block means) and the CTO's bounds of 23 October 2026 (16 MiB, side 2,000, 4,000,000 cells, 32,000,000 Float64 bytes, 10 s). One refusal records an existing parser routing: an array whose first entry has apaekey is read as the single-element wrapper, so ani/j/paepair list is refused (REL-694).pipeline/protein-explorer/f3/check_v1.py(standard library) restates each rule and re-derives every literal;python -I pipeline/protein-explorer/f3/check_v1.pyprints "F3 check: 20 construct and 35 PAE cases re-derived".pipeline/protein-explorer/f3/check_v1.test.tschecks the publication's hashes and bytes.
Protein retained-image capture reference
pipeline/protein-explorer/evidence/reference_v1.py uses Python hashlib, independent residue enumeration, decimal formatting and zlib/struct PNG generation. Its write-once pipeline/protein-explorer/fixtures/evidence-v1.json includes script digest, environment, source origins and exact tolerances. It precedes Feature 9 adapter code. Recorded provider bodies remain unchanged; their recording date is not a provider release or runtime retrieval timestamp. Browser proof of actual scene/focus application remains separate.
The second independent receipt reference, reference_v2.py / evidence-v2.json, corrects the RCSB provider identity to the existing registry ID rcsb before the adapter consumes the scene fixture. The original v1 reference is preserved; rcsb-pdb was a draft label and is not the registered source identifier.
The independent v3 reference enumerates recorded canonical and chain sequence pairs across the 3IKA SIFTS segment. It finds the isolated T790M substitution at label residue 99. It does not claim renderer application. See the 18 October CSO correction memo; v1 and v2 remain frozen.
The first public set is /verification/protein-explorer/0.3.0/. Its manifest
pins all three frozen JSON references and Python scripts; v1 stays available as
the superseded draft reference, while the runtime tests consume v2 and v3.
The registry's v2 count of 13 means seven source byte identities, three PNG
boundary inputs and three scene identities; the v3 count of one means the
recorded 3IKA sequence comparison. Mapping, camera-rounding and confidence
inputs also remain in v2, without inflating these two published case counts. src/apps/protein-explorer/evidence-oracle.test.ts reads them, in v1 and v2, against parseSegments/mapToLabel, bandForScore, encodeCamera and, for 3IKA residue 790, compareChain (20 October 2026).
The oracle is Python 3.9.6 standard library (PSF-2.0); v2 records zlib 1.2.12.
The frozen scripts and manifest describe reference provenance, not a green
browser run or a complete validation of procedure 1.0.0.
Protein native settings reference (18 October 2026)
pipeline/protein-explorer/evidence/native_settings_v1.cjs independently reads the pinned Mol* 5.11.0 upstream parameter schemas and invokes its screenshot-helper property transformation with explicit synthetic hardware conditions, without Drybench imports or GPU execution. pipeline/protein-explorer/fixtures/evidence-native-settings-v1.json retains the complete literal specimen and upstream file hashes. pipeline/protein-explorer/evidence/native-settings-v1.md states the method and required fields. These values are test conditions, never production defaults. The fixture precedes the stricter receipt parser.
Protein fallback transport references (18 October2026)
e2e/fixtures/protein-evidence-fallback/index.json pins the RCSB3IKA BinaryCIF response recorded on18 October and a clearly labelled mmCIF transcode of the already recorded AlphaFold BinaryCIF, produced by pinned upstream Mol*5.11.0 without product imports. The live AlphaFold text endpoint returned403; the transcode is not represented as its response bytes. pipeline/protein-explorer/evidence/fallback_v1.py independently checks their lengths, SHA-256 and text-format prefix before browser fallback consumers are introduced.
Protein native camera event fixture
pipeline/protein-explorer/evidence/camera_events_v1.cjs imports only pinned Mol* 5.11.0 Camera and records native events and source hashes. Its independent evidence-camera-events-v1.json shows three anti-aliasing offset events with unchanged snapshots, then two genuine camera moves away and back. It is committed before the evidence observer consumes it. Reproduce with node pipeline/protein-explorer/evidence/camera_events_v1.cjs . /tmp/new-camera-events.json; existing output is refused. This is an event fixture, not proof of browser rendering.
Protein construct tag model reference (20 October 2026)
The audit of 20 October 2026 (docs/reviews/2026-10-20-gemini-codex-audit/reports/gemini-wp.md, C6 and m5) found the construct cropper treating AviTag-His6 as protease-cleavable, reporting a "cleaved" mass equal to the whole construct, forcing a C-terminal request to the N-terminus, and naming the wrong lysine in its BirA note. The CSO's ruling (docs/board/memos/2026-10-20-cso-audit-fixes.md, section 2 item 2) is the specification for the tag model, and it asked for a second published reference.
pipeline/protein-explorer/construct-cropper/reference.py was extended to write pipeline/protein-explorer/fixtures/construct-cropper-v2.json (Biopython 1.88 ProtParam, CPython 3.13.16) and committed before the engine change that consumes it. It types the tag rules from the published tag sequences, assembles every construct itself and refuses to rewrite either fixture; construct-cropper-v1.json is unchanged and still checked. Six cases:
| Case | What it fixes |
|---|---|
ubq_avihis6_nterm, ubq_avihis6_cterm |
AviTag-His6 carries no protease, so it has no cleaved sequence and no cleaved mass at either terminus, and it may sit at either. The C-terminal construct is the core followed by GLNDIFEAQKIEWHEGHHHHHH |
ubq_his6tev_cterm_refused |
A protease tag asked for at the C-terminus is refused with c-terminal-cleavage-not-modelled; the case carries the code and no construct or mass |
ubq_untagged_10_70_initiator_met |
An untagged crop (residues 10–70) that would not start with Met gets one, and the one reported mass, 6,980.82 Da, is of the sequence shown |
ubq_untagged_full_length |
A full-length untagged construct already starts with Met, so none is added |
gfp_his6tev_nterm |
Regression: His6-TEV on GFP is unchanged by the new model, matching the v1 numbers |
The fixture also records the tag model itself (each tag's sequence, its protease or null, whether it is cleavable, the termini it allows and its N-terminal scar) and the AviTag biotinylation facts. The lysine position is derived in the oracle by locating the published 15-residue AviTag GLNDIFEAQKIEWHE inside the stored tag and finding its single lysine: K10 of the motif, residue 11 of MGLNDIFEAQKIEWHEGHHHHHH as stored, because the leading Met shifts it by one. The engine's old note said K12. No biotinylated mass is reported, and the extent of biotinylation is not predicted.
Published, write-once, as /verification/protein-explorer/0.4.0/: the fixture, this oracle as construct_cropper_reference.py, and a manifest.json giving each file's source path, byte length and SHA-256 (the same shape as 0.3.0's manifest). Reproduce and publish with:
venv/bin/python pipeline/protein-explorer/construct-cropper/reference.py
venv/bin/python pipeline/protein-explorer/construct-cropper/reference.py publish 0.4.0
Both refuse to replace a fixture or a version directory that exists, and publish also refuses a directory already on origin/main. Consumer: src/lib/seq/construct-cropper.test.ts, which drives every v2 case through the engine (ε280 and the aliphatic index exact, MW within 1 Da as in v1, assembled sequences, refusal code and added initiator Met exact) and checks the published bytes against the manifest. These are reference values for a designed construct's composition: they say nothing about whether a construct expresses, folds, is cleaved in practice or is biotinylated.
Contract 07A run-comparison selection reference
pipeline/workspace/run_comparison_reference_v1.py independently computes 23
selection, grouping and integrity cases in
src/lib/workspace/fixtures/run-comparison-v1.json before the TypeScript module
exists. Every expected group, task identity, result list, count, integrity reason
and selected-pair refusal is derived from complete input objects. Snapshot
envelopes satisfy the existing validators; adverse cases contain deliberately
missing/tampered evidence. Python compact sorted JSON digests are checked against
the existing snapshot verifier. All outputs have exact tolerance (absolute and
relative zero); no new scientific arithmetic is introduced.
The fixture records script SHA-256, date, contract/memo sources, environment and
tolerances. The script and fixture freeze at this commit; corrections require
new versioned files. The existing Calc publisher accepts arbitrary fixture paths:
python3 pipeline/lab-calculators/publish_preparation.py 0.8.0 src/lib/workspace/fixtures/run-comparison-v1.json.
This publishes the shared-workspace fixture under Calc's verification hosting,
not a new calculator method. 0.8.0 is the next directory above 0.7.0 on
origin/main; no old directory or manifest entry is replaced.
src/lib/fixture-hashes.test.ts checks both script/fixture hashes and public bytes.
Engine, runtime and browser acceptance remain separate work.
src/lib/workspace/run-comparison.test.ts now passes each of those complete
inputs through readRunComparisonInput, runs the independent TypeScript
selection implementation and compares the entire nested output with exact
equality. Separate strict-reader probes reject incomplete/unknown shapes,
unsupported versions, duplicated identities and scope/Study mismatches. The
fixture and oracle remain byte-identical to their first commit.
Patent multiple-NCE presentation reference (18 October 2026)
pipeline/patent-exclusivity/fixtures/multiple-nce-1/ is a new synthetic
reference committed before its consumer tests. The independent standard-library
generate.py reads four input exclusivity rows, groups matching source facts,
and computes calendar-year subtraction with leap-day clamping. Expected output
contains five trace rows, two distinct R4 rows and a latest-summary pointer
into that existing set; an R3 row shares the latest R4 date. This detects both
duplicate-summary appends (SCI-237) and unsafe date-only deduplication. Input
and generator SHA-256 are embedded. --check rederives; output creation
refuses overwrite. No application or established trace helper is imported.
Bench execution component references (20 October 2026)
Calc publication 0.10.0 contains three independent Feature 14 components,
committed before application consumers: 30 frozen-plan cases (15 accepted),
96 plan-reader descriptors, and 395 reducer transitions (98 accepted).
pipeline/workspace/bench_execution_plan_reference_v1.py derives scientific
rows using the existing independent Fraction plate oracle, retains raw units,
source specimens and exact canonical hashes, and checks null positive controls
versus explicit-zero blanks. Scientific comparisons use relative tolerance
1e-9 with zero exact; retained hashes and identities compare exactly.
bench_execution_plan_reader_reference_v1.py specifies structural, consistency,
hash and unsupported-version outcomes independently as literal mutations of
those retained plans. bench_execution_reducer_reference_v1.py is standalone
Python standard library: a synthetic identity catalogue, explicit expected
refusals, retained histories and complete before/after states. It never imports
application code or recalculates scientific quantities. Each script supports
exclusive output creation and byte-for-byte --check. Runtime-only non-JSON
inputs use named descriptors rather than silently repaired JSON.
These references cover their named components only. Full source admission, command replay, transaction atomicity, recovery, interruption and browser acceptance remain separate. In particular, synthetic reducer entry labels are not evidence that persistence UUID or sequence validation has passed.
A fourth component, bench-execution-source-1.json, freezes 54 saved-source
preconditions: 15 accepted specimens, 29 creation/stale-review refusals and
ten consumed-binding cases. Its generator pins the plan fixture and the exact
independent bench_execution_reference.py helper bytes, which are now frozen
for this use. The broader helper remains explicitly incomplete as a full
execution oracle. These source cases precede the creation adapter and do not
establish database or scope atomicity.
The fifth component, bench-execution-commands-1.json, specifies 133 append
commands (20 accepted, seven identical replays and 106 refusals) and 15 strict
record-reader cases. Its independent Python reference imports the frozen plan
and reducer helpers and retains full inputs in deduplicated pools. Cases cover
immutable origin identity through recovery remapping, historical request hashes,
correction chains, revision gaps, replay before stale-revision checks, 5,000
entries and the exact 2 MiB body boundary. The fixture and script are frozen
before consumers. This evidence does not establish database atomicity, live
context permissions, creation admission or recovery integration.
Application consumers now include the strict plan/record readers, pure reducer
and append commands, saved-source capture and transaction wrappers. Focused
tests apply all published command cases and check rollback, concurrency,
idempotent acknowledgements and unchanged sources. Recovery tests use the exact
prior bundle.ts from main 232feefe, retained as test data with SHA-256
bb9af986a09f0d51bd7ddab1a045565a5f911b749274692f2b5d1daa90f3fefa, to prove
its version dispatch refuses a valid format-5 package before writing. Only
relative import locations are rewritten in that compatibility harness; its
dependencies are current, a limitation recorded in the test.
The plan reference pins its original primary-source files. Full scientific
rederivation therefore uses the fixture publication's source snapshot
0ada4f17; later extraction of the shared creation adapter changes source-file
hashes but does not change the published fixture. Consumer tests compare
scientific values under the stated tolerance and hashes against their own exact
retained bytes. No published fixture or expected scientific output was replaced.
Protein source confidence and exact projection reference (Stage F F-1b)
pipeline/protein-explorer/confidence/reference_v1.py is an independent offline
Biopython 1.88/Python reference, committed before the live consumers. Its method
is pipeline/protein-explorer/confidence/method-v1.md; the new immutable
pipeline/protein-explorer/fixtures/confidence-v1.json and source/method are
published as /verification/protein-explorer/0.6.0/.
The reference recomputes every F-1a input using its independent Python oracle, then applies strict source-confidence parsing, exact-only transfer, source colour runs, split selection targets and mean/n/selected-N arithmetic. It includes 39 parser cases, 49 projection cases and 14 source-binding cases. Missing values are explicit JSON nulls; finite scores use binary64 and preserve band boundaries. Integer partners, bands, null masks and counts compare exactly; means tolerate 1e-12 absolute error. Non-finite runtime adverse values use explicit JSON tags, never invalid JSON. Historical offset facts and new exact scientific receipt examples precede writer changes; the accepted input/adapter/ capture version matrix is explicit.
A separate independent check reads existing recorded EGFR model metadata, confidence and transcoded mmCIF using Biopython MMCIF2Dict. It checks the model identity and all 1,210 sequence/label positions and atom B factors, recording the original asset hashes. This establishes representative full-model provider semantics, not runtime coordinate verification for arbitrary models. Unverified fragment domains and conflicting identities/bounds are withheld. It adds no source, changes no licence and never uses AlphaMissense rows.
Generation refuses an existing fixture. Publication requires a numeric direct- child version, refuses existing local/main versions, and recomputes the entire document before creating a destination. Six focused Python checks cover path and symlink refusal, full recomputation, strict duplicates, zero/empty means, split targets and canonical number spelling. Worker lifecycle, actual bundling, consumer coherence and complete receipt round trips require implementation checks separately; the reference alone does not establish live D2 completion.
The live consumer tests now read this frozen reference: src/lib/protein/confidence.test.ts compares every parser/projection/binding result, including binary64 band edges, source runs, holes, means and split targets. Source admission tests cover malformed-present bounds and conflicting CIF fallback identity. residue-map-client.test.ts covers abort during hashing/execution, response generations/bindings/closed shapes, timeout, constructor/posting failures and explicit legacy-only admission. Retained-image tests exercise the new schema2/payload2/adapter2/capture3 tuple, all mixed-version refusals, source-independent hash rereading, loaded-asset consistency, current/source insertion boundaries and lazy/joined projection parity. use-attempt-key.test.ts pins A→B→A ownership and capture identity changes. The focused browser confidence spec uses an explicitly synthetic API double derived from the independent historical insertion case with recorded native atoms; rendering does not establish structural sequence verification.
F-2 runtime reader boundary (25 October 2026)
src/lib/protein/experimental-receipt.test.ts replays the frozen experimental inner, tuple, wire-byte and application-relation publications. Noninteger tuple tokens are tested from original JSON text; ordinary JSON parsing would erase that distinction. New schema3 envelopes preserve their authoritative inner as bounded JSON text through structured cloning and Study bundle transfers. The outer envelope validates stored integer values and does not claim original numeric spelling. Unicode tests reject lone surrogates to match Python UTF-8 rejection; valid astral pairs use their actual UTF-8 widths.
The readable publisher was replayed on CPython3.12.14 with Biopython1.88 and reproduced all three0.7.0 outputs byte-identically. No published bytes or expectations changed. Application-relation cases remain synthetic; live Mol* application is separately tested in the browser. Runtime encoded-retention/output budgets are pinned to limits-v1.json; they are not whole-process memory measurements.
Survival and slow-binding supplemental reference, 25 October 2026
pipeline/assay-analyser/oracles/survival_slow_degeneracy_v1.py publishes only survival-slow-degeneracy-v1.json and its manifest into the new Assay verification version 0.17.0. It was recorded with Python 3.12.14 and no external packages. The 14 survival cases use exact fractions of decimal-string inputs for event-time risk sets, observed and expected counts, hypergeometric covariance, contributing groups and degrees of freedom; finite controls have one degree of freedom, with the chi-square tail from math.erfc(sqrt(x/2)). The 15 slow-binding cases decide the approved relative spread bound exactly, preserve refusal precedence, include tiny genuine rises and rational linear controls, and generate a hyperbola from declared exact constants. Existing R/SciPy slow-binding fixtures remain the test of noisy fitted values and intervals; no new R or SciPy cross-check is claimed.
Replay without rewriting published files: python pipeline/assay-analyser/oracles/survival_slow_degeneracy_v1.py --check. The script refuses existing output paths in creation mode and checks fixture bytes, publication bytes and the source/fixture hashes in check mode. The engine gate is src/lib/stats/survival-slow-degeneracy-oracle.test.ts: statuses, reasons and counts exact, survival statistics and rational line controls 1e-9 relative, p-values 1e-8 relative, hyperbola constants 1e-5 relative. These cases cover SCI-197 and SCI-198 and patch versions 1.0.1 of survival-tests, slow-binding-one-step and slow-binding-two-step; they do not establish general numerical robustness.