Developer Test Lanes

vibe-qc has outgrown a single “run everything” development loop. The executable gate still uses per-file process isolation, but it is now partitioned into lanes so a chat runs the tests for the area it touched.

The lane manifest is:

scripts/test_gate/lane_manifest.json

The lane-aware runner is:

PY=.venv/bin/python
WT=$PWD

$PY scripts/test_gate/run_full_suite.py --wt "$WT" --list-lanes
$PY scripts/test_gate/run_full_suite.py --wt "$WT" --lane pbc-gdf --dry-run
$PY scripts/test_gate/run_full_suite.py --wt "$WT" --changed-since v0.15.41 \
    --list-affected-lanes
$PY scripts/test_gate/run_full_suite.py --wt "$WT" --py "$PY" --lane smoke

Rule

For ordinary development:

  1. Run smoke.

  2. Run every area lane that matches the files you changed.

  3. For release and patch candidates, run --changed-since <base> first. That impact gate runs smoke plus only the affected non-aggregate lanes.

  4. Run full-fast only when the change touches shared infrastructure, core numerical primitives, pytest/gate machinery, final pre-tag confidence, or when no recent nightly covers the tree.

  5. Run slow-nightly and external-reference before release tags, or when those areas are touched.

  6. Keep experimental research lanes out of the every-change gate unless you changed that research area.

This means AICCM real-Gamma/Gamma/chi tests are not run for routine molecular, semiempirical, or unrelated PBC work. PBC lanes are not run for molecular-only work. Molecular semiempirical, periodic semiempirical, AICCM molecular, and AICCM periodic lanes are separate. GAPW, BIPOLE, GDF, embedding, and external reference-code lanes are independent unless a change crosses those boundaries.

Lanes

Lane

Use it for

smoke

Every non-trivial change. Imports, compiled submodule registration, packaging, output manifest, and gate self-tests.

output-docs

vibeqc.output, citations, QVF, manifest, docs-symbol, golden output, and formatting work.

molecular-scf-dft

Molecular HF/DFT, grids, SCF convergence, molecular gradients, Hessians, properties, dispersion.

molecular-correlation

MP2, CC, DLPNO, CAS, CI, TDDFT, excited-state, and solver work.

basis-ecp-integrals

Basis sets, ECPs, integral kernels, DF core, symmetry integrals, and basissetdev architecture.

semiempirical-molecular

Molecular MSINDO, PM6/OMx, GFN2/xTB, DFTB0, runner/output, PES, MD, NEB, and route-plan tests.

semiempirical-periodic

Periodic and k-point semiempirical adapters, DFTB smearing/convergence/FD, and periodic native route checks.

semiempirical-aiccm

MSINDO-CCM and SECCM finite-group bridge tests. On-touch/nightly only.

semiempirical

Aggregate semiempirical lane. Prefer a narrower semiempirical sublane unless the refactor crosses all of them.

pbc-core

Periodic core excluding GDF, BIPOLE, GAPW/GPW, embedding, megacell, and AICCM.

pbc-gdf

Periodic GDF, RSGDF, MDF/compcell, AO-pair FT, and k-point GDF.

pbc-bipole

BIPOLE Ewald split, screened exchange, pair-resolved SR domains, and BIPOLE-backed periodic NEB.

pbc-gapw-gpw

GAPW/GPW, plane-wave grids, augmentation, stress, phonons, PW reference, FFT-Poisson.

aiccm-molecular

Molecular/finite-cluster AICCM and CCM tests.

aiccm-periodic

Periodic AICCM, periodic CCM, low-D, finite-torus, megacell, toroidal-MP2, and SECCM tests.

aiccm-experimental

Aggregate AICCM/CCM research lane. Prefer aiccm-molecular or aiccm-periodic unless the change crosses both.

embedding-surfaces-neb

Periodic embedding, slabs, surface reactions, ASE periodic calculators, and NEB paths.

external-reference

Out-of-process PySCF/ORCA/CRYSTAL/CP2K/GPAW parity boundaries and examples/regression runner tests.

full-fast

All non-slow pytest files. Use before risky shared pushes and release candidates.

slow-nightly

All @slow tests plus env-gated heavy tests. Use on a large box, low concurrency.

Examples

Molecular runner or SCF change:

$PY scripts/test_gate/run_full_suite.py --wt "$WT" --py "$PY" \
    --lane smoke --out smoke.jsonl
$PY scripts/test_gate/run_full_suite.py --wt "$WT" --py "$PY" \
    --lane molecular-scf-dft --out molecular-scf-dft.jsonl

Periodic GDF change:

$PY scripts/test_gate/run_full_suite.py --wt "$WT" --lane pbc-gdf --dry-run
$PY scripts/test_gate/run_full_suite.py --wt "$WT" --py "$PY" \
    --lane smoke --out smoke.jsonl
$PY scripts/test_gate/run_full_suite.py --wt "$WT" --py "$PY" \
    --lane pbc-gdf --out pbc-gdf.jsonl

Molecular semiempirical change:

$PY scripts/test_gate/run_full_suite.py --wt "$WT" --lane semiempirical-molecular --dry-run
$PY scripts/test_gate/run_full_suite.py --wt "$WT" --py "$PY" \
    --lane smoke --out smoke.jsonl
$PY scripts/test_gate/run_full_suite.py --wt "$WT" --py "$PY" \
    --lane semiempirical-molecular --out semiempirical-molecular.jsonl

Periodic semiempirical change:

$PY scripts/test_gate/run_full_suite.py --wt "$WT" --lane semiempirical-periodic --dry-run
$PY scripts/test_gate/run_full_suite.py --wt "$WT" --py "$PY" \
    --lane smoke --out smoke.jsonl
$PY scripts/test_gate/run_full_suite.py --wt "$WT" --py "$PY" \
    --lane semiempirical-periodic --out semiempirical-periodic.jsonl

AICCM real-Gamma/Gamma/chi change:

$PY scripts/test_gate/run_full_suite.py --wt "$WT" --lane aiccm-periodic --dry-run
$PY scripts/test_gate/run_full_suite.py --wt "$WT" --py "$PY" \
    --lane smoke --out smoke.jsonl
$PY scripts/test_gate/run_full_suite.py --wt "$WT" --py "$PY" \
    --lane aiccm-periodic --out aiccm-periodic.jsonl

Patch or release-candidate impact gate:

BASE=v0.15.41
$PY scripts/test_gate/run_full_suite.py --wt "$WT" \
    --changed-since "$BASE" --list-affected-lanes
$PY scripts/test_gate/run_full_suite.py --wt "$WT" \
    --changed-since "$BASE" --dry-run
$PY scripts/test_gate/run_full_suite.py --wt "$WT" --py "$PY" \
    --changed-since "$BASE" --out impact.jsonl
$PY scripts/test_gate/gate_verdict.py impact.jsonl \
    scripts/test_gate/known_reds_baseline.json --reverify --wt "$WT" --py "$PY" \
    --markexpr "not slow"

Final pre-tag confidence:

$PY scripts/test_gate/run_full_suite.py --wt "$WT" --py "$PY" \
    --lane full-fast --out fast.jsonl
$PY scripts/test_gate/run_full_suite.py --wt "$WT" --py "$PY" \
    --lane slow-nightly --out slow.jsonl
cat fast.jsonl slow.jsonl > all.jsonl
$PY scripts/test_gate/gate_verdict.py all.jsonl \
    scripts/test_gate/known_reds_baseline.json --reverify --wt "$WT" --py "$PY"

Regression Suite Under examples/regression

The examples/regression suite is a calculation-parity suite with full artefacts, external-code subprocess runners, and paper/SI candidate outputs. It is not the default pytest lane for every code change.

Use it when the work changes:

  • cross-code parity infrastructure,

  • paper or SI numbers,

  • external reference-code runners,

  • output artefact retention for calculations,

  • public scientific claims that need full input/output evidence.

Routine unit and integration coverage should go through the pytest lanes first. Run examples/regression only for the subset that matches the touched method, system family, or publication claim.

Maintenance

When adding a new expensive test, do one of these in the same change:

  • keep it cheap enough for the relevant area lane,

  • mark it @pytest.mark.slow and make sure it belongs in slow-nightly,

  • add or update a lane glob in scripts/test_gate/lane_manifest.json,

  • document why the test is experimental or external-reference only.

When a lane’s file list becomes noisy, update the manifest rather than telling every chat to remember a private command. The manifest is the contract.