QVF writer — handover / status

Last updated: 2026-06-09 Branch: main Agent session: QVF writer — v1.1 infrastructure (M20-M23)

What this is

The vibe-qc producer half of the QVF (Quantum Visualisation Format) pipeline. Writes .qvf files from SCF results. The consumer half lives in vibe-view/ (see vibe-view/HANDOVER.md). Both halves share the design document at docs/design_qvf_format.md.

Status: v1.1 complete. All 23 milestones done (99 tests).

Current progress

2026-06-09 — basis-coefficient normalization fix. qvf_wf_data now divides each contraction coefficient by the primitive norm N(alpha, l) = (2*alpha/pi)^(3/4) * (4*alpha)^(l/2) / sqrt((2l-1)!!) before writing, so the output matches QVF spec Sec. 4.6. Previously the writer emitted libint’s stored coefficients verbatim (which already include the norm factor), causing the vibe-view consumer to double-normalize. The fix is the same transform molden.py has always applied at line 174.

Milestone	Status	Tests
M1–M13 (v1.0 — core writer, all original section kinds, validator, CLI, round-trip, runner integration, guardrails, schema)	✅ done	38
M14 — dos.total / dos.projected writer	✅ done	3 (TestDOSCoexistence)
M15 — volume.potential writer	✅ done	1 (TestRoundTrip.test_volume_potential_roundtrip)
M16 — volume.rdg writer	✅ done	1 (TestRoundTrip.test_volume_rdg_roundtrip)
M17 — fermi_surface writer	✅ done	1 (TestRoundTrip.test_fermi_surface_roundtrip)
M18 — phonon_bands / phonon_dos writer	✅ done	2 (TestRoundTrip.test_phonon_bands_roundtrip, test_phonon_dos_roundtrip)
M19 — equation_of_state writer	✅ done	1 (TestRoundTrip.test_eos_roundtrip)
M20 — Root metadata (thermochemistry, dipole_moment, constraints)	✅ done	4 (TestRoundTrip: thermo, dipole, constraints, extensions)
M21 — extensions governance block in manifest + per-section critical/schema_uri	✅ done	3 (TestValidateQVF: critical reserved, critical vendor, extensions block)
M22 — JSON Schema v1.1 update	✅ done	— (schema is the contract)
M23 — Validator v1.1 updates (critical check, extensions resolution, new kinds)	✅ done	3 (critical enforcement + extensions validation)

Section kinds implemented (v1.0)

The authoritative list — and the contract for each kind’s member set — is the JSON Schema at python/vibeqc/output/formats/qvf_manifest.schema.json. The summary below matches the schema’s Section.oneOf branches one-to-one; the schema-drift guard test (tests/test_qvf_schema_drift.py) keeps the two in lock-step.

Kind	Required members
structure	structure (JSON: atoms + pbc + optional lattice_vectors).
volume.density / volume.orbital / volume.spin / volume.elf / volume.difference / volume.generic	grid (JSON) + data (binary float32/64 3-D).
wavefunction.gto	basis + mo_metadata (JSON) + mo_coefficients (or alpha+beta).
atom_properties	one or more of mulliken_charge, loewdin_charge, spin_population.
trajectory	metadata (JSON) + coords (binary float64 [n_frames, n_atoms, 3]).
reaction.path	same as trajectory + waypoints.
reaction.waypoints	waypoints (JSON) + trajectory_ref.
vibrations	metadata (JSON) + displacements (binary float64 [n_modes, n_atoms, 3]).
spectra.* (ir / raman / uvvis / ecd / vcd / nmr / generic)	spectrum (JSON).
bands	kpath (JSON) + eigenvalues (binary float64 [n_spin, n_kpoints, n_bands]).
structure.symmetry	data (JSON).
bonds	bonds (JSON: {pairs: [...]}).
scf_history	iterations (JSON).
citations	references (binary BibTeX).

v1.1 — New section kinds to implement

All seven new kinds are specified in docs/design_qvf_format.md §§ 4.8–4.14. Each follows the established pattern: a JSON metadata member + one or more binary members. Add them to the same three places: schema Section.oneOf, a _write_<kind>_section() function, and _IMPLEMENTED_KINDS.

M14 — dos.total

Role members:

Role	Format	Content
energies	binary float64 [n_points]	Energy grid in eV, referenced to E_F at 0.0 eV
dos	binary float64 [n_points] or [2, n_points]	Total DOS; rank-1 for restricted, rank-2 [alpha, beta] for spin-polarized
meta (JSON)	optional	Smearing, smearing_type, fermi_energy_ev, n_electrons, n_spin

Writer function: _write_dos_total_section()

Accepts energies: np.ndarray, dos: np.ndarray, and optional meta: dict. Period: one section per dos.total. vibe-qc’s band- structure pipeline already has DOS data in python/vibeqc/periodic_bands.py (the get_dos() / get_pdos() methods on BandStructure). The writer should extract the relevant arrays and emit them.

Validator additions:

• energies.shape[0] == dos.shape[-1]

• dos.ndim in (1, 2); if ndim == 2, dos.shape[0] in (1, 2) (spin channels)

Test pattern: Build a synthetic energy grid + Lorentzian DOS, write, round-trip, assert shape + values within tolerance.

M15 — dos.projected

Role members:

Role	Format	Content
energies	binary float64 [n_points]	Same energy grid (may share binary with dos.total)
projections	binary float64 [n_channels, n_points] or [n_spin, n_channels, n_points]	Per-channel DOS intensities
meta (JSON)	required	channels array: each channel has atom_index, symbol, l, label

Writer function: _write_dos_projected_section()

Accepts energies, projections, and meta (with channels list). The projections tensor must have the documented rank semantics.

Validator additions:

• energies.shape[0] == projections.shape[-1]

• If projections.ndim == 2, treat as [n_ch, n_pts] restricted.

• If projections.ndim == 3, projections.shape[0] == n_spin.

• len(meta["channels"]) must match projections.shape[-2].

Test pattern: 3-atom system with s + p channels per atom, write, round-trip, assert channel labels match.

M16 — volume.potential

Already reserved; promoted to implemented. Identical member structure to volume.density. The writer already has _write_volume_density_section() which is the template; a new _write_volume_potential_section() is a thin wrapper that sets kind = "volume.potential".

Role members: Same as volume.density — grid (JSON) + data (binary float32/64).

Data convention: Electrostatic potential in hartree/e (atomic units). Conversion from V to E_h/e: 1 hartree/e ≈ 27.2114 V.

Writer function: _write_volume_potential_section()

If vibe-qc’s ESP evaluator exists (e.g., via vibeqc.cube), call it here. Otherwise, the writer takes a pre-computed numpy array (same caller contract as qvf_density_data()).

Validator additions: None beyond the standard volume checks.

M17 — volume.rdg

A new volume kind for the reduced density gradient. Same structure as volume.density.

Role members: Same as volume.density — grid (JSON) + data (binary float32/64).

Data convention: Dimensionless s(r) = |∇ρ| / (2(3π²)^⅓ ρ^⅔). The field is dimensionless float32/64.

Writer function: _write_volume_rdg_section()

Producer computes RDG from density + gradient. If vibe-qc has an existing RDG evaluator (in the cube module or DFT grid), wrap it here. Otherwise, pre-computed numpy array.

M18 — fermi_surface

Role members:

Role	Format	Content
mesh (JSON)	required	nk1, nk2, nk3, n_spin, fermi_energy_ev, band_indices, lattice_vectors
energies	binary float64 [nk1, nk2, nk3, n_bands]	Signed band energies: E(k) − E_F

Writer function: _write_fermi_surface_section()

Accepts a Monkhorst-Pack mesh shape, a band_indices list, and the corresponding rank-4 energy tensor. The lattice vectors come from the periodic system.

Data flow in vibe-qc: After a periodic SCF run with a full k-mesh (not just a band-structure k-path), the MO energies at each k-point are available. The writer selects bands within a window around E_F (producer’s choice, ±2 eV suggested) and writes the sub-tensor.

Validator additions:

• energies.ndim == 4

• energies.shape[:3] == (mesh.nk1, mesh.nk2, mesh.nk3)

• energies.shape[3] == len(mesh.band_indices)

Design note: The grid is assumed gamma-centered and uniformly spaced. If vibe-qc shifts the MP mesh (e.g., gamma-centered convention), the consumer needs to know the offset. Include an optional k_offset field in mesh JSON: [0.0, 0.0, 0.0] for gamma-centered, [0.25, 0.25, 0.25] for a typical shifted mesh.

M19 — phonon_bands and phonon_dos

These require a phonon calculation (Hessian on a q-point grid), which vibe-qc does not yet produce. The writer should implement the section writer so that when phonon data becomes available (v0.22+ roadmap), the QVF path is ready.

phonon_bands

Role members:

Role	Format	Content
qpath (JSON)	required	n_atoms, n_modes, has_eigenvectors, segments (same structure as bands.kpath)
frequencies	binary float64 [n_qpts, n_modes]	Phonon frequencies in cm⁻¹
eigenvectors (optional)	binary float64 [n_qpts, n_modes, n_atoms, 3]	Cartesian displacement vectors

Writer function: _write_phonon_bands_section()

phonon_dos

Role members:

Role	Format	Content
meta (JSON)	optional	smearing, smearing_type, n_atoms, n_modes
frequencies	binary float64 [n_points]	Frequency grid in cm⁻¹
dos	binary float64 [n_points]	Total phonon DOS
projected (optional)	binary float64 [n_atoms, n_points]	Atom-projected phonon DOS

Writer function: _write_phonon_dos_section()

M20 — equation_of_state

Role members:

Role	Format	Content
volumes	binary float64 [n_points]	Cell volumes in Angstrom^3
energies	binary float64 [n_points]	Total energies in eV
fit (JSON)	required	EOS fit parameters: model, V0, E0, B0, B0_prime, energy_unit, volume_unit, pressure_unit, residual_rms, pressures_gpa

Writer function: _write_eos_section()

Accepts volume and energy arrays plus a fit dictionary. The fit parameters should ideally come from an EOS fitting routine (e.g., vibeqc.periodic_eos if it exists, or ASE’s eos module).

Validator additions:

• volumes.shape == energies.shape

• fit.model is one of "birch_murnaghan", "murnaghan", "vinet"

• len(fit.pressures_gpa) matches volumes.shape[0] or is null

M21 — Root metadata blocks

The manifest root now accepts three optional metadata blocks alongside provenance and viewer_defaults:

• thermochemistry — dictionary with keys from docs/design_qvf_format.md § 2.1.

• dipole_moment — dictionary with total_debye, vector_debye, origin.

• constraints — dictionary with frozen_atoms, frozen_lattice, and constraint arrays.

Implementation: These are emitted by the caller into write_qvf() as an optional root_metadata keyword argument (dict). The writer merges them into the manifest root before serialization. Example:

write_qvf(
    "result.qvf", sections=..., molecule=...,
    root_metadata={
        "thermochemistry": {"zpve_eh": 0.0294, ...},
        "dipole_moment": {"total_debye": 1.85, ...},
    }
)

Manifest shape:

{
  "qvf_version": 1,
  "source": { ... },
  "thermochemistry": { ... },
  "dipole_moment": { ... },
  "constraints": { ... },
  "sections": [ ... ]
}

M22 — extensions governance block

The writer must emit the root extensions block whenever the archive contains x_<vendor>.* sections with critical: true.

Implementation: Add an optional extensions keyword argument to write_qvf():

write_qvf(
    "result.qvf", sections=...,
    extensions={
        "x_vendor_ecp": {"version": "1.0", "critical": True}
    }
)

The writer merges this into the manifest root under "extensions".

Each section that is critical: true must carry "critical": true in its section object. Add an optional section_critical argument to section builder helpers, or accept it in the section dict directly.

Schema update: The root schema must now accept:

• Optional extensions object with "^x_[a-z][a-z_]*$" pattern keys.

• Each value: version (string), optional schema_uri (string, URI), optional critical (boolean, default false).

• Per-section critical (boolean, default false) and schema_uri (string, URI, optional).

M23 — Validator v1.1 updates

Add to validate_qvf():

• If any section has critical: true and the kind is not in _IMPLEMENTED_KINDS (and not x_<vendor>.*), flag an error.

• If extensions declares an extension as critical: true, verify that the archive actually contains a section with a matching x_<vendor> prefix.

• Cross-check: a section with critical: true in an x_* namespace must have its namespace declared in extensions.

• Validate the new kind shapes (dos.*, fermi_surface, phonon_bands, equation_of_state).

JSON Schema v1.1 — checklist

Update python/vibeqc/output/formats/qvf_manifest.schema.json:

• Add "thermochemistry", "dipole_moment", "constraints" as optional properties of the root manifest schema. Each with the field shapes from the spec.

• Add "extensions" with "additionalProperties" constrained to the extension value schema.

• Add dos.total, dos.projected, volume.potential, volume.rdg, fermi_surface, phonon_bands, phonon_dos, equation_of_state to Section.oneOf with their member contracts.

• Add critical and schema_uri to the base section schema.

• Bump the schema $id to reflect v1.1 (e.g., https://vibe-qc.org/spec/qvf/1/manifest.schema.json — the 1 is the major version; minor is implicit).

• Run the schema-drift guard test (tests/test_qvf_schema_drift.py) to verify the schema and writer are in sync.

Implementation order (recommended)

1. M22 (extensions governance + per-section critical) — touches the manifest root structure least invasively and is a prerequisite for any vendor extension workflow.

2. M21 (root metadata) — another manifest-root change, orthogonal to sections.

3. M23 (schema + validator updates) — do this after M21+M22 so the schema validates the new root fields.

4. M16 (volume.potential) — easiest new kind, reuses existing volume infrastructure.

5. M14+M15 (dos.total + dos.projected) — natural next step given vibe-qc already computes DOS in periodic_bands.py.

6. M17 (volume.rdg) — depends on having gradient data from the cube evaluator.

7. M18 (fermi_surface) — needs full-mesh k-point data from the periodic SCF.

8. M19+M20 (phonons, EOS) — depend on upstream vibe-qc compute capabilities not yet available; writer can be stubbed with a NotImplementedError and a clear message.

Blockers

• volume.rdg writer needs the density gradient evaluator in vibeqc.cube to be callable for RDG (reduced-gradient computation from ρ and ∇ρ). If the evaluator doesn’t exist yet, the writer accepts pre-computed arrays (same pattern as qvf_density_data()).

• fermi_surface writer needs the periodic SCF to expose the full Monkhorst-Pack eigenvalue tensor (not just the k-path subset used by bands). This may require a refactor in the periodic SCF driver.

• phonon_bands / phonon_dos cannot be written until vibe-qc has a periodic phonon/Hessian module (planned for v0.22+). The section writer can be implemented on spec, tested with synthetic data, and left dormant.

• equation_of_state writer needs an EOS fitting routine. If none exists in vibe-qc, the writer can accept pre-fitted parameters and just package them.

Tests

Add for each new kind (follow tests/test_qvf_writer.py pattern):

• Unit test: construct synthetic input → call the writer → verify manifest structure and member byte count.

• Round-trip test: write synthetic section → re-read with QVFReader → assert data values within tolerance.

• Schema conformance: emit a manifest JSON for the new kind and assert it passes validate_qvf().

• Guardrail test: misshapen data rejects cleanly (wrong ndim, wrong dtype, mismatched lengths).

• Extension governance test: archive with critical: true on an unsupported kind → validation error.

• Root metadata test: archive with thermochemistry / dipole_moment / constraints → manifest contains the expected fields.