Design — QVF container format for visualization data

Status: draft — this document describes the QVF format as understood by the vibe-view consumer. The producer-side contract lives in docs/design_output_module.md Phase O6 (volumetric writers) and will be extended as writers land.

A QVF file (.qvf, “quantum visualization format”) is a zip archive containing a JSON manifest and typed binary payloads. It is the interchange format between vibe-qc’s computational core and both the terminal viewer (moltui, already handled) and the GPU-accelerated interactive viewer (vibe-view, this design).

1. QVF archive layout

example.qvf
├── manifest.json          # required, must be first logical entry
├── sections/
│   ├── structure.json     # atoms, lattice vectors, pbc flags
│   ├── bonds.json         # optional explicit bond table
│   ├── density.dat        # binary volumetric blob (float32)
│   ├── orbital_5.dat      # another volumetric blob
│   ├── bands.json         # band path metadata (segments, Fermi)
│   ├── eigenvalues.dat    # binary eigenvalues array
│   ├── ir_spectrum.json   # frequencies + intensities
│   ├── trajectory.json    # frames metadata
│   ├── traj_coords.dat    # binary per-frame coordinates
│   ├── vib_modes.json     # normal mode metadata
│   ├── vib_displacements.dat  # binary normal mode displacements
│   └── viewer_defaults.json   # optional viewer hints

1.1 manifest.json — the central index

{
  "qvf_version": 1,
  "source": {
    "program": "vibe-qc",
    "version": "0.9.0",
    "calculation": "h2o_pbe0_def2tzvp"
  },
  "sections": [
    {
      "id": "structure",
      "kind": "structure",
      "members": {
        "structure": {
          "path": "sections/structure.json",
          "format": "json",
          "sha256": "abcdef..."
        }
      }
    },
    {
      "id": "density",
      "kind": "volume.density",
      "members": {
        "grid": {
          "path": "sections/density.grid.json",
          "format": "json",
          "sha256": "123456..."
        },
        "data": {
          "path": "sections/density.dat",
          "format": "binary",
          "dtype": "float32",
          "shape": [72, 80, 80],
          "sha256": "789abc..."
        }
      }
    },
    {
      "id": "homo",
      "kind": "volume.orbital",
      "component": "real",
      "members": {
        "grid": {
          "path": "sections/homo.grid.json",
          "format": "json",
          "sha256": "..."
        },
        "data": {
          "path": "sections/homo.dat",
          "format": "binary",
          "dtype": "float32",
          "shape": [72, 80, 80],
          "sha256": "..."
        }
      }
    },
    {
      "id": "bands",
      "kind": "bands",
      "members": {
        "kpath": {
          "path": "sections/kpath.json",
          "format": "json",
          "sha256": "..."
        },
        "eigenvalues": {
          "path": "sections/eigenvalues.dat",
          "format": "binary",
          "dtype": "float64",
          "shape": [100, 26],
          "sha256": "..."
        }
      }
    },
    {
      "id": "ir",
      "kind": "spectra.ir",
      "members": {
        "spectrum": {
          "path": "sections/ir_spectrum.json",
          "format": "json",
          "sha256": "..."
        }
      }
    },
    {
      "id": "opt_trajectory",
      "kind": "trajectory",
      "members": {
        "metadata": {
          "path": "sections/trajectory.json",
          "format": "json",
          "sha256": "..."
        },
        "coords": {
          "path": "sections/traj_coords.dat",
          "format": "binary",
          "dtype": "float32",
          "shape": [15, 3, 3],
          "sha256": "..."
        }
      }
    },
    {
      "id": "vib",
      "kind": "vibrations",
      "members": {
        "metadata": {
          "path": "sections/vib_modes.json",
          "format": "json",
          "sha256": "..."
        },
        "displacements": {
          "path": "sections/vib_displacements.dat",
          "format": "binary",
          "dtype": "float64",
          "shape": [9, 3, 3],
          "sha256": "..."
        }
      }
    }
  ],
  "viewer_defaults": {
    "auto_open": ["density"],
    "density": {
      "isovalue": 0.05,
      "colormap": "viridis",
      "opacity": 0.6
    }
  }
}

1.2 structure.json

{
  "atoms": [
    {"symbol": "O", "position": [0.0, 0.0, 0.1173], "atomic_number": 8},
    {"symbol": "H", "position": [0.0, 0.7572, -0.4692], "atomic_number": 1},
    {"symbol": "H", "position": [0.0, -0.7572, -0.4692], "atomic_number": 1}
  ],
  "pbc": [false, false, false],
  "lattice_vectors": null
}

For periodic systems, lattice_vectors is a [3, 3] array of floats in angstroms and pbc has one or more true entries.

1.3 volume.*.grid.json

{
  "origin": [0.0, 0.0, 0.0],
  "voxel_vectors": [
    [0.2, 0.0, 0.0],
    [0.0, 0.2, 0.0],
    [0.0, 0.0, 0.2]
  ],
  "shape": [72, 80, 80]
}

The grid is a 3D Cartesian grid defined by an origin plus three voxel basis vectors (columns of the cell matrix). For orthogonal grids the off-diagonals are zero. For non-orthogonal grids (e.g. from a periodic cell), the off-diagonals position the volume correctly in 3D space.

1.4 Content kinds

Kind

Description

v1 supported?

structure

3D molecular/crystal view

✅

bonds

Explicit bond table (consulted if present, inferred if absent)

✅ (via structure renderer)

volume.density

Scalar field for isosurface (electron density)

✅

volume.orbital

Scalar field for isosurface (molecular orbital)

✅

bands

Band structure path + eigenvalues

✅

spectra.ir

IR spectrum (frequencies + intensities)

✅

trajectory

Frame-by-frame animation (opt/IRC)

✅

vibrations

Normal-mode displacement animation

✅

atom_properties

Mulliken/Löwdin charges table

✅

x_<vendor>.*

Vendor-namespace extensions

❌ (skipped, vendor namespace)

2. Design note — vibe-view interactive viewer

2.1 Tech stack choice: PyVista + Trame

Choice: PyVista with the Trame web backend.

Why not the alternatives:

Alternative

Rejected because

Trame + VTK directly

Steeper learning curve; PyVista wraps VTK with a much cleaner Python API that still exposes all the GPU features. We get the same VTK power without the boilerplate.

Plotly Dash + VTK.js

Requires a JavaScript build step (npm, webpack) — breaks the “Python-only” repo pattern. VTK.js is a JS reimplementation with a smaller feature set than native VTK.

FastAPI + React/Three.js

Two-language project, frontend build toolchain, npm lockfile churn, bundler config. Massive complexity for a viewer that should “just work” after pip install.

Why PyVista + Trame:

  1. Pure Python — no npm, no JavaScript build step, no two-language project. Matches the existing repo pattern (vibe-queue is also Python-only: FastAPI + Jinja2).

  2. GPU-accelerated marching cubes — VTK’s vtkMarchingCubes runs on the GPU; interactive frame rates on 200³ grids are well within VTK’s capabilities.

  3. Built-in interactivity — rotate, zoom, pan all handled by VTK’s interactor. Trame adds web-serving with zero additional code.

  4. Isosurface + colormap + opacity — all first-class PyVista features (add_mesh with scalars, cmap, opacity).

  5. Maturity — PyVista and VTK are the standard Python 3D visualization stack in scientific computing. Trame is Kitware’s recommended web backend.

  6. Existing ecosystem alignment — vibe-qc already ships moltui[viewer] for terminal visualization; PyVista+Trame is its natural GPU-accelerated GUI/web complement.

Licensing: PyVista (MIT), Trame (Apache 2.0), VTK (BSD 3-clause). All MPL 2.0 compatible. No bundling needed — pure pip dependencies.

2.2 Module layout

Following the vibe-queue pattern (separate top-level directory with own pyproject.toml, co-located in the same repo):

vibe-view/
├── pyproject.toml              # package "vibeview", CLI "vibe-view"
├── src/
│   └── vibeview/
│       ├── __init__.py         # version, package metadata
│       ├── __main__.py         # python -m vibeview
│       ├── cli.py              # click CLI: vibe-view open <file.qvf>
│       ├── qvf.py              # QVF reader: zip open, manifest parse,
│       │                       #   schema validate, sha256 verify,
│       │                       #   lazy binary extraction
│       ├── schema.json         # JSON Schema for manifest.json
│       ├── kinds.py            # SUPPORTED_KINDS registry + filter/sort
│       ├── viewer_defaults.py  # parse + apply viewer_defaults hints
│       ├── banner.py           # summary banner (section listing)
│       ├── renderers/
│       │   ├── __init__.py     # kind → renderer dispatch table
│       │   ├── structure.py    # 3D structure: atoms, bonds, unit cell
│       │   ├── volume.py       # isosurface: marching cubes, slider, cmap
│       │   ├── bands.py        # 2D band structure plot
│       │   ├── spectra.py      # 1D IR spectrum plot
│       │   ├── trajectory.py   # frame animation + energy plot
│       │   └── vibrations.py   # normal-mode displacement animation
│       └── app.py              # Trame application: sidebar, layout,
│                               #   lazy section activation, UI controls
└── tests/
    ├── __init__.py
    ├── test_qvf.py
    ├── test_kinds.py
    └── test_renderers.py

Relationship to the main vibe-qc package: vibe-view is a peer sub-project, not a sub-package of vibeqc. The main pyproject.toml gains an optional viewer-gpu extra group that depends on vibeview (editable install from the vibe-view/ directory).

2.3 Kind-to-renderer mapping

# vibeview/kinds.py
SUPPORTED_KINDS: set[str] = {
    "structure",
    "volume.density",
    "volume.orbital",
    "bands",
    "spectra.ir",
    "trajectory",
    "vibrations",
}

# vibeview/renderers/__init__.py
_KIND_RENDERER: dict[str, type] = {
    "structure":      StructureRenderer,
    "volume.density": VolumeRenderer,
    "volume.orbital": VolumeRenderer,  # same class, different defaults
    "bands":          BandsRenderer,
    "spectra.ir":     SpectraRenderer,
    "trajectory":     TrajectoryRenderer,
    "vibrations":     VibrationsRenderer,
}

def dispatch(section: Section, viewer: "QVFViewer") -> BaseRenderer:
    kind = section.kind
    cls = _KIND_RENDERER.get(kind)
    if cls is None:
        raise UnsupportedKindError(kind)
    return cls(section, viewer)

2.4 File-open lifecycle

User: vibe-view open h2o.qvf
│
├─ 1. QVFReader.__init__(path)
│     ├─ Open zipfile.ZipFile(path)
│     ├─ Read manifest.json from zip → parse JSON
│     ├─ Validate against JSON Schema (jsonschema)
│     │   └─ ValidationError → hard error, exit
│     └─ Parse sections list into Section dataclasses
│
├─ 2. banner.print_summary(sections)
│     ├─ For each section:
│     │   ├─ kind in SUPPORTED_KINDS → "rendered"
│     │   ├─ kind matches x_<vendor>.* → "skipped, vendor namespace (<vendor>)"
│     │   └─ else → "skipped, unsupported"
│     └─ Print table to stdout
│
├─ 3. Process viewer_defaults (if present)
│     ├─ Parse auto_open list: which section ids to open by default
│     ├─ Parse per-section hints: isovalue, colormap, opacity, replication
│     └─ Store as mutable state (UI can override any hint)
│
├─ 4. Load structure section eagerly
│     ├─ Verify sha256 of structure.json member against manifest
│     ├─ Parse structure.json → atoms, pbc, lattice_vectors
│     └─ Render 3D structure in main viewport
│
├─ 5. Build sidebar tree
│     ├─ Every section listed (rendered + skipped)
│     ├─ Skipped sections greyed out with reason
│     └─ Click on a section → lazy activation
│
├─ 6. Auto-open viewer_defaults sections
│     └─ For each section id in auto_open:
│         ├─ If section is a volume.*: read grid.json (eager),
│         │   defer .dat blob (lazy — read when user activates)
│         ├─ Verify sha256 of grid.json → if mismatch, skip this section
│         └─ Add isosurface to viewport with default hints
│
└─ 7. Start Trame server, open browser

2.5 Partial-support contract — the four rules

Implemented in vibeview/kinds.py and vibeview/qvf.py:

Rule 1 — supported kinds are declared explicitly:

# vibeview/kinds.py
SUPPORTED_KINDS: frozenset[str] = frozenset({
    "structure",
    "volume.density",
    "volume.orbital",
    "bands",
    "spectra.ir",
    "trajectory",
    "vibrations",
})

Rule 2 — unknown kinds are listed as “skipped, unsupported”:

def classify_section(kind: str) -> tuple[str, str | None]:
    """Return (status, detail) for a section kind."""
    if kind in SUPPORTED_KINDS:
        return ("rendered", None)
    if kind.startswith("x_"):
        # vendor namespace: "x_<vendor>.<rest>"
        parts = kind.split(".", 1)[0].split("_", 1)
        vendor = parts[1] if len(parts) > 1 else "unknown"
        return ("skipped", f"vendor namespace ({vendor})")
    return ("skipped", "unsupported")

Rule 3 — vendor-namespace sections get the vendor name extracted:

The x_<vendor> prefix is parsed and the vendor name is displayed in the summary banner. The section is not rendered, nor does it cause an error.

Rule 4 — sha256 verification before use:

def _verify_member(zf: zipfile.ZipFile, member: MemberSpec) -> bytes:
    """Read a binary member and verify its sha256. Hard error on mismatch."""
    data = zf.read(member.path)
    actual = hashlib.sha256(data).hexdigest()
    if actual != member.sha256:
        raise SHA256MismatchError(
            f"sha256 mismatch for {member.path}: "
            f"expected {member.sha256}, got {actual}"
        )
    return data

Every call site that reads a binary member calls _verify_member first. JSON members are also verified. The mismatch is a hard error for that section only — the viewer continues to render other sections and lists the failed section in the banner as “error, sha256 mismatch”.

2.6 Dependencies

Core (pip install -e '.[gpu]' or pip install vibeview):

  • pyvista>=0.44 — 3D rendering, marching cubes, VTK bindings

  • trame>=3.6 — web application server

  • trame-vtk>=2.8 — VTK widgets in Trame

  • trame-vuetify>=2.7 — Material Design UI components (sliders, dropdowns, tree views)

  • click>=8.0 — CLI entry point

  • pydantic>=2.0 — data models for manifest sections

  • jsonschema>=4.20 — manifest.json validation

  • numpy>=1.26 — array handling

  • matplotlib>=3.8 — 2D plots (bands, spectra, energy curve)

Optional:

  • ase>=3.22 — covalent radii for bond inference (or we ship a small table)

3. QVF file generation (producer side)

The producer is implemented at python/vibeqc/output/formats/qvf.py. See docs/handover_qvf_writer.md for status, coverage, and next steps.

Quick start:

result = run_job(mol, basis="sto-3g", method="rhf",
                 output="h2o", output_qvf=True)
# → h2o.qvf

The manifest shape emitted by the producer matches this consumer contract exactly (qvf_version, source, sections[].members). 10 v1 section kinds are implemented.

4. JSON Schema for manifest.json

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://vibe-qc.com/schemas/qvf-manifest-v1.json",
  "title": "QVF Manifest",
  "type": "object",
  "required": ["qvf_version", "source", "sections"],
  "properties": {
    "qvf_version": {"type": "integer", "const": 1},
    "source": {
      "type": "object",
      "required": ["program", "version", "calculation"],
      "properties": {
        "program": {"type": "string"},
        "version": {"type": "string"},
        "calculation": {"type": "string"}
      }
    },
    "sections": {
      "type": "array",
      "items": {
        "type": "object",
        "required": ["id", "kind", "members"],
        "properties": {
          "id": {"type": "string"},
          "kind": {"type": "string"},
          "component": {"type": "string"},
          "members": {
            "type": "object",
            "additionalProperties": {
              "type": "object",
              "required": ["path", "format", "sha256"],
              "properties": {
                "path": {"type": "string"},
                "format": {"type": "string", "enum": ["json", "binary"]},
                "dtype": {"type": "string"},
                "shape": {
                  "type": "array",
                  "items": {"type": "integer"}
                },
                "sha256": {"type": "string", "pattern": "^[a-f0-9]{64}$"}
              }
            }
          }
        }
      }
    },
    "viewer_defaults": {
      "type": "object",
      "properties": {
        "auto_open": {
          "type": "array",
          "items": {"type": "string"}
        }
      },
      "additionalProperties": {
        "type": "object",
        "properties": {
          "isovalue": {"type": "number"},
          "colormap": {"type": "string"},
          "opacity": {"type": "number", "minimum": 0, "maximum": 1},
          "replication": {
            "type": "array",
            "items": {"type": "integer", "minimum": 1},
            "minItems": 3,
            "maxItems": 3
          }
        }
      }
    }
  }
}