Memory budget¶

Every run_job calculation runs through a pre-flight memory check that (a) reports the estimated peak memory in the text output and (b) aborts with an explanation if that estimate exceeds the memory available to the process. On Linux this includes finite scheduler or container cgroup limits, even when the host itself has substantially more free RAM. This is a guardrail against the most common catastrophic failure mode of a QC code, calculations that silently thrash to disk and freeze the host.

What you’ll see in the `.out` file¶

Job: RKS / PBE  basis=cc-pvdz
Atoms (bohr)
------------------------------------------------------
   1  Z=  8       0.00000000      0.00000000      0.00000000
   2  Z=  1       0.00000000      1.43000000     -0.98000000
   3  Z=  1       0.00000000     -1.43000000     -0.98000000
charge=0  multiplicity=1  n_electrons=10

vibe-qc estimates this calculation will require ~0.13 GB of memory:
    ERI tensor            0.00 GB
    Fock + density + 1e   0.00 GB
    DIIS history          0.00 GB
    MO workspace          0.00 GB
    DFT grid + chi        0.10 GB
    Python runtime + NumPy overhead   0.10 GB
Available on this machine: 24.0 GB. Proceeding.

  iter     energy (Ha)            dE          ||[F,DS]||   DIIS
  ...

The headline figure already carries a 1.5x default safety headroom over the sum of the per-category numbers. Set VIBEQC_MEMORY_HEADROOM=2.0 (or another value >= 1.0) to tune the factor for a site or scheduler wrapper.

When the estimate exceeds available RAM¶

vibe-qc estimates this calculation will require ~218.4 GB of memory:
    ERI tensor      186.0 GB
    ...
Available on this machine: 7.2 GB. ABORTING.

InsufficientMemoryError: Set `options.memory_override = True` (or pass
`memory_override=True` to `run_job`) to proceed anyway. Consider a
smaller basis or, once shipped in v0.6+, density fitting / on-disk
scratch.

Overriding the check¶

Pass memory_override=True to run_job:

from vibeqc import Molecule, run_job

mol = Molecule.from_xyz("large.xyz")
run_job(
    mol, basis="def2-tzvp", method="rhf",
    output="huge",
    memory_override=True,    # accept the risk of swap / freeze
)

The output then reads Proceeding (override) instead of Proceeding so anyone reading the log later knows what happened.

Estimators covered¶

Method	Dominant cost	Notes
RHF / UHF	Dense ERI tensor or direct-SCF shell-pair scratch	Direct SCF charges about 1 KB per shell pair
RKS / UKS	HF baseline + DFT grid, weights, coordinates, libxc scratch	meta-GGA adds tau buffers
MP2 / UMP2	OVOV tensor or DF auxiliary integrals and OV workspace	spin-channel storage for UMP2
CCSD / CCSD(T)	T1/T2 amplitudes, D1/D2 intermediates, triples workspace	DF auxiliary integrals are charged separately
DLPNO-MP2 / DLPNO-CCSD(T)	PAO/PNO domains, pair lists, auxiliary pair workspaces, triples domains	conservative local-domain estimate
CAS/CI/FCI	CI or determinant vectors, RDMs, MO integral transforms	exponential methods report their vector storage
Periodic GDF / GPW / GAPW	GDF Lpq factors, bounded Ewald-J FT cache/batches, or FFT-grid collocation/cache arrays	Lpq and grid estimates use dry-run/live preflight; exact Ewald-J uses explicit cache and batch targets

NEB uses a separate per-image-worker model and is intentionally not folded into the single-point estimator.

For molecular grid consumers, including RKS/UKS integration and COSX workspace, the estimate follows the active angular scheme. Product grids use n_theta * n_phi; unpruned Lebedev grids use the bundled point count for lebedev_order. Angularly pruned and ORCA-style five-region grids conservatively charge the densest active angular tier, so preflight remains an upper bound even though some radial shells use fewer points.

Native molecular RKS and UKS evaluate AO values, AO derivatives, density contractions, libxc vectors, and Fock projections in contiguous batches of at most 4,096 points. Analytic molecular XC gradients use the same bound, including their AO-Hessian tables. This does not coarsen or prune the grid: every original Becke point and weight is included in the same order, and only the association of the accumulated floating-point sums changes. Grid coordinates, weights, and atom-ownership indices remain full-grid arrays.

VV10-paired functionals are the nonlocal exception. Their density and gradient invariants are collected over the full grid, the VV10 double integral is evaluated once with all cross-grid pairs present, and only the AO projection of the resulting potential is batched. ROKS, TDDFT response setup, and explicitly enabled molecular Newton/TRAH XC kernels still retain whole-grid AO tables and are charged as dense routes. If the automatic large-RKS tail recovery decides to enable TRAH after a non-converged first attempt, it performs a second dense memory check before starting that retry.

The 3D true-multi-k pure-DFT GDF route retains its analytic-FT Ewald J cache only when the full tensor plus one similarly sized construction temporary fits VIBEQC_J_EWALD3D_CACHE_MIB (4096 MiB by default). Larger cases contract reciprocal vectors and output cells in batches, so they never retain an (n_cells, n_AO, n_AO, n_G) complex tensor. Those batches use the separate VIBEQC_J_EWALD3D_FT_CELL_CHUNK_MIB target (512 MiB by default). Lowering either positive value trades reuse/batch size for memory; neither changes the VIBEQC_J_EWALD3D_KE reciprocal cutoff or the energy convention.

Reading the probe yourself¶

The cross-platform “how much memory is available right now” probe is exposed for scripting:

>>> import vibeqc
>>> vibeqc.available_memory_bytes() / 1024**3
24.3...

It first obtains a host-wide value from psutil.virtual_memory().available (install with pip install psutil to get the highest-quality number), then falls back to /proc/meminfo on Linux and os.sysconf on macOS. On Linux, finite cgroup v2 (memory.max and memory.current) or cgroup v1 memory-controller limits bound that host value. Parent cgroup limits are included, which is important when a scheduler gives each task an unlimited leaf inside a limited job cgroup. The returned value is therefore no greater than either the host’s available memory or the allocation’s remaining memory.

The probe returns 0 if every applicable source fails. check_memory treats that as “unknown” and silently proceeds rather than false-aborting on an unsupported platform. A known exhausted cgroup is not unknown and fails the preflight check.

Writing your own estimator¶

If you call the low-level SCF drivers (run_rhf, run_rks, …) directly instead of through run_job, you can invoke the estimator yourself:

from vibeqc import estimate_memory, check_memory

est = estimate_memory(mol, basis, method="rhf", options=rhf_options)
print(est)                       # human-readable block
check_memory(est)                # raises InsufficientMemoryError if over budget
# ... your own driver call ...

The estimator returns a MemoryEstimate dataclass with a by_category dict, so you can inspect exactly where the memory goes.