Updating vibe-qc¶
For an existing checkout you’ve already used. If this is your first install, see installation instead.
This page covers the native vibe-qc engine. For the companion viewer, queue, and basis driver commands, see Install and maintain the vibe toolset.
Variants¶
./scripts/update.sh --release # latest tagged release (default; flag is for symmetry)
./scripts/update.sh --dev # bleeding-edge main (X.Y.devN banner)
./scripts/update.sh --branch main # verbose form of --dev
./scripts/update.sh --branch basissetdev # any other branch -- e.g. the basissetdev paper-writing branch
./scripts/update.sh --branch v0.9.0 # pin to a specific tag
./scripts/update.sh --ref v0.9.0 # older spelling of --branch, kept for back-compat
./scripts/update.sh --rebuild-native-deps # see "Vendored library version bumps" below
./scripts/update.sh --recreate-venv # atomic venv replacement with rollback
./scripts/update.sh --recreate-venv --python python3.13 # intentionally change Python
./scripts/update.sh --with-openblas # add or retain vendored OpenBLAS
./scripts/update.sh --clean # native sources/builds + venv + build cache
./scripts/update.sh --dry-run # preview: branch / drift / venv state, no changes
./scripts/update.sh --extras dev # test/dev/docs/viewer/viewer-gpu/basisopt/ase/dispersion/mace/all/none
./scripts/update.sh --venv .venv-bsd # explicit venv (for side-by-side per-branch venvs)
./scripts/update.sh --dev --rebuild-native-deps # combine flags
--release / --dev are named shortcuts; --branch NAME accepts
any branch or tag (--ref NAME is kept as the older spelling).
The branch-selection flags are mutually exclusive, pick one. The
default (release) is the right choice for the vast majority of
users, it tracks whatever the project author has tagged as the
current public release. --dev switches to main for previewing
in-flight features before they ship.
--recreate-venv is the targeted fix for a broken venv: stale
pip shebang after copying a venv across trees, hybrid
pyvenv.cfg after python3 -m venv ran against a different
interpreter, or just “import vibeqc segfaults and I don’t know
why”. The prior environment waits in a same-parent backup until its
replacement installs and verifies; any failure restores it. By default the
replacement preserves the old venv’s base interpreter. Use --python only
when you intentionally want to change it.
Replacement also requires this checkout’s lifecycle ownership marker. For an
installation created before markers existed, add --adopt-legacy; the script
then reads PEP 610 metadata with a trusted external Python and proceeds only
when that metadata points to this exact checkout. A foreign marker is never
adopted. Ordinary in-place updates do not claim ownership of shared venvs.
--clean is the nuclear option: combines --rebuild-native-deps
--recreate-venv, removes native source/build/install trees, and wipes Python build caches. It never removes the tracked basis library. Takes 15-40 min on a clean run but is guaranteed reproducible, use after a known-bad build state, before publishing benchmark numbers, or when bisecting a regression.
--dry-run previews everything without touching anything: the
target commit, build-stamp drift against the currently-checked-out
build_*.sh files, venv health. Safe to run during an in-flight
calculation.
An update requires an existing usable venv. If none is found, it exits before
fetching or rebuilding anything and directs you to install.sh; it never
reports a partial update as success. For environment-only lifecycle work, use
reinstall.sh or uninstall.sh.
If you keep two checkouts side-by-side (one for production runs,
one for dev), use update.sh (or --release) inside the release
tree and update.sh --dev inside the dev tree. The lighter-weight
zsh vibe-update
helper later in this page knows about both trees and refreshes them
in one shot.
-h / --help prints the same option list inline.
Build-pressure safety¶
On Linux, update.sh (and setup_native_deps.sh, and each
build_*.sh) self-re-execs under nice -n 19 ionice -c 3 and
sets a memory-safe default for CMAKE_BUILD_PARALLEL_LEVEL. The
first line of output on a fresh build will look like:
==> CMAKE_BUILD_PARALLEL_LEVEL=8 (nproc=32, mem=128721MB; cap@8 / 15GB-per-worker)
The cap is min(nproc, max(2, mem_mb // 15000), 8), which keeps
ninja from firing more cc1plus workers than the host has RAM
for. Each cc1plus on vibe-qc’s template-heavy translation units
(libint integrals, periodic_*.cpp, gradient.cpp) peaks at
8-10 GB resident; the formula budgets 15 GB/worker with a hard
ceiling of 8, above 8, ninja serializes on link/IO contention
and extra parallelism just thrashes the page cache. The
nice -n 19 ionice -c 3 prefix keeps the foreground shell
responsive during long builds.
This exists because unbounded ninja parallelism on planetx
(32 threads / 125 GB) triggered global-OOM and required a hard
reset twice on 2026-05-16. The helper script
(scripts/_safe_build_env.sh) is sourced by every entry-point
script that triggers heavy compilation, so the cap propagates
through the full chain.
Overrides (set in your env before invoking update.sh):
CMAKE_BUILD_PARALLEL_LEVEL=12 ./scripts/update.sh --dev
# → pin parallelism explicitly (skips the formula)
VIBEQC_BUILD_NICED=1 ./scripts/update.sh --dev
# → skip the nice/ionice re-exec (full CPU + IO priority)
VIBEQC_BUILD_ENV_QUIET=1 ./scripts/update.sh --dev
# → silence the cap-announcement banner
macOS: both side-effects skip cleanly (no /proc/meminfo,
no ionice). Set CMAKE_BUILD_PARALLEL_LEVEL manually if your
dev box is RAM-constrained.
See operations.md in the
vibe-queue tree for the full post-mortem of the planetx
incident and the recovery procedures for related failure modes
(zombie user-systemd, stale daemon-in-memory after pip install -e ., etc.).
Advanced diagnostic outline¶
The managed updater also holds cross-platform checkout and environment locks, checks ownership and build provenance, targets only drifted dependencies, verifies the installed extension, and rolls back a replaced environment on failure. The commands below are only a diagnostic outline for understanding an update; they are not a safe drop-in equivalent for normal maintenance:
# 1. Refuse if working tree is dirty (silently stashing user work
# is worse than failing loud)
git diff --quiet && git diff --cached --quiet || \
{ echo "uncommitted changes -- commit or stash"; exit 1; }
# 2. Fetch and check out the target ref (release branch by default)
git fetch origin --tags
git checkout release
git pull --ff-only origin release # only on branches; tags don't pull
# 3. Rebuild any native deps whose source changed
./scripts/setup_native_deps.sh
# 4. Refresh the Python package (rebuilds the C++ extension)
.venv/bin/pip install -e '.[test]'
# 5. Confirm the new banner
.venv/bin/python -c "import vibeqc; vibeqc.print_banner()"
Step 3 is the heavy one if anything changed. The managed build stamps and artifact checks skip current dependencies and rebuild only drifted or damaged ones, so a vanilla update usually takes seconds.
Switching between releases¶
Sometimes you want to drop to a previous release temporarily, to reproduce an old result, or to confirm a regression bisects to a specific tag.
./scripts/update.sh --ref v0.6.3
# ... run the calculation, compare ...
./scripts/update.sh --ref release # back to current
The on-disk state is fully consistent at each step. Outputs from your previous calculations survive, only the vibe-qc code changes.
Going to bleeding-edge main¶
./scripts/update.sh --ref main
main is where active development lands. Banner reads
dev X.Y.devN (main @ <sha>), the SHA pins the build for
reproducibility. Use this when you need a feature that’s
already merged but not yet released; don’t use this for
publication-quality numbers (subtle bugs may be undetected until
the next regression-test pass).
To go back to the published release:
./scripts/update.sh --ref release
Common issues¶
Vendored library version bumps¶
Each per-dep build script’s idempotency check looks at “does
third_party/<dep>/install/lib/cmake/X/XConfig.cmake exist?”, and
skips if yes. If a release bumps libxc 7.0.0 → 7.0.1 (or libint /
spglib / FFTW / libecpint), the user’s stale install silently sticks
around unless the orchestrator reconciles provenance first.
Diagnostic + auto-fix: update.sh checks the third_party/.build-stamp
file (written on every successful native-deps build) against the
currently-checked-out scripts/build_*.sh files. If the version pinned
in build_libxc.sh differs from the stamp, or the build-relevant
content of build_libxc.sh changed (source pin, compiler selection,
configure flags, or CMake arguments), update.sh rebuilds the drifted dep(s)
automatically and targeted, via scripts/update_native_deps.sh: only
the changed dep’s pinned source/build/install trees are removed and rebuilt,
not all of them. setup_native_deps.sh performs the same reconciliation
before its idempotent build calls, so it cannot relabel an old binary with a
new recipe hash. Legacy install trees without a stamp stay explicitly
uncertified until a full rebuild. Pass
--rebuild-native-deps to force a full rebuild of every dep instead.
./scripts/doctor.sh runs the same drift check on demand and only
reports (never rebuilds). You can also invoke the targeted rebuild
directly: ./scripts/update_native_deps.sh (--dep NAME / --all /
--dry-run).
The recipe fingerprint is conservative: every executable builder line counts unless it is inside the narrowly marked lifecycle-only locking block. Full-line comments, blank lines, and changes confined to that lock block do not change the native artifact, so they do not trigger a costly libint/libxc rebuild. Malformed or unmatched lifecycle markers fail closed and report recipe drift. Build stamps written before this distinction have three fields; the next normal update accepts a matching legacy fingerprint and upgrades it to the four-field format without rebuilding the dependency.
One libint builder from immediately before the unified lifecycle move placed the same lock acquisition later in the script, so its legacy fingerprint does not equal either current fingerprint even though its native recipe is unchanged. That reviewed case is admitted through an exact compatibility triple: builder path, full historical SHA-256, and full current build-relevant SHA-256 must all match. This is not a prefix or fuzzy migration. Unknown or partial fingerprints, malformed lifecycle markers, and any change to the source pin, configure inputs, CMake arguments, or other unmarked command still report drift and rebuild libint.
The same targeted auto-rebuild also fires when a dep’s install/ tree is
present but its actual shared library was deleted, the drift check
looks for the real libint2.so / libxc.so / libsymspg.so / … on disk,
not just the surviving CMake config file. So a normal update.sh (and
vq admin update) now repairs a wiped library on its own; you no longer
need the --rebuild-native-deps force flag for that case. See Wiped
shared library below.
Diagnostic, by hand: the banner’s
linked: libint X.Y.Z · libxc A.B.C ... line should match the
versions pinned in the various scripts/build_<dep>.sh. If it
doesn’t, the stale-install case has bitten.
Fix:
./scripts/update.sh --rebuild-native-deps
That removes every pinned native source/build/install tree (and the stamp) before re-running the native-deps orchestrator, forcing each dependency to fetch and rebuild its currently pinned version. An existing vendored OpenBLAS selection is retained automatically, even if its install tree is damaged, instead of silently falling back to system BLAS.
Rebuild one native dependency¶
Current build stamps include the pinned version and recipe fingerprint. A changed libint build recipe, missing artifact, or stale source is detected instead of trusting an existing directory. To force only libint through the locked native updater:
./scripts/update_native_deps.sh --dep libint
./scripts/update.sh
Use ./scripts/update.sh --rebuild-native-deps when every vendored dependency
needs a clean rebuild. Do not remove native trees manually while another
lifecycle or build command may be active.
You normally don’t have to do this by hand. update.sh
hashes each build_<dep>.sh’s build-relevant lines into the
native-dep stamp, so a changed cmake flag registers as drift
and triggers a rebuild of just that dep. The manual
rm -rf recipe above is for installs that pre-date the stamp
(update.sh says “Legacy install (no build stamp)”), or when
you’ve bypassed update.sh entirely.
Concretely, the 2026-08-06 bump of the 2nd-derivative
max_am tier from 3 to 4 (analytic Hessians on g-function
basis sets) is a flag change of exactly this kind: running
./scripts/update.sh after pulling it rebuilds libint
automatically. Budget 1.5-2 h for that rebuild, since the
generated-kernel count for the deriv-2 stratum roughly
triples.
During that rebuild the terminal shows one heartbeat line per
minute rather than the raw codegen/compile stream; the full
output is in third_party/libint/build.log (tail -f it, or
set VIBEQC_BUILD_VERBOSE=1 to stream). A failed build
replays the last 200 log lines automatically.
Choosing libint’s max_am (--libint-max-am)¶
That rebuild is optional. The spec is three numbers: the max angular momentum at derivative orders 0, 1 and 2:
./scripts/update.sh --libint-max-am 5_4_3
Spec |
Build |
Energies |
Gradients |
Hessians |
|---|---|---|---|---|
|
~30 min |
h (L=5) |
g (L=4) |
f (L=3) |
|
~1.5-2 h |
h |
g |
g |
|
longer |
i (L=6) |
h |
g |
|
longest |
i |
i |
i |
Each part may be 1..6. Above a stratum’s limit libint
throws rather than returning a wrong number, so the spec
only decides which basis sets each derivative order
accepts: no computed value changes. Most users want
5_4_3 or 5_4_4; the 6_* specs are for cc-pV6Z-class
work.
6 is the ceiling, and it is vibe-qc’s rather than
libint’s. The periodic AO-pair Fourier transform
transforms Cartesian to spherical through the generated
table in cpp/include/vibeqc/cart_to_sph_data.hpp
(kMaxL = 6) and throws on any shell above it, so a
higher libint would emit integrals the periodic stack
cannot consume. Raising it means bumping MAX_L in
scripts/codegen_cart_to_sph.py, regenerating that
header, re-validating the fit precision (it gets
rank-revealing past L = 6), and moving
VIBEQC_LIBINT_MAX_AM_CEILING in lockstep.
The setting is not stored in build_libint.sh, so the build
stamp folds it into libint’s recipe hash on purpose, and
build_libint.sh refuses to short-circuit on an install
tree built with a different spec. Both directions therefore
work as a plain update, with no force flags:
./scripts/update.sh --libint-max-am 5_4_4
doctor.sh reports the installed spec, and drift shows up
as BUILD-SETTING DRIFT (max_am=5_4_4 → max_am=5_4_3)
rather than as a phantom change to build_libint.sh. The
same flag is on install.sh and update_native_deps.sh;
VIBEQC_LIBINT_MAX_AM is the underlying environment
variable if you need to set it once for a shell.
An install tree built before this option landed carries no
marker. It is assumed to match the default, which is correct
for anything built from a 2026-08-06-or-later checkout; if
you need certainty, rm -rf third_party/libint/install and
let it rebuild.
OSError: cannot load library 'libxc.so.7'¶
A vendored dep’s install/ tree was wiped (or never built). Re-run:
./scripts/update.sh --rebuild-native-deps
CMake complains about a stale build cache after update¶
Rare but possible if scikit-build-core’s build cache went stale:
./scripts/update.sh --clean
--clean removes the native source/build/install trees and Python build caches
under the shared lifecycle and native-build locks, then recreates and verifies
the environment with rollback.
Tests fail after update¶
.venv/bin/python -m pytest tests/
If a few tests fail right after a major update, that’s a real signal worth investigating, please open an issue with the banner output and the pytest tail. Don’t ignore the failures and run “real” calculations on the build until the regression is understood.
Shell aliases for daily workflow¶
If you activate the venv and run updates many times a day, a few zsh aliases save typing. Adapt the paths to your install.
# --- vibeqc helpers (~/.zshrc) ---
alias vibe-up='cd /path/to/vibeqc && source .venv/bin/activate'
alias vibe-up-experimental='cd /path/to/vibeqc-experimental && source .venv/bin/activate'
alias vibe-down='deactivate 2>/dev/null; cd ~'
vibe-up jumps to the repo and activates the venv; vibe-down
deactivates and returns home. vibe-up-experimental activates a
separate venv tracking an experimental feature branch (e.g.
feature/v0.7-pyscf-pbc-parity while the periodic-SCF bug-fix work
is in flight), see the next section for what an experimental tree
is for.
Maintaining dev + release (+ experimental) side-by-side¶
If you keep two checkouts, one tracking main for development, one
tracking release for production runs, the function below pulls
both, reinstalls the Python package, and verifies each tree by
importing the freshly-installed package.
The function also knows about an optional experimental tree:
a third checkout tracking a feature branch you want to keep current
without disrupting your day-to-day dev / release setup. Typical
uses: previewing in-flight bug-fix branches (e.g. the
feature/v0.7-pyscf-pbc-parity periodic-SCF work), evaluating a
breaking-change branch before it merges to main, or running
side-by-side comparisons against the dev tree on the same data.
The experimental tree is opt-in, if /path/to/vibeqc-experimental
doesn’t exist, vibe-update silently skips it.
vibe-update() {
local target="${1:-all}"
local errors=0
local repo # NOT 'path' -- see footgun note below
case "$target" in
dev|release|experimental|all|both) ;;
*) echo "vibe-update: unknown target '$target' -- try dev / release / experimental / all"
return 2 ;;
esac
local GIT
for candidate in /opt/homebrew/bin/git /usr/local/bin/git /usr/bin/git; do
[[ -x "$candidate" ]] && { GIT="$candidate"; break; }
done
[[ -n "$GIT" ]] || { echo "vibe-update: no git found"; return 1; }
local PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
export PATH # zsh's `local PATH` does not auto-export to subprocesses
for tree in dev release experimental; do
# Selection rules:
# target=all → every configured tree
# target=both → dev + release (back-compat alias for the
# pre-experimental default)
# target=<tree-name> → just that one
case "$target" in
all|"$tree") ;;
both) [[ "$tree" == experimental ]] && continue ;;
*) continue ;;
esac
case "$tree" in
dev) repo=/path/to/vibeqc-dev ;;
release) repo=/path/to/vibeqc-release ;;
experimental) repo=/path/to/vibeqc-experimental ;;
esac
# Experimental is opt-in: if the directory doesn't exist and the
# user didn't ask for it explicitly, skip silently rather than
# erroring out -- most users won't have an experimental checkout.
if [[ ! -d "$repo" ]]; then
[[ "$tree" == experimental && "$target" != experimental ]] && continue
echo; echo "=== $tree ($repo) ==="
echo " ✗ no checkout at $repo -- skipping"; ((errors++))
continue
fi
echo; echo "=== Updating $tree ($repo) ==="
[[ -d "$repo/.venv" ]] || { echo " ✗ no .venv at $repo -- skipping"; ((errors++)); continue; }
pushd "$repo" >/dev/null || { ((errors++)); continue; }
"$GIT" fetch origin && "$GIT" pull --ff-only \
|| { echo " ✗ pull failed"; ((errors++)); popd >/dev/null; continue; }
# Keep pip itself current -- without this, every install prints a
# "[notice] A new release of pip is available" pestering the user
# with the `python -m pip install --upgrade pip` command they could
# run to silence it. Doing it here suppresses the notice and keeps
# the venv aligned with whatever pip version `pip install -e` wants.
"$repo/.venv/bin/python" -m pip install --quiet --upgrade pip \
|| { echo " ✗ pip self-upgrade failed"; ((errors++)); popd >/dev/null; continue; }
"$repo/.venv/bin/pip" install -e "$repo" --no-deps --quiet \
|| { echo " ✗ pip install failed"; ((errors++)); popd >/dev/null; continue; }
"$repo/.venv/bin/python" -c \
"import vibeqc; print(f' ✓ vibe-qc {vibeqc.__version__} from {vibeqc.__file__}')"
popd >/dev/null
done
echo
(( errors > 0 )) && { echo "vibe-update: $errors tree(s) failed"; return 1; }
echo "vibe-update: done"
}
Usage:
vibe-update, every configured tree (dev + release, plus experimental if you’ve checked one out).vibe-update dev/vibe-update release/vibe-update experimentalthat one tree only.
vibe-update both, dev + release only, skipping experimental. Back-compat alias from the pre-experimental version of this function.
Setting up the experimental tree¶
git clone ssh://git@gitlab.peintinger.com:26/mpei/vibeqc.git /path/to/vibeqc-experimental
cd /path/to/vibeqc-experimental
git checkout feature/v0.7-pyscf-pbc-parity # or whichever branch
./scripts/setup_native_deps.sh # if it differs from your dev tree
python3 -m venv .venv
.venv/bin/pip install -e '.[test]'
After that, vibe-update (no args) will keep all three trees current
on every invocation; vibe-up-experimental activates its venv when
you want to actually use it.
This is a lighter-weight cousin of ./scripts/update.sh, it skips
the native-deps rebuild and the banner print, so it’s only safe when
no vendored library version bumped between pulls. For release-day
updates or after a known native-dep bump, prefer update.sh.
Two zsh footguns baked into the function above¶
Both bit during the original write-up; documenting them here so the next reader doesn’t rediscover them the hard way.
Don’t name the variable path. zsh ties the lowercase path
array to the uppercase PATH scalar via typeset -T. Assigning
path=/some/dir silently overwrites $PATH, after which git can no
longer find ssh and the pull fails with
error: cannot run ssh: No such file or directory. The function uses
repo for this reason.
local PATH=... needs an explicit export PATH. Inside a zsh
function, local declares but does not re-export the variable to
forked subprocesses. Without export, the localized PATH stays
inside the shell and git’s child processes (in particular ssh)
don’t see it. Same symptom as the path/PATH tie above, different
cause.
Bash users can adapt the same function, local semantics differ
slightly, but the path/PATH collision is a zsh-only issue.
Long-running calculations across an update¶
If you have a long calculation in flight and want to update vibe-qc
without disturbing it, don’t: in-flight python processes hold
references to the loaded _vibeqc_core.so, so an update mid-flight
doesn’t break the running job, but starting a new calculation in
the same shell will pick up the new code immediately. Cleanest is:
Wait for the running job to finish (or pause it via
Ctrl-Zandbgif you must).Update.
Start subsequent jobs.
For background-run patterns (tmux + tee), see the
running guide.
Read-only health check¶
If you only want to know “is my install OK?” without triggering anything, run:
./scripts/doctor.sh
It reports on:
working-tree state (clean / dirty, current branch + SHA);
which
third_party/<dep>/install/trees exist;build-stamp drift against the current
build_*.shfiles (the same checkupdate.shruns automatically);venv health, broken python, stale pip shebang, hybrid
pyvenv.cfg/ runtime-python mismatch;the banner (which library versions are actually linked).
No builds. No installs. No git operations. Exit code is 0 if every check passes, 1 if anything emitted a warning, so it’s scriptable:
if ./scripts/doctor.sh >/dev/null; then
echo "install OK"
fi
Where to go next¶
Quickstart, confirm the new install with a smoke-test calculation.
Release process, the upstream side of the release flow that drives what you’re updating to.
Changelog, what shipped in the release you just installed.