Updating vibe-qc¶
For an existing checkout you’ve already used. If this is your first install, see installation instead.
Variants¶
./scripts/update.sh --release # latest tagged release (default; flag is for symmetry)
./scripts/update.sh --dev # bleeding-edge main (X.Y.devN banner)
./scripts/update.sh --branch main # verbose form of --dev
./scripts/update.sh --branch basissetdev # any other branch — e.g. the basissetdev paper-writing branch
./scripts/update.sh --branch v0.9.0 # pin to a specific tag
./scripts/update.sh --ref v0.9.0 # older spelling of --branch, kept for back-compat
./scripts/update.sh --rebuild-native-deps # see "Vendored library version bumps" below
./scripts/update.sh --recreate-venv # rm -rf .venv + rebuild it (fix a broken venv)
./scripts/update.sh --clean # nuke everything: native deps + venv + build cache
./scripts/update.sh --dry-run # preview: branch / drift / venv state, no changes
./scripts/update.sh --extras dev # pip extras: test / dev / docs / viewer / all / none
./scripts/update.sh --venv .venv-bsd # explicit venv (for side-by-side per-branch venvs)
./scripts/update.sh --dev --rebuild-native-deps # combine flags
--release / --dev are named shortcuts; --branch NAME accepts
any branch or tag (--ref NAME is kept as the older spelling).
The branch-selection flags are mutually exclusive — pick one. The
default (release) is the right choice for the vast majority of
users — it tracks whatever the project author has tagged as the
current public release. --dev switches to main for previewing
in-flight features before they ship.
--recreate-venv is the targeted fix for a broken venv: stale
pip shebang after copying a venv across trees, hybrid
pyvenv.cfg after python3 -m venv ran against a different
interpreter, or just “import vibeqc segfaults and I don’t know
why”. It wipes .venv/ and recreates it before reinstalling.
--clean is the nuclear option: combines --rebuild-native-deps
--recreate-venv+ wipesbuild/and the basis-library cache. Takes 15–40 min on a clean run but is guaranteed reproducible — use after a known-bad build state, before publishing benchmark numbers, or when bisecting a regression.
--dry-run previews everything without touching anything: the
target commit, build-stamp drift against the currently-checked-out
build_*.sh files, venv health. Safe to run during an in-flight
calculation.
If you keep two checkouts side-by-side (one for production runs,
one for dev), use update.sh (or --release) inside the release
tree and update.sh --dev inside the dev tree. The lighter-weight
zsh vibe-update
helper later in this page knows about both trees and refreshes them
in one shot.
-h / --help prints the same option list inline.
Build-pressure safety¶
On Linux, update.sh (and setup_native_deps.sh, and each
build_*.sh) self-re-execs under nice -n 19 ionice -c 3 and
sets a memory-safe default for CMAKE_BUILD_PARALLEL_LEVEL. The
first line of output on a fresh build will look like:
==> CMAKE_BUILD_PARALLEL_LEVEL=8 (nproc=32, mem=128721MB; cap@8 / 15GB-per-worker)
The cap is min(nproc, max(2, mem_mb // 15000), 8), which keeps
ninja from firing more cc1plus workers than the host has RAM
for. Each cc1plus on vibe-qc’s template-heavy translation units
(libint integrals, periodic_*.cpp, gradient.cpp) peaks at
8–10 GB resident; the formula budgets 15 GB/worker with a hard
ceiling of 8 — above 8, ninja serializes on link/IO contention
and extra parallelism just thrashes the page cache. The
nice -n 19 ionice -c 3 prefix keeps the foreground shell
responsive during long builds.
This exists because unbounded ninja parallelism on planetx
(32 threads / 125 GB) triggered global-OOM and required a hard
reset twice on 2026-05-16. The helper script
(scripts/_safe_build_env.sh) is sourced by every entry-point
script that triggers heavy compilation, so the cap propagates
through the full chain.
Overrides (set in your env before invoking update.sh):
CMAKE_BUILD_PARALLEL_LEVEL=12 ./scripts/update.sh --dev
# → pin parallelism explicitly (skips the formula)
VIBEQC_BUILD_NICED=1 ./scripts/update.sh --dev
# → skip the nice/ionice re-exec (full CPU + IO priority)
VIBEQC_BUILD_ENV_QUIET=1 ./scripts/update.sh --dev
# → silence the cap-announcement banner
macOS: both side-effects skip cleanly (no /proc/meminfo,
no ionice). Set CMAKE_BUILD_PARALLEL_LEVEL manually if your
dev box is RAM-constrained.
See operations.md in the
vibe-queue tree for the full post-mortem of the planetx
incident and the recovery procedures for related failure modes
(zombie user-systemd, stale daemon-in-memory after pip install -e ., etc.).
Manual equivalent¶
If you’d rather invoke the steps by hand (e.g. when debugging an
update that didn’t take), here’s what update.sh does:
# 1. Refuse if working tree is dirty (silently stashing user work
# is worse than failing loud)
git diff --quiet && git diff --cached --quiet || \
{ echo "uncommitted changes — commit or stash"; exit 1; }
# 2. Fetch and check out the target ref (release branch by default)
git fetch origin --tags
git checkout release
git pull --ff-only origin release # only on branches; tags don't pull
# 3. Rebuild any native deps whose source changed
./scripts/setup_native_deps.sh
# 4. Refresh the Python package (rebuilds the C++ extension)
.venv/bin/pip install -e '.[test]'
# 5. Confirm the new banner
.venv/bin/python -c "import vibeqc; vibeqc.print_banner()"
Step 3 is the heavy one if anything changed. Each build_*.sh skips
work when its install/ tree already exists, so a vanilla pull
usually takes seconds.
Switching between releases¶
Sometimes you want to drop to a previous release temporarily — to reproduce an old result, or to confirm a regression bisects to a specific tag.
./scripts/update.sh --ref v0.6.3
# ... run the calculation, compare ...
./scripts/update.sh --ref release # back to current
The on-disk state is fully consistent at each step. Outputs from your previous calculations survive — only the vibe-qc code changes.
Going to bleeding-edge main¶
./scripts/update.sh --ref main
main is where active development lands. Banner reads
dev X.Y.devN (main @ <sha>) — the SHA pins the build for
reproducibility. Use this when you need a feature that’s
already merged but not yet released; don’t use this for
publication-quality numbers (subtle bugs may be undetected until
the next regression-test pass).
To go back to the published release:
./scripts/update.sh --ref release
Common issues¶
Vendored library version bumps¶
setup_native_deps.sh’s per-dep idempotency check looks at “does
third_party/<dep>/install/lib/cmake/X/XConfig.cmake exist?” — and
skips if yes. If a release bumps libxc 7.0.0 → 7.0.1 (or libint /
spglib / FFTW / libecpint), the user’s stale install silently sticks
around and pip install links against it.
Diagnostic + auto-fix: update.sh checks the third_party/.build-stamp
file (written on every successful native-deps build) against the
currently-checked-out scripts/build_*.sh files. If the version pinned
in build_libxc.sh differs from the stamp — or the build-relevant
content of build_libxc.sh changed (configure flags / cmake args;
full-line comments and blank lines are ignored, so a comment-only edit
does not count) — update.sh rebuilds the drifted dep(s)
automatically and targeted, via scripts/update_native_deps.sh: only
the changed dep is wiped and rebuilt, not all of them. Pass
--rebuild-native-deps to force a full rebuild of every dep instead.
./scripts/doctor.sh runs the same drift check on demand and only
reports (never rebuilds). You can also invoke the targeted rebuild
directly: ./scripts/update_native_deps.sh (--dep NAME / --all /
--dry-run).
Diagnostic — by hand: the banner’s
linked: libint X.Y.Z · libxc A.B.C ... line should match the
versions pinned in the various scripts/build_<dep>.sh. If it
doesn’t, the stale-install case has bitten.
Fix:
./scripts/update.sh --rebuild-native-deps
That nukes every third_party/<dep>/install/ tree (and the stamp)
before re-running the native-deps orchestrator, forcing each dep
to rebuild from source against its currently-pinned version.
libint stale-install footgun (build flags changed)¶
scripts/build_libint.sh early-exits at the existence-check
on line ~42 if third_party/libint/install/ exists, even
when the libint build flags have changed since the last
build. This is a silent stale-install footgun: a
pre-existing libint install may lack ERI3 / ERI2 derivative
support (vibe-qc’s DF code segfaults on compute_2c_eri /
compute_3c_eri with auxiliary bases containing l ≥ 1
shells if the build was missing those flags).
Symptom: SIGSEGV in DF SCF or DF gradient code paths that worked on a different machine. No traceback (below the Python boundary).
Fix:
rm -rf third_party/libint/{install,build}
bash scripts/build_libint.sh
.venv/bin/pip install -e . --no-build-isolation
If that resolves the segfault, the prior libint build was
missing the ERI2 / ERI3 derivative flags. The flags are
fixed in current build_libint.sh; the issue only bites
on installs that pre-date the flag pin.
OSError: cannot load library 'libxc.so.7'¶
A vendored dep’s install/ tree was wiped (or never built). Re-run:
./scripts/update.sh --rebuild-native-deps
CMake complains about a stale build cache after update¶
Rare but possible if scikit-build-core’s build cache went stale:
rm -rf build/ .scikit-build*
.venv/bin/pip install -e '.[test]' --force-reinstall --no-deps
Tests fail after update¶
.venv/bin/python -m pytest tests/
If a few tests fail right after a major update, that’s a real signal worth investigating — please open an issue with the banner output and the pytest tail. Don’t ignore the failures and run “real” calculations on the build until the regression is understood.
Shell aliases for daily workflow¶
If you activate the venv and run updates many times a day, a few zsh aliases save typing. Adapt the paths to your install.
# --- vibeqc helpers (~/.zshrc) ---
alias vibe-up='cd /path/to/vibeqc && source .venv/bin/activate'
alias vibe-up-experimental='cd /path/to/vibeqc-experimental && source .venv/bin/activate'
alias vibe-down='deactivate 2>/dev/null; cd ~'
vibe-up jumps to the repo and activates the venv; vibe-down
deactivates and returns home. vibe-up-experimental activates a
separate venv tracking an experimental feature branch (e.g.
feature/v0.7-pyscf-pbc-parity while the periodic-SCF bug-fix work
is in flight) — see the next section for what an experimental tree
is for.
Maintaining dev + release (+ experimental) side-by-side¶
If you keep two checkouts — one tracking main for development, one
tracking release for production runs — the function below pulls
both, reinstalls the Python package, and verifies each tree by
importing the freshly-installed package.
The function also knows about an optional experimental tree:
a third checkout tracking a feature branch you want to keep current
without disrupting your day-to-day dev / release setup. Typical
uses: previewing in-flight bug-fix branches (e.g. the
feature/v0.7-pyscf-pbc-parity periodic-SCF work), evaluating a
breaking-change branch before it merges to main, or running
side-by-side comparisons against the dev tree on the same data.
The experimental tree is opt-in — if /path/to/vibeqc-experimental
doesn’t exist, vibe-update silently skips it.
vibe-update() {
local target="${1:-all}"
local errors=0
local repo # NOT 'path' — see footgun note below
case "$target" in
dev|release|experimental|all|both) ;;
*) echo "vibe-update: unknown target '$target' — try dev / release / experimental / all"
return 2 ;;
esac
local GIT
for candidate in /opt/homebrew/bin/git /usr/local/bin/git /usr/bin/git; do
[[ -x "$candidate" ]] && { GIT="$candidate"; break; }
done
[[ -n "$GIT" ]] || { echo "vibe-update: no git found"; return 1; }
local PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
export PATH # zsh's `local PATH` does not auto-export to subprocesses
for tree in dev release experimental; do
# Selection rules:
# target=all → every configured tree
# target=both → dev + release (back-compat alias for the
# pre-experimental default)
# target=<tree-name> → just that one
case "$target" in
all|"$tree") ;;
both) [[ "$tree" == experimental ]] && continue ;;
*) continue ;;
esac
case "$tree" in
dev) repo=/path/to/vibeqc-dev ;;
release) repo=/path/to/vibeqc-release ;;
experimental) repo=/path/to/vibeqc-experimental ;;
esac
# Experimental is opt-in: if the directory doesn't exist and the
# user didn't ask for it explicitly, skip silently rather than
# erroring out — most users won't have an experimental checkout.
if [[ ! -d "$repo" ]]; then
[[ "$tree" == experimental && "$target" != experimental ]] && continue
echo; echo "=== $tree ($repo) ==="
echo " ✗ no checkout at $repo — skipping"; ((errors++))
continue
fi
echo; echo "=== Updating $tree ($repo) ==="
[[ -d "$repo/.venv" ]] || { echo " ✗ no .venv at $repo — skipping"; ((errors++)); continue; }
pushd "$repo" >/dev/null || { ((errors++)); continue; }
"$GIT" fetch origin && "$GIT" pull --ff-only \
|| { echo " ✗ pull failed"; ((errors++)); popd >/dev/null; continue; }
# Keep pip itself current — without this, every install prints a
# "[notice] A new release of pip is available" pestering the user
# with the `python -m pip install --upgrade pip` command they could
# run to silence it. Doing it here suppresses the notice and keeps
# the venv aligned with whatever pip version `pip install -e` wants.
"$repo/.venv/bin/python" -m pip install --quiet --upgrade pip \
|| { echo " ✗ pip self-upgrade failed"; ((errors++)); popd >/dev/null; continue; }
"$repo/.venv/bin/pip" install -e "$repo" --no-deps --quiet \
|| { echo " ✗ pip install failed"; ((errors++)); popd >/dev/null; continue; }
"$repo/.venv/bin/python" -c \
"import vibeqc; print(f' ✓ vibe-qc {vibeqc.__version__} from {vibeqc.__file__}')"
popd >/dev/null
done
echo
(( errors > 0 )) && { echo "vibe-update: $errors tree(s) failed"; return 1; }
echo "vibe-update: done"
}
Usage:
vibe-update— every configured tree (dev + release, plus experimental if you’ve checked one out).vibe-update dev/vibe-update release/vibe-update experimental— that one tree only.vibe-update both— dev + release only, skipping experimental. Back-compat alias from the pre-experimental version of this function.
Setting up the experimental tree¶
git clone https://gitlab.peintinger.com/mpei/vibeqc.git /path/to/vibeqc-experimental
cd /path/to/vibeqc-experimental
git checkout feature/v0.7-pyscf-pbc-parity # or whichever branch
./scripts/setup_native_deps.sh # if it differs from your dev tree
python3 -m venv .venv
.venv/bin/pip install -e '.[test]'
After that, vibe-update (no args) will keep all three trees current
on every invocation; vibe-up-experimental activates its venv when
you want to actually use it.
This is a lighter-weight cousin of ./scripts/update.sh — it skips
the native-deps rebuild and the banner print, so it’s only safe when
no vendored library version bumped between pulls. For release-day
updates or after a known native-dep bump, prefer update.sh.
Two zsh footguns baked into the function above¶
Both bit during the original write-up; documenting them here so the next reader doesn’t rediscover them the hard way.
Don’t name the variable path. zsh ties the lowercase path
array to the uppercase PATH scalar via typeset -T. Assigning
path=/some/dir silently overwrites $PATH, after which git can no
longer find ssh and the pull fails with
error: cannot run ssh: No such file or directory. The function uses
repo for this reason.
local PATH=... needs an explicit export PATH. Inside a zsh
function, local declares but does not re-export the variable to
forked subprocesses. Without export, the localized PATH stays
inside the shell and git’s child processes (in particular ssh)
don’t see it. Same symptom as the path/PATH tie above, different
cause.
Bash users can adapt the same function — local semantics differ
slightly, but the path/PATH collision is a zsh-only issue.
Long-running calculations across an update¶
If you have a long calculation in flight and want to update vibe-qc
without disturbing it, don’t: in-flight python processes hold
references to the loaded _vibeqc_core.so, so an update mid-flight
doesn’t break the running job, but starting a new calculation in
the same shell will pick up the new code immediately. Cleanest is:
Wait for the running job to finish (or pause it via
Ctrl-Zandbgif you must).Update.
Start subsequent jobs.
For background-run patterns (tmux + tee), see the
running guide.
Read-only health check¶
If you only want to know “is my install OK?” without triggering anything, run:
./scripts/doctor.sh
It reports on:
working-tree state (clean / dirty, current branch + SHA);
which
third_party/<dep>/install/trees exist;build-stamp drift against the current
build_*.shfiles (the same checkupdate.shruns automatically);venv health — broken python, stale pip shebang, hybrid
pyvenv.cfg/ runtime-python mismatch;the banner (which library versions are actually linked).
No builds. No installs. No git operations. Exit code is 0 if every check passes, 1 if anything emitted a warning — so it’s scriptable:
if ./scripts/doctor.sh >/dev/null; then
echo "install OK"
fi
Where to go next¶
Quickstart — confirm the new install with a smoke-test calculation.
Release process — the upstream side of the release flow that drives what you’re updating to.
Changelog — what shipped in the release you just installed.