Repository navigation
maintenance: keep Python/JAX caches off $HOME on HPC (autolens_inference#13) - #51
Merged
Merged
Conversation
…nce#13)
On RAL every node's /home (and /tmp) sits on a small root disk; the RAL admin
found nodes broken by ~/.cache filling it (2026-09-25). A hot-fix lives in the RAL venv's
bin/activate but is lost on a venv rebuild; this makes it permanent in tracked activate.sh.
The HPC branch now exports XDG_CACHE_HOME, PIP_CACHE_DIR, MPLCONFIGDIR, NUMBA_CACHE_DIR,
CUDA_CACHE_PATH, TRITON_CACHE_DIR, JAX_COMPILATION_CACHE_DIR and ASTROPY_CACHE_DIR under
${PYAUTO_HPC_CACHE:-$(dirname "$PYAUTO_HPC_BASE")/.cache}, filling only unset variables
(JAX via ${VAR-...} so an explicit empty value still disables the cache).
Block is inside the PYAUTO_HPC_BASE branch only; laptop .venv users are unaffected.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Keep Python/JAX caches off
$HOME(and/tmp) on HPC: the HPC branch ofactivate.shnow exports XDG/pip/matplotlib/numba/CUDA/Triton/JAX/astropy caches under${PYAUTO_HPC_CACHE:-$(dirname "$PYAUTO_HPC_BASE")/.cache}(=/mnt/ral/jnightin/.cacheon RAL), filling only unset variables. Requested by the RAL admin after node root disks filled with~/.cache. Part of PyAutoLabs/autolens_inference#13; phase 2 (autolens_profiling, euclid pipeline) is planned/blocked.Scripts Changed
activate.shHeart RED override (development only)
hpc-cache-off-home— push the 4 branches and open pending-release PRs; merge still needs a separate/prmwith green checks; no release.pyauto-heart readiness):release validation FAILED (stage integrate)workspace validation not passing (4 failed, cloud#35579888156: autolens notebooks/cluster/modeling.ipynb, autolens notebooks/weak/a2744.ipynb, autolens scripts/cluster/modeling.py, +1 more)manifest drift: hub organism blurb (organs present) — 7 mismatch(es) vs PyAutoMind/repos.yamlactivate.sh(default HPC path, preset/empty JAX cache preserved, SLURM errexit guard quiet, laptop.venvpath untouched),bash -non every edited script; autolens_inferencecheck_submits.py --check12/12,build_readme.py --check, ruff check/format,pytest scripts/misc/test57 passed; autogalaxy_assistant--check-provenance0 errors,--lint-idiomsclean,--check-citations0 missing; in-session diff review. No smoke-coveredscripts/Python changed.🤖 Generated with Claude Code