Skip to content

Load vectors from the community index (replaces BUNDLED_VECTORS) - #9

Merged
JenD86 merged 3 commits into
mainfrom
feat/load-community-index
Oct 28, 2026
Merged

JenD86 merged 3 commits into
mainfrom
feat/load-community-index

Conversation

@JenD86

@JenD86 JenD86 commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Point 1 of community contributions: the library now reads community/vectors.yaml (the repo's copy) instead of the hard-coded BUNDLED_VECTORS.

How a model's vectors are assembled at startup

  1. Your own vectors from vectors/<model>/registry.yaml. These win on a name clash.
  2. Index entries for the model. verified entries load by default. Unverified community entries load only with SteeringAgent(..., include_community=True); they're capped at ±8 unless a maintainer has calibrated them, and marked [Unverified community vector] in the model's prompt.
  3. Auto-extraction, unchanged, if the model ends up with no vectors at all.

Safety of index files

  • Each file is fetched at its pinned HF revision, then checked against its sha256 and the model's hidden_size/layer count before it's cached in vectors/<model>/.
  • A cached file that already matches is reused with no network call.
  • A local file that doesn't match is reported and never overwritten.

registry.yaml stays yours: index entries live in memory only, and save() never writes them. This avoids rewriting a hand-commented, tracked file on every run. A removed or demoted index entry simply stops loading.

Migration

  • The hand-written assistant_axis entry is removed from the Qwen registry.yaml, so the index provides it, using the same committed file (sha matches, no download).
  • Two warnings from the old hand-written entry ("Strong effect", "can get weird at high magnitude") move into the index entry.
  • setup.sh fetches through the index instead of hard-coding the Butanium download.

Also: load_index/fetch_vector move into modulo/community.py, shared by the library, CI and gpu_verify.py. The docs (AGENTS, CONTRIBUTING, README) are updated.

Testing

  • 66 CPU tests pass. The 8 new ones cover:
    • verified loading by default, community opt-in with ±8 and the unverified marker
    • reusing a cached file
    • not overwriting a mismatched local file
    • not caching a bad download
    • the model-dimension check
    • your own vectors winning, and index entries never being saved
    • an invalid index being ignored
    • the real repo index serving Qwen's committed file
  • GPU (RTX 4090 RunPod, fresh clone): setup.sh fetched assistant_axis through the index and loaded the model; pytest 69 passed (CPU + GPU); a 3-turn run.sh --auto session loaded the vector from the index, steered at -8 and decayed as expected.

🤖 Generated with Claude Code

JenD86 and others added 3 commits October 27, 2026 18:30
At startup the registry now combines your own vectors (registry.yaml) with the
model's entries from community/vectors.yaml: verified by default, unverified
community ones with include_community=True (±8 unless calibrated, and marked as
unverified in the prompt). Files are fetched at their pinned revision, checked
against sha256 and the model's dimensions, and cached in vectors/<model>/; a
cached file that doesn't match is reported, never overwritten.

Index entries live in memory only, so registry.yaml stays yours: own vectors
win on a name clash and save() never writes index entries. Removes
BUNDLED_VECTORS; the Qwen assistant_axis now comes from the index (same file).
setup.sh fetches through the index too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GPU verification on Qwen2.5-7B showed odd capitalization and muddled answers
at +15 that the repetition detector misses; -15 stays coherent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JenD86
JenD86 merged commit 6d3ca1d into main Oct 28, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant