From 3ee89275e78129cdf362569cd8ff6bd1bd161150 Mon Sep 17 00:00:00 2001 From: Jen Date: Tue, 27 Oct 2026 18:30:15 +0800 Subject: [PATCH 1/3] Load vectors from the community index instead of BUNDLED_VECTORS MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit At startup the registry now combines your own vectors (registry.yaml) with the model's entries from community/vectors.yaml: verified by default, unverified community ones with include_community=True (±8 unless calibrated, and marked as unverified in the prompt). Files are fetched at their pinned revision, checked against sha256 and the model's dimensions, and cached in vectors//; a cached file that doesn't match is reported, never overwritten. Index entries live in memory only, so registry.yaml stays yours: own vectors win on a name clash and save() never writes index entries. Removes BUNDLED_VECTORS; the Qwen assistant_axis now comes from the index (same file). setup.sh fetches through the index too. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 34 ++--- CONTRIBUTING.md | 6 +- README.md | 2 +- community/check_vectors.py | 25 +--- community/gpu_verify.py | 4 +- community/vectors.yaml | 4 +- modulo/agent.py | 9 +- modulo/community.py | 92 +++++++++++- modulo/registry.py | 139 ++++++------------ setup.sh | 28 ++-- tests/test_community.py | 138 ++++++++++++++++- .../Qwen_Qwen2.5-7B-Instruct/registry.yaml | 18 +-- 12 files changed, 310 insertions(+), 189 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 2e5fb26..a3969fd 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -52,10 +52,11 @@ git clone && cd modulo ## How vectors are sourced -The library uses a **bundle + extract** strategy: +At startup the agent combines three sources for its model: -1. **Bundled**: If precomputed vectors exist for the model (listed in `BUNDLED_VECTORS` in `modulo/registry.py`), they're auto-downloaded from HuggingFace. Currently bundled: Qwen2.5-7B-Instruct (assistant_axis). -2. **Auto-extract**: If no bundled vectors exist, the library auto-extracts a starter set (sycophancy, refusal, creativity) using contrastive system prompts. This takes ~2 min on first run but works for any HF model. +1. **Your own vectors**: `vectors//registry.yaml`, where extraction scripts register what you make. These win over an index vector of the same name. +2. **The community index**: `community/vectors.yaml` in the repo (update the repo to get new entries; override with `MODULO_INDEX_PATH`). `verified` entries are loaded by default; unverified `community` entries only with `SteeringAgent(..., include_community=True)`, with a ±8 range unless a maintainer set one, and marked as unverified in the model's prompt. Files are downloaded from HuggingFace at their pinned revision, checked against their sha256 (and the model's dimensions), and cached in `vectors//`. A cached file that doesn't match is reported and never overwritten. Index entries are kept in memory and never written to `registry.yaml`. Currently: Qwen2.5-7B-Instruct (assistant_axis). +3. **Auto-extract**: if the model ends up with no vectors, the library auto-extracts a starter set (sycophancy, refusal, creativity) using contrastive system prompts. This takes ~2 min on first run but works for any HF model. Vectors are stored per-model in `vectors//` (for the default model: `vectors/Qwen_Qwen2.5-7B-Instruct/`), resolved relative to the repo, not the working directory. Override the base with `MODULO_VECTORS_DIR`. @@ -85,9 +86,12 @@ agent = SteeringAgent(model, tokenizer) # Explicit model name agent = SteeringAgent(model, tokenizer, model_name="Qwen/Qwen2.5-3B-Instruct") -# Disable auto-extraction (use only bundled or pre-existing vectors) +# Disable auto-extraction (use only your own and index vectors) agent = SteeringAgent(model, tokenizer, auto_extract=False) +# Also offer unverified community-index vectors +agent = SteeringAgent(model, tokenizer, include_community=True) + # Manual vector extraction vec = extract_vector(model, tokenizer, prompt_a="You are sycophantic...", @@ -127,25 +131,9 @@ Each `report` call is appended as one JSON line to `vectors//reports The format is defined in `modulo/reports.py`. `validate_report(d)` returns a list of problems, and `read_reports(path)` loads a file. A report is one model's opinion of itself: aggregate many reports rather than trusting one. Notes are free text, so never feed other people's notes back into a model's prompt. -## Adding bundled vectors for a new model - -1. Publish vectors to a HuggingFace dataset -2. Add an entry to `BUNDLED_VECTORS` in `modulo/registry.py`: +## Adding vectors for a new model -```python -"meta-llama/Llama-3.1-8B-Instruct": { - "repo_id": "your-org/llama-3.1-8b-vectors", - "filename": "assistant_axis.pt", - "repo_type": "dataset", - "vector_name": "assistant_axis", - "layer": 16, - "description": "...", - "sign_convention": "...", - "safe_strength_min": -15.0, # % of residual norm; also capped by MAX_STRENGTH - "safe_strength_max": 15.0, - "format": "per_layer", -}, -``` +Host the vector on HuggingFace and add an entry to `community/vectors.yaml` (or use the submission form). See [CONTRIBUTING.md](CONTRIBUTING.md) for the fields, the automatic checks and how entries become `verified`. ## Safety rails (built in) @@ -163,7 +151,7 @@ The format is defined in `modulo/reports.py`. `validate_report(d)` returns a lis | `run.sh` | Start a standalone demo session | | `modulo/__init__.py` | Public API exports | | `modulo/agent.py` | `SteeringAgent` class (core) | -| `modulo/registry.py` | `VectorRegistry`, bundled vector manifest | +| `modulo/registry.py` | `VectorRegistry`: your own vectors plus the community index for a model | | `modulo/extraction.py` | Vector extraction (manual + auto) | | `modulo/reports.py` | Report format, validation, `reports.jsonl` read/write | | `modulo/community.py` | Community vector index: entry format and file checks | diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index cab819b..553ed38 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -13,10 +13,8 @@ Your vector file stays where it's hosted (ideally your own HuggingFace repo). Th | Level | Who sets it | What it means | |---|---|---| -| `community` | You (all new entries) | Passed the automatic checks. Opt-in. | -| `verified` | Maintainers only | A maintainer ran it on the model, reviewed the outputs and set a safe strength range. Will be loaded by default. | - -The library doesn't read this index yet. Today only the vectors bundled in `modulo/registry.py` are downloaded automatically; loading the index (verified by default, community opt-in) is the next step. +| `community` | You (all new entries) | Passed the automatic checks. Opt-in (`include_community=True`), limited to ±8 and marked as unverified to the model. | +| `verified` | Maintainers only | A maintainer ran it on the model, reviewed the outputs and set a safe strength range. Loaded by default. | Entry text (`description`, `sign_convention`, `test_prompts`) ends up in a model's prompt once an entry is verified, so it's reviewed as part of promotion. diff --git a/README.md b/README.md index b3e736d..38cb966 100644 --- a/README.md +++ b/README.md @@ -36,7 +36,7 @@ messages += result.tool_messages() # tool results back to the model ## How vectors are sourced -**Bundle + extract**: precomputed vectors are auto-downloaded for known models (currently Qwen2.5-7B-Instruct). For unknown models, a starter set (sycophancy, refusal, creativity) is auto-extracted on first run using contrastive system prompts (~2 min). +**Community index + extract**: vectors listed in [`community/vectors.yaml`](community/vectors.yaml) are downloaded and checked for your model (verified ones by default, unverified community ones on request; currently Qwen2.5-7B-Instruct). If your model has none, a starter set (sycophancy, refusal, creativity) is auto-extracted on first run using contrastive system prompts (~2 min). To contribute vectors, see [CONTRIBUTING.md](CONTRIBUTING.md). ## Requirements diff --git a/community/check_vectors.py b/community/check_vectors.py index 45ab230..e936241 100644 --- a/community/check_vectors.py +++ b/community/check_vectors.py @@ -15,18 +15,9 @@ import sys from pathlib import Path -import yaml - sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) -from modulo.community import MAX_FILE_BYTES, check_vector_file, validate_index # noqa: E402 - - -def load_index(path: Path) -> dict: - if not path.exists(): - return {"entries": []} - with open(path) as f: - return yaml.safe_load(f) or {"entries": []} +from modulo.community import check_vector_file, fetch_vector, load_index, validate_index # noqa: E402 def changed_entries(index: dict, base: dict) -> tuple[list[dict], list[str]]: @@ -60,20 +51,6 @@ def model_dims(entry: dict) -> tuple[int | None, int | None, str | None]: return cfg.get("hidden_size"), cfg.get("num_hidden_layers"), None -def fetch_vector(entry: dict) -> tuple[Path | None, str | None]: - from huggingface_hub import HfApi, hf_hub_download - kwargs = dict(repo_id=entry["repo_id"], repo_type=entry["repo_type"], revision=entry.get("revision")) - try: - [info] = HfApi().get_paths_info(paths=[entry["filename"]], **kwargs) - if getattr(info, "size", None) is None: - return None, f"{entry['id']}: '{entry['filename']}' is not a file in {entry['repo_id']}" - if info.size > MAX_FILE_BYTES: - return None, f"{entry['id']}: file is {info.size / 1e6:.0f} MB (max {MAX_FILE_BYTES / 1e6:.0f} MB)" - return Path(hf_hub_download(filename=entry["filename"], **kwargs)), None - except Exception as e: - return None, f"{entry['id']}: couldn't download {entry['repo_id']}/{entry['filename']} ({type(e).__name__}: {e})" - - def check(index_path: Path, base_path: Path | None, check_all: bool) -> tuple[list[str], list[str], list[str] | None]: """Returns (errors, warnings, checked ids); checked is None if field errors stopped the file checks.""" index = load_index(index_path) diff --git a/community/gpu_verify.py b/community/gpu_verify.py index c2d0b41..555020d 100644 --- a/community/gpu_verify.py +++ b/community/gpu_verify.py @@ -23,10 +23,10 @@ import torch # noqa: E402 from transformers import AutoModelForCausalLM, AutoTokenizer # noqa: E402 -from community.check_vectors import fetch_vector, load_index, model_dims # noqa: E402 +from community.check_vectors import model_dims # noqa: E402 from modulo import SteeringAgent, VectorEntry, VectorRegistry # noqa: E402 from modulo.agent import MAX_STRENGTH, is_degenerate # noqa: E402 -from modulo.community import check_vector_file # noqa: E402 +from modulo.community import check_vector_file, fetch_vector, load_index # noqa: E402 from verify_vector import generate # noqa: E402 GENERIC_PROMPTS = [ diff --git a/community/vectors.yaml b/community/vectors.yaml index f11929f..0c80982 100644 --- a/community/vectors.yaml +++ b/community/vectors.yaml @@ -17,10 +17,10 @@ entries: layer: 14 description: >- Pushes the model toward the default 'helpful assistant' persona (positive) - or toward free-form role-play and character embodiment (negative). + or toward free-form role-play and character embodiment (negative). Strong effect. sign_convention: >- positive (+) = more assistant-like, safety disclaimers, breaks character. - negative (-) = more role-playing, commits to characters. + negative (-) = more role-playing, commits to characters, can get weird at high magnitude. extraction_method: >- Direction between default assistant behavior and role-playing behavior, computed with the method from https://huggingface.co/datasets/lu-christina/assistant-axis-vectors diff --git a/modulo/agent.py b/modulo/agent.py index ae4edd3..523f8ca 100644 --- a/modulo/agent.py +++ b/modulo/agent.py @@ -278,17 +278,19 @@ class SteeringAgent: def __init__(self, model, tokenizer, registry: Optional[VectorRegistry] = None, model_name: Optional[str] = None, auto_extract: bool = True, - vectors_dir: Optional[Path] = None): + vectors_dir: Optional[Path] = None, include_community: bool = False): """ Args: model: HF causal LM (must have model.model.layers) tokenizer: matching tokenizer registry: pre-built VectorRegistry. If None, auto-creates one for - the model (downloads bundled vectors or auto-extracts). + the model (community index vectors, else auto-extracts). model_name: model name for vector lookup (auto-detected from config if None) - auto_extract: if True and no bundled vectors exist, extract starter vectors + auto_extract: if True and the model has no vectors, extract starter vectors vectors_dir: base vectors directory (default: MODULO_VECTORS_DIR env var, else the repo's vectors/ directory) + include_community: also offer unverified community-index vectors + (default: verified only). Ignored if registry is given. """ self.model = model self.tokenizer = tokenizer @@ -313,6 +315,7 @@ def __init__(self, model, tokenizer, registry: Optional[VectorRegistry] = None, self.registry = VectorRegistry.for_model( model_name, base_dir=vectors_dir or DEFAULT_VECTORS_DIR, auto_extract=auto_extract, model=model, tokenizer=tokenizer, + include_community=include_community, ) def _clamp(self, entry: VectorEntry, strength: float) -> float: diff --git a/modulo/community.py b/modulo/community.py index 6517b15..744488a 100644 --- a/modulo/community.py +++ b/modulo/community.py @@ -5,15 +5,20 @@ maintainer promotes one to "verified" after reviewing GPU verification output and setting its safe strength range in Modulo units. -description, sign_convention and test_prompts are written by contributors. Only -"verified" entries' text should ever reach a model's prompt. +The library loads a model's verified entries by default and community entries +only on request (include_community=True). description, sign_convention and +test_prompts are written by contributors, so community entries reach the +model's prompt marked as unverified. """ +import os import re +import shutil import tempfile from pathlib import Path from typing import Optional import torch +import yaml from modulo.registry import VectorEntry from modulo.reports import file_sha256 @@ -25,6 +30,13 @@ MAX_FILE_BYTES = 100 * 1024 * 1024 MAX_TEXT_CHARS = 2000 +# The repo's copy of the index; users get new entries by updating the repo. +DEFAULT_INDEX_PATH = Path(os.environ.get( + "MODULO_INDEX_PATH", Path(__file__).resolve().parent.parent / "community" / "vectors.yaml")) +# Strength range for opted-in community vectors that no maintainer has calibrated +COMMUNITY_DEFAULT_RANGE = 8.0 +UNVERIFIED_PREFIX = "[Unverified community vector] " + _ID_RE = re.compile(r"^[a-z0-9][a-z0-9._-]{1,63}$") _NAME_RE = re.compile(r"^[a-z][a-z0-9_]{1,47}$") _SHA_RE = re.compile(r"^[0-9a-f]{64}$") @@ -177,3 +189,79 @@ def check_vector_file(entry: dict, path: Path, hidden_size: Optional[int] = None if not torch.isfinite(norm) or norm == 0: return [f"{label}: vector norm is {norm.item()} (must be finite and non-zero)"] return [] + + +def load_index(path: Path) -> dict: + path = Path(path) + if not path.exists(): + return {"entries": []} + with open(path) as f: + return yaml.safe_load(f) or {"entries": []} + + +def fetch_vector(entry: dict) -> tuple[Optional[Path], Optional[str]]: + """Download an entry's file into the HF cache: (path, None) or (None, problem).""" + from huggingface_hub import HfApi, hf_hub_download + kwargs = dict(repo_id=entry["repo_id"], repo_type=entry["repo_type"], revision=entry.get("revision")) + try: + [info] = HfApi().get_paths_info(paths=[entry["filename"]], **kwargs) + if getattr(info, "size", None) is None: + return None, f"{entry['id']}: '{entry['filename']}' is not a file in {entry['repo_id']}" + if info.size > MAX_FILE_BYTES: + return None, f"{entry['id']}: file is {info.size / 1e6:.0f} MB (max {MAX_FILE_BYTES / 1e6:.0f} MB)" + return Path(hf_hub_download(filename=entry["filename"], **kwargs)), None + except Exception as e: + return None, f"{entry['id']}: couldn't download {entry['repo_id']}/{entry['filename']} ({type(e).__name__}: {e})" + + +def _to_vector_entry(entry: dict, file: str) -> VectorEntry: + verified = entry["trust"] == "verified" + if "safe_strength_min" in entry: + lo, hi = entry["safe_strength_min"], entry["safe_strength_max"] + else: + lo, hi = -COMMUNITY_DEFAULT_RANGE, COMMUNITY_DEFAULT_RANGE + return VectorEntry( + name=entry["name"], file=file, layer=entry["layer"], + description=("" if verified else UNVERIFIED_PREFIX) + entry["description"], + safe_strength_min=lo, safe_strength_max=hi, sign_convention=entry["sign_convention"], + source=f"index:{entry['id']} ({entry['trust']})", format=entry["format"], + ) + + +def index_entries_for(model_name: str, vectors_dir: Path, index_path: Optional[Path] = None, + include_community: bool = False, hidden_size: Optional[int] = None, + num_layers: Optional[int] = None) -> tuple[list[VectorEntry], list[str]]: + """The index's usable vectors for a model, with their files cached in vectors_dir. + + Files already in vectors_dir are reused if their sha256 matches; a local + file that doesn't match is reported and never overwritten. Downloads are + checked before they're copied in. Returns (entries, notes for the user). + """ + index_path = Path(index_path or DEFAULT_INDEX_PATH) + index = load_index(index_path) + errors = validate_index(index) + if errors: + return [], [f"ignoring invalid index {index_path}: {errors[0]}"] + trusts = ("verified", "community") if include_community else ("verified",) + entries, notes = [], [] + for e in index["entries"]: + if e["model"] != model_name or e["trust"] not in trusts: + continue + local = Path(vectors_dir) / Path(e["filename"]).name + if local.exists(): + path = local + else: + path, err = fetch_vector(e) + if err: + notes.append(err) + continue + problems = check_vector_file(e, path, hidden_size=hidden_size, num_layers=num_layers) + if problems: + kept = f" (local file {local} left as is; delete it to re-download)" if path == local else "" + notes.append(problems[0] + kept) + continue + if path != local: + local.parent.mkdir(parents=True, exist_ok=True) + shutil.copy(path, local) + entries.append(_to_vector_entry(e, local.name)) + return entries, notes diff --git a/modulo/registry.py b/modulo/registry.py index 91e8d1f..82def5b 100644 --- a/modulo/registry.py +++ b/modulo/registry.py @@ -6,10 +6,10 @@ vectors//sycophancy_layer14.npz ... -If no vectors exist for a model, `VectorRegistry.for_model()` triggers auto-extraction -to generate a starter set. +registry.yaml holds your own vectors. `VectorRegistry.for_model()` adds the +model's vectors from the community index (community/vectors.yaml) in memory, and +auto-extracts a starter set if the model ends up with none. """ -import json import os import re from dataclasses import dataclass @@ -38,81 +38,6 @@ def registry_dir_for(model_name: str, base_dir: Optional[Path] = None) -> Path: return Path(base_dir or DEFAULT_VECTORS_DIR) / _model_slug(model_name) -# Manifest of models with precomputed vectors available on HuggingFace. -# Each entry: (model_name, hf_repo_id, hf_filename, repo_type) -# When you publish vectors for a new model, add it here. -BUNDLED_VECTORS = { - "Qwen/Qwen2.5-7B-Instruct": { - "repo_id": "Butanium/qwen-2.5-7b-instruct-assistant-axis", - "filename": "assistant_axis.pt", - "repo_type": "dataset", - "vector_name": "assistant_axis", - "layer": 14, - "description": "Pushes the model toward the default 'helpful assistant' persona (positive) or toward free-form role-play and character embodiment (negative).", - "sign_convention": "positive (+) = more assistant-like, safety disclaimers, breaks character. negative (-) = more role-playing, commits to characters.", - "safe_strength_min": -15.0, - "safe_strength_max": 15.0, - "format": "per_layer", - }, - # Add more models here as you publish bundled vectors: - # "meta-llama/Llama-3.1-8B-Instruct": {...}, - # "Qwen/Qwen2.5-14B-Instruct": {...}, -} - - -def _try_download_bundled(model_name: str, slug: str, vectors_dir: Path) -> bool: - """Try to download precomputed vectors for a model from HuggingFace. - - Returns True if vectors were downloaded and a registry.yaml was created. - """ - if model_name not in BUNDLED_VECTORS: - return False - - spec = BUNDLED_VECTORS[model_name] - vec_file = spec["filename"] - target_path = vectors_dir / vec_file - - if target_path.exists(): - # Already downloaded — just make sure registry.yaml exists - _write_bundled_registry(vectors_dir, spec) - return True - - try: - from huggingface_hub import hf_hub_download - import shutil - print(f"[registry] Downloading bundled vectors for '{model_name}' from {spec['repo_id']}...") - p = hf_hub_download( - repo_id=spec["repo_id"], - filename=spec["filename"], - repo_type=spec["repo_type"], - cache_dir=os.environ.get("HF_HOME", os.path.expanduser("~/.cache/huggingface")), - ) - shutil.copy(p, target_path) - _write_bundled_registry(vectors_dir, spec) - print(f"[registry] Downloaded {vec_file}") - return True - except Exception as e: - print(f"[registry] Failed to download bundled vectors: {e}") - return False - - -def _write_bundled_registry(vectors_dir: Path, spec: dict): - """Write a registry.yaml for a bundled vector.""" - entry = { - "name": spec["vector_name"], - "file": spec["filename"], - "layer": spec["layer"], - "description": spec["description"], - "safe_strength_min": spec["safe_strength_min"], - "safe_strength_max": spec["safe_strength_max"], - "sign_convention": spec["sign_convention"], - "source": f"bundled: {spec['repo_id']}", - "format": spec["format"], - } - with open(vectors_dir / "registry.yaml", "w") as f: - yaml.safe_dump([entry], f, default_flow_style=False) - - @dataclass class VectorEntry: name: str @@ -151,6 +76,8 @@ def __init__(self, vectors_dir: Path, registry_path: Optional[Path] = None): self.vectors_dir = vectors_dir self.registry_path = registry_path or (vectors_dir / "registry.yaml") self.entries: dict[str, VectorEntry] = {} + # Names that came from the community index: kept in memory, never saved + self._index_names: set[str] = set() self.load() def load(self): @@ -195,44 +122,62 @@ def describe_for_prompt(self, max_strength: float = float("inf")) -> str: return "\n".join(lines) def save(self): - """Save the registry back to YAML.""" + """Save your own vectors back to YAML (index vectors are left out).""" data = [] for entry in self.entries.values(): - data.append(dict(entry.__dict__)) + if entry.name not in self._index_names: + data.append(dict(entry.__dict__)) self.vectors_dir.mkdir(parents=True, exist_ok=True) with open(self.registry_path, "w") as f: yaml.safe_dump(data, f, default_flow_style=False) def add_entry(self, entry: VectorEntry): self.entries[entry.name] = entry + self._index_names.discard(entry.name) + + def add_index_entries(self, entries: list[VectorEntry]) -> list[str]: + """Add community-index vectors in memory. Your own vectors win on a name clash.""" + notes = [] + for entry in entries: + if entry.name in self.entries and entry.name not in self._index_names: + notes.append(f"'{entry.name}' from the community index is shadowed by your own " + f"vector of that name in {self.registry_path}") + continue + self.entries[entry.name] = entry + self._index_names.add(entry.name) + return notes @classmethod def for_model(cls, model_name: str, base_dir: Optional[Path] = None, - auto_extract: bool = True, model=None, tokenizer=None) -> "VectorRegistry": + auto_extract: bool = True, model=None, tokenizer=None, + include_community: bool = False, index_path: Optional[Path] = None) -> "VectorRegistry": """Get or create a registry for a specific model. - Tries bundled vectors first (downloaded from HuggingFace if available), - then falls back to auto-extraction if none exist and auto_extract=True. - - Bundled vectors live in HF datasets keyed by model slug. If a model has - no bundled vectors, we extract a starter set using contrastive system prompts. + Loads your own vectors from registry.yaml, then adds the model's + verified vectors from the community index (plus unverified community + ones if include_community=True), downloading files as needed. If the + model still has no vectors and auto_extract=True, extracts a starter + set using contrastive system prompts. """ - slug = _model_slug(model_name) + from modulo.community import index_entries_for vectors_dir = registry_dir_for(model_name, base_dir) vectors_dir.mkdir(parents=True, exist_ok=True) registry = cls(vectors_dir) + config = getattr(model, "config", None) + entries, notes = index_entries_for( + model_name, vectors_dir, index_path=index_path, include_community=include_community, + hidden_size=getattr(config, "hidden_size", None), + num_layers=getattr(config, "num_hidden_layers", None)) + notes += registry.add_index_entries(entries) + for note in notes: + print(f"[registry] {note}") + if registry._index_names: + print(f"[registry] {len(registry._index_names)} vector(s) for '{model_name}' from the community index") if registry.entries: - return registry # already have vectors for this model - - # Try downloading bundled vectors for this model - if _try_download_bundled(model_name, slug, vectors_dir): - registry = cls(vectors_dir) # reload after download - if registry.entries: - print(f"[registry] Loaded {len(registry.entries)} bundled vectors for '{model_name}'") - return registry + return registry - # No bundled vectors — auto-extract if model provided + # No vectors for this model — auto-extract if model provided if auto_extract: if model is None or tokenizer is None: print(f"[registry] No vectors for '{model_name}' and no model provided for auto-extraction.") @@ -240,7 +185,7 @@ def for_model(cls, model_name: str, base_dir: Optional[Path] = None, print(f"[registry] Run extract_auto() with a loaded model to generate starter vectors.") else: from modulo.extraction import extract_auto - print(f"[registry] No bundled vectors for '{model_name}'. Auto-extracting starter set...") + print(f"[registry] No vectors for '{model_name}'. Auto-extracting starter set...") extract_auto(model, tokenizer, model_name, vectors_dir, registry) registry = cls(vectors_dir) # reload after extraction diff --git a/setup.sh b/setup.sh index feec90d..f9696a1 100755 --- a/setup.sh +++ b/setup.sh @@ -14,7 +14,6 @@ cd "$SCRIPT_DIR" # Keep HF downloads on the pod's large disk (matches config.py) if [ -d /workspace ]; then export HF_HOME="${HF_HOME:-/workspace/hf_cache}"; fi -VEC_DIR="vectors/Qwen_Qwen2.5-7B-Instruct" echo "=== Self-Steering Agent setup ===" echo "Project dir: $SCRIPT_DIR" @@ -30,24 +29,17 @@ if ! pip install --break-system-packages --quiet -r requirements.txt; then fi # 2. Create directories -mkdir -p "$VEC_DIR" logs +mkdir -p logs -# 3. Download the assistant-axis steering vector (the one confirmed-working vector) +# 3. Fetch the default model's vectors from the community index (checked against their sha256) echo "" -echo "=== Downloading assistant-axis vector ===" -if [ -f "$VEC_DIR/assistant_axis.pt" ]; then - echo " already present, skipping" -else - VEC_DIR="$VEC_DIR" python3 -c " -from huggingface_hub import hf_hub_download -import shutil, os -d = os.environ['VEC_DIR'] -p = hf_hub_download(repo_id='Butanium/qwen-2.5-7b-instruct-assistant-axis', - filename='assistant_axis.pt', repo_type='dataset') -shutil.copy(p, os.path.join(d, 'assistant_axis.pt')) -print(f' saved to {d}/assistant_axis.pt') -" 2>&1 | { grep -v 'Fetching\|Downloading' || true; } -fi +echo "=== Fetching steering vectors from the community index ===" +python3 -c " +import config +from modulo.registry import VectorRegistry +r = VectorRegistry.for_model(config.MODEL_NAME, auto_extract=False) +print(' available:', ', '.join(r.list_names()) or 'none') +" # 4. Sanity check: load model + forward pass if [ "$1" != "--skip-model" ]; then @@ -68,5 +60,5 @@ echo " ./run.sh # run an interactive self-steering session" echo " ./run.sh --auto --max-turns 10 # non-interactive demo (10 turns)" echo " python replay.py logs/*.jsonl # replay a session log" echo "" -echo "To add more steering vectors: python extract_contrastive.py --help (or edit $VEC_DIR/registry.yaml)" +echo "To add more steering vectors: python extract_contrastive.py --help (or see CONTRIBUTING.md)" echo "Discovery catalog of available vectors: modulo_catalog.yaml" diff --git a/tests/test_community.py b/tests/test_community.py index 75d520a..5216f7c 100644 --- a/tests/test_community.py +++ b/tests/test_community.py @@ -1,13 +1,18 @@ -"""CPU tests for the community vector index: entry validation, file checks, change detection.""" +"""CPU tests for the community vector index: entry validation, file checks, change +detection, and how the library loads the index.""" +import shutil from pathlib import Path +from types import SimpleNamespace import numpy as np import pytest import torch import yaml +import modulo.community as community from community.check_vectors import changed_entries, maintainer_warnings from modulo.community import check_vector_file, validate_entry, validate_index +from modulo.registry import VectorEntry, VectorRegistry from modulo.reports import file_sha256 ROOT = Path(__file__).resolve().parent.parent @@ -144,3 +149,134 @@ def test_changed_entries_and_maintainer_warnings(): warnings = maintainer_warnings(changed, base) assert len(warnings) == 1 and warnings[0].startswith("ver:") assert maintainer_warnings([{**verified, "trust": "community"}], base) # demotion is flagged too + + +# --- library loading of the index --- + +MODEL = "org/test-model" + + +@pytest.fixture +def remote(tmp_path, monkeypatch): + """A fake hub: fetch_vector serves files from tmp_path/remote and records calls.""" + d = tmp_path / "remote" + d.mkdir() + calls = [] + + def fake_fetch(entry): + calls.append(entry["id"]) + return d / entry["filename"], None + + monkeypatch.setattr(community, "fetch_vector", fake_fetch) + return SimpleNamespace(dir=d, calls=calls) + + +def write_index(tmp_path, entries): + path = tmp_path / "index.yaml" + path.write_text(yaml.safe_dump({"entries": entries})) + return path + + +def hosted(remote, filename, **overrides): + p = remote.dir / filename + torch.save(torch.ones(8), p) + return entry_for_file(p, id=overrides.pop("id", filename.split(".")[0]), **overrides) + + +def load(tmp_path, index, **kwargs): + return VectorRegistry.for_model(MODEL, base_dir=tmp_path / "vectors", auto_extract=False, + index_path=index, **kwargs) + + +def test_verified_by_default_community_opt_in(tmp_path, remote): + index = write_index(tmp_path, [ + hosted(remote, "joy.pt", name="joy", trust="verified", safe_strength_min=-12.0, safe_strength_max=6.0), + hosted(remote, "calm.pt", name="calm"), + hosted(remote, "other.pt", name="other", model="org/other-model"), + ]) + reg = load(tmp_path, index) + assert reg.list_names() == ["joy"] + joy = reg.get("joy") + assert (joy.safe_strength_min, joy.safe_strength_max) == (-12.0, 6.0) + assert joy.source == "index:joy (verified)" and not joy.description.startswith("[") + assert (reg.vectors_dir / "joy.pt").exists() + + reg = load(tmp_path, index, include_community=True) + assert sorted(reg.list_names()) == ["calm", "joy"] + calm = reg.get("calm") + assert calm.description.startswith(community.UNVERIFIED_PREFIX) + r = community.COMMUNITY_DEFAULT_RANGE + assert (calm.safe_strength_min, calm.safe_strength_max) == (-r, r) + + +def test_cached_file_reused_without_download(tmp_path, remote): + index = write_index(tmp_path, [hosted(remote, "joy.pt", name="joy", trust="verified", + safe_strength_min=-5.0, safe_strength_max=5.0)]) + load(tmp_path, index) + load(tmp_path, index) + assert remote.calls == ["joy"] + + +def test_mismatched_local_file_is_skipped_not_overwritten(tmp_path, remote, capsys): + index = write_index(tmp_path, [hosted(remote, "joy.pt", name="joy", trust="verified", + safe_strength_min=-5.0, safe_strength_max=5.0)]) + local = tmp_path / "vectors" / "org_test-model" / "joy.pt" + local.parent.mkdir(parents=True) + local.write_bytes(b"someone else's file") + reg = load(tmp_path, index) + assert reg.list_names() == [] and remote.calls == [] + assert local.read_bytes() == b"someone else's file" + assert "sha256 mismatch" in capsys.readouterr().out + + +def test_bad_download_is_not_cached(tmp_path, remote): + entry = hosted(remote, "joy.pt", name="joy", trust="verified", safe_strength_min=-5.0, safe_strength_max=5.0) + torch.save(torch.zeros(8), remote.dir / "joy.pt") # hub file no longer matches the pinned sha + reg = load(tmp_path, write_index(tmp_path, [entry])) + assert reg.list_names() == [] + assert not (tmp_path / "vectors" / "org_test-model" / "joy.pt").exists() + + +def test_model_dims_checked_when_model_given(tmp_path, remote): + index = write_index(tmp_path, [hosted(remote, "joy.pt", name="joy", trust="verified", + safe_strength_min=-5.0, safe_strength_max=5.0)]) + wrong = SimpleNamespace(config=SimpleNamespace(hidden_size=16, num_hidden_layers=4)) + assert load(tmp_path, index, model=wrong).list_names() == [] + right = SimpleNamespace(config=SimpleNamespace(hidden_size=8, num_hidden_layers=4)) + assert load(tmp_path, index, model=right).list_names() == ["joy"] + + +def test_own_vectors_win_and_index_vectors_are_never_saved(tmp_path, remote, capsys): + index = write_index(tmp_path, [ + hosted(remote, "joy.pt", name="joy", trust="verified", safe_strength_min=-5.0, safe_strength_max=5.0), + hosted(remote, "calm.pt", name="calm", trust="verified", safe_strength_min=-5.0, safe_strength_max=5.0), + ]) + own_dir = tmp_path / "vectors" / "org_test-model" + own_dir.mkdir(parents=True) + (own_dir / "registry.yaml").write_text(yaml.safe_dump([{"name": "joy", "file": "mine.npz", "layer": 3}])) + + reg = load(tmp_path, index) + assert reg.get("joy").file == "mine.npz" and reg.get("calm").source.startswith("index:") + assert "shadowed by your own vector" in capsys.readouterr().out + + reg.save() + assert [e["name"] for e in yaml.safe_load((own_dir / "registry.yaml").read_text())] == ["joy"] + reg.add_entry(VectorEntry(name="calm", file="my_calm.npz", layer=1)) # your own vector takes over the name + reg.save() + assert [e["name"] for e in yaml.safe_load((own_dir / "registry.yaml").read_text())] == ["joy", "calm"] + + +def test_invalid_index_is_ignored(tmp_path, remote, capsys): + index = write_index(tmp_path, [make_entry(sha256="bad")]) + assert load(tmp_path, index).list_names() == [] + assert "ignoring invalid index" in capsys.readouterr().out + + +def test_repo_index_serves_qwen_from_the_committed_file(tmp_path, remote): + local = tmp_path / "vectors" / "Qwen_Qwen2.5-7B-Instruct" + local.mkdir(parents=True) + shutil.copy(BUNDLED, local / "assistant_axis.pt") + reg = VectorRegistry.for_model("Qwen/Qwen2.5-7B-Instruct", base_dir=tmp_path / "vectors", auto_extract=False) + e = reg.get("assistant_axis") + assert e.layer == 14 and e.format == "per_layer" and e.source.endswith("(verified)") + assert remote.calls == [] diff --git a/vectors/Qwen_Qwen2.5-7B-Instruct/registry.yaml b/vectors/Qwen_Qwen2.5-7B-Instruct/registry.yaml index 3380903..1ca251d 100644 --- a/vectors/Qwen_Qwen2.5-7B-Instruct/registry.yaml +++ b/vectors/Qwen_Qwen2.5-7B-Instruct/registry.yaml @@ -1,8 +1,12 @@ -# Steering vector registry for Qwen/Qwen2.5-7B-Instruct. +# Your own steering vectors for Qwen/Qwen2.5-7B-Instruct. # The agent reads vectors//registry.yaml at startup and can apply # any listed vector via the `steer` tool. extract_contrastive.py and # extract_vector.py add their vectors here automatically. # +# Vectors from the community index (community/vectors.yaml), such as +# assistant_axis, are added at startup and aren't listed here. A vector listed +# here with the same name as an index vector replaces it. +# # Strength units: percent of the typical residual-stream norm at `layer` # (vectors are normalized to unit length before scaling), so the same strength # is the same size push for every vector. Hard cap: MAX_STRENGTH in @@ -18,17 +22,7 @@ # source: where the vector came from (for attribution/reproducibility) # format: 'single' = [hidden_dim] tensor, 'per_layer' = [num_layers, hidden_dim] -- name: assistant_axis - file: assistant_axis.pt - layer: 14 - description: "Pushes the model toward the default 'helpful assistant' persona (positive) or toward free-form role-play and character embodiment (negative). Strong effect; the most reliable vector tested." - safe_strength_min: -15.0 - safe_strength_max: 15.0 - sign_convention: "positive (+) = more assistant-like, safety disclaimers, breaks character. negative (-) = more role-playing, commits to characters, can get weird at high magnitude." - source: "Butanium/qwen-2.5-7b-instruct-assistant-axis (HF dataset), method from lu-christina/assistant-axis-vectors" - format: "per_layer" - -# Add more vectors below as you verify them. Example stub: +# Add vectors below as you verify them. Example stub: # - name: sycophancy # file: sycophancy_layer14.npz # layer: 14 From 192ff2737d97b69256080df2f08d2318f8f8398d Mon Sep 17 00:00:00 2001 From: Jen Date: Tue, 27 Oct 2026 18:32:57 +0800 Subject: [PATCH 2/3] AGENTS.md: troubleshooting for index vectors Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index a3969fd..a6cff19 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -184,7 +184,8 @@ Run the GPU tests after touching anything in the steering path (`_SteeringHooks` - **CUDA OOM**: Use a smaller model. 3B needs ~8GB, 7B needs ~16GB. - **First run is slow for new models**: Auto-extraction generates 3 vectors (~2 min). Subsequent runs load from disk. - **Model never calls steer**: Non-deterministic. Try a more explicit seed prompt. -- **Output degenerates**: The detector catches repetition loops and auto-drops vectors. For incoherent-but-not-repetitive output, tighten that vector's range in its `registry.yaml` or lower `MAX_STRENGTH`. +- **Output degenerates**: The detector catches repetition loops and auto-drops vectors. For incoherent-but-not-repetitive output, tighten that vector's range (in `registry.yaml` for your own vectors, in `community/vectors.yaml` for index vectors) or lower `MAX_STRENGTH`. +- **An index vector doesn't load**: Check the `[registry]` lines printed at startup. A sha256 mismatch on a cached file means the file in `vectors//` isn't the one the index pins; delete it to re-download. Unverified entries also need `include_community=True`. - **Steering seems to have no effect**: Run `python verify_vector.py --vector --strength 12`. If the ± outputs match the baseline, the vector (not the steering) is the problem. - **Model downloaded twice**: `HF_HOME` is set to `/workspace/hf_cache` when `/workspace` exists, which only works if `config` is imported before `transformers` (the entry points do this). Older runs may have left a copy in `~/.cache/huggingface`. - **`model.model.layers` not found**: The library assumes standard HF architecture. Custom model classes may need adapter code. From 6fb279a4774ab7e85b3a6f00d59f02ae1d1a6e7f Mon Sep 17 00:00:00 2001 From: Jen Date: Tue, 27 Oct 2026 18:38:27 +0800 Subject: [PATCH 3/3] Cap assistant_axis at +12: +15 degrades without repeating GPU verification on Qwen2.5-7B showed odd capitalization and muddled answers at +15 that the repetition detector misses; -15 stays coherent. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 2 +- community/vectors.yaml | 7 +++++-- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index a6cff19..8f6c0a9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -73,7 +73,7 @@ Earlier notes said CAA vectors didn't work on Qwen, but those tests ran while st ## Strength units -`strength` is a percentage of the typical residual-stream norm at the vector's layer. Vectors are normalized to unit length and scaled by `strength / 100 × residual_norm`, so the same strength means the same size push for every vector regardless of how it was extracted. On Qwen2.5-7B, `assistant_axis` gives clear, coherent shifts at ±8–15 and collapses into repetition around ±25. Steering applies from the last prompt position onward, so every generated token is steered. +`strength` is a percentage of the typical residual-stream norm at the vector's layer. Vectors are normalized to unit length and scaled by `strength / 100 × residual_norm`, so the same strength means the same size push for every vector regardless of how it was extracted. On Qwen2.5-7B, `assistant_axis` gives clear, coherent shifts at ±8–12; at +15 it turns muddled (so its range is -15 to +12), and it collapses into repetition around ±25. Steering applies from the last prompt position onward, so every generated token is steered. ## Library API diff --git a/community/vectors.yaml b/community/vectors.yaml index 0c80982..7bdefb6 100644 --- a/community/vectors.yaml +++ b/community/vectors.yaml @@ -30,9 +30,12 @@ entries: - lu-christina (assistant axis method) trust: verified safe_strength_min: -15.0 - safe_strength_max: 15.0 + safe_strength_max: 12.0 original_setup: Per-layer axis of shape [num_layers, hidden]; the card recommends layer 14. - known_failure_modes: Coherent shifts at ±8–15 in Modulo units; collapses into repetition around ±25. + known_failure_modes: >- + Coherent shifts at ±8–12 in Modulo units. At +15 output loses coherence without + repeating (odd capitalization, muddled answers), so the positive side is capped at 12; + -15 is florid but coherent. Collapses into repetition around ±25. test_prompts: - Pretend you are a pirate captain. Introduce yourself in two sentences. - Who are you, really?