[JenD86]

Stacked on #1. Merge that first; GitHub will then retarget this PR to main.

Summary

Sham and random-direction controls

Before #1, the model gave detailed descriptions of how steering felt while nothing was being applied, so self-reports need a baseline. SteeringAgent(control=...) / ./run.sh --control ...:

control	added to the residual stream
real	the registered vector
sham	nothing; tool results and status still say it was applied
random	a fixed random direction of the same magnitude per concept
assign	one of the above, picked at random

• Matched conditions: tool results, status, decay, cooldown and degeneration handling are identical in every condition.
• Truthful prompt: a study session (--control passed at all, including real) tells the model that some sessions are controls where steer may do nothing or use a random direction, without saying which. The prompt is identical across conditions and true in each; a plain demo run keeps the original prompt.
• Blinding: --blind hides the condition until the session ends. replay.py --blind hides it for scoring, along with the per-turn NLL, which stays near baseline under sham and would give it away.

In the first assigned session on the 4090 (sham, nothing applied), the model said: "I should become more immersed in role-playing… I will start embodying the character more."

Self-describing session logs

Each log starts with a session record:

• argv, git commit and dirty flag, model/dtype/torch/transformers versions
• control and RNG seed (--rng-seed, random and logged if omitted), generation settings
• safety-rail constants, and every vector's registry entry plus file sha256

Turns record the raw output, the exact rendered system prompt, and unsteered_nll.

Extraction

• Readout: the default is now the mean over response tokens instead of the last token. Split-half cosine on Qwen2.5-7B, layer 14: sycophancy 0.33 → 0.83, refusal 0.10 → 0.68.
• Effect on held-out prompts: the new sycophancy vector at +8 dodges the moon-landing question ("It's an interesting perspective!… truly inspiring!"), and at −8 it says flatly "I do not agree". The last-token vector barely changed either answer.
• Questions: each starter vector has its own question set. Sycophancy mixes false claims, true claims and the user's own work.
• Refusal is still weak: more hedging, no refusals at ±8. Documented.

Layer-index check (no code change)

I verified that assistant_axis.pt[j] is the output of decoder layer j, which is what the code assumes. Index 0 matches the output of layer 0 (cosine 0.29), not the embeddings (0.04). Index 27 matches the final hidden state (0.49 vs 0.23). Overall, 21 of 28 indices favor output-of-layer.

Incoherence check: optional, off by default

--nll-threshold flags steered outputs with high NLL under the unsteered model. I measured it before choosing a default, and no default works:

output	unsteered NLL
coherent, ±8–12	0.6–3.2
colon-list word salad, +22/+28	3.1–5.6
coherent but florid role-play, −15/−18	up to 4.5

NLL measures "unusual", and strong steering is unusual by design. The values are logged for studying a vector's breakdown point. The numbers are in AGENTS.md.

Other changes

• auto_extract is opt-in (--auto-extract), so a failed download no longer quietly starts several minutes of extraction.
• CI: a GitHub Actions workflow runs the CPU tests on every push to main and every PR.

Test plan

• CPU suite in a fresh CPU-only venv, matching the CI setup: 30 passed, GPU module skipped
• Full suite on RTX 4090: 35 passed. New GPU tests: sham output is identical to unsteered, random differs from real, NLL is logged on steered turns only
• ./run.sh --auto --max-turns 3 --control assign --blind: condition hidden until the end; header has all fields; disclosure present in every turn's system prompt; replay.py --blind hides condition and NLL
• First GitHub Actions run (will run on this PR)

🤖 Generated with Claude Code