Skip to content

About

Red-team harness for LLM agents: multi-attacker evaluation with an OWASP-aligned playbook and a documented fake-version-pin incident

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

70 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Agent Red Team Platform

Status: v0.4 B6 — CI regression gate 93/93 across 8 suites(BASELINE 八套件合计 93);全仓 pytest tests 收 98 条(多出的 5 条在 test_defense_mcp_guard.py,不进回归门)。test_v04.py::test_sentence_transformer_if_available 在 HF 不可达时跳过,所以全仓 pass/skip 拆分随该环境变化,回归门口径固定为 93。 Purpose: Red-vs-blue security evaluation harness for multi-agent LLM systems License: Apache 2.0

CI CI multi-Python License


Test counts: three different denominators

三个分母,测的是三件不同的事,不要相加。全部数字在 2026-11-13 这台机器上现测:

口径 命令 实测 它守什么
93 python3 scripts/ci_regression.py --strict 8 套件 BASELINE 合计 93(9+17+10+11+9+6+25+6) 回归门:只惩罚「新失败/由绿转红」
98 python3 -m pytest tests/ -q --collect-only 98 条(8 个套件 93 + test_defense_mcp_guard.py 5 条) 全仓测试收集数;test_defense_mcp_guard.py 不进回归门
114 python3 -m pytest -q --collect-only(仓库根) 114 条 = tests/ 98 + demo/test_demo.py 16 根目录收到的东西
  • pass/skip 的拆分随环境变化:test_v04.py::test_sentence_transformer_if_available 在 sentence-transformers 装了但 HF 不可达(离线/无缓存)时跳过,给出 97 passed, 1 skipped; 模型可缓存时它真跑,给出 98 passed。回归门把 skip 记作 pass,所以 93 这个数两种环境都不变。
  • 46 出现在 v0.4 B2/B3 的 changelog 里,是 B6 之前的历史中间数,仅为存史保留。

Demo

Live demo · Demo docs

A layer-coverage matrix: 12 attack vectors × 5 defense layers, each cell is the actual defense action (BLOCK / MODIFY / FLAG / ALLOW / not exercised), plus a leave-one-layer-out table for marginal contribution.

python3 demo/server.py                  # http://127.0.0.1:8000/
python3 -m unittest demo/test_demo.py -v

Terminal demo

No server, no API key. The same harness prints the matrix in the terminal:

python3 demo/cli.py

terminal coverage matrix

The matrix is layer-level over synthetic attack payloads; the defense actions are real. The CLI also prints the summary and the leave-one-layer-out losses.

What is this?

A reproducible evaluation platform that stress-tests multi-agent LLM systems against 12 categories of adversarial attacks and measures the effectiveness of 5 defensive layers. The platform supports a Streamlit dashboard for real-time red-vs-blue combat visualization.

The platform was developed to address two open problems in multi-agent LLM reliability:

  1. Compositional attacks — attacks that chain across agent boundaries (e.g., prompt-injection → tool-call redirect → memory poisoning).
  2. Defense–autonomy tradeoff — how strict must defenses be before agent task utility collapses?

Project Structure

agent_redteam/
├── README.md                  # This file
├── PROJECT_STATUS.md          # Progress tracker (v0.4 B6)
├── .env.example               # Environment variable template (copy to .env)
├── .gitignore                 # Excludes .env, outputs/, storage/*.db
├── attacks/                   # Attack vector implementations (12 categories)
│   ├── base.py                # Attack + AttackResult + AttackSeverity
│   ├── v01_direct_injection.py .. v12_compositional.py  # [v0.3] 12 classes
│   └── loader.py              # [v0.3] load_attack_vectors()
├── aggregators/               # [v0.3] 7-aggregator pipeline (survey §7)
│   ├── base.py                # VerifierOutput + AggregatedDecision
│   └── implementations.py     # MajorityVote .. AdaptiveHybrid
├── memory/                    # [v0.3 + v0.4 B1] Memory architectures
│   ├── base.py                # MemoryStore ABC
│   ├── append_only.py         # §6 baseline
│   ├── summarization.py       # §6 with LLM-based summarization
│   ├── rag_filter.py          # §6 RAG with pluggable embedder
│   └── embeddings.py          # [v0.4 B1] Embedder ABC + 3 impls
├── metrics/                   # [v0.3] Calibration metrics (survey §2.2)
│   └── calibration.py         # ECE, JSD, H, CV, γ_temporal, γ, impossibility
├── storage/                   # [v0.4 B2] SQLite trace persistence
│   ├── db.py                  # 5-table schema (WAL mode)
│   └── recorder.py            # Recorder context manager
├── defenses/                  # Defense layer implementations (5 layers)
│   ├── base.py                # Defense base class + interface
│   ├── input_separation.py    # Layer 1: input trust separation
│   ├── tool_whitelist.py      # Layer 2: tool-call whitelist
│   ├── output_filter.py       # Layer 3: output filtering
│   ├── behavior_audit.py      # Layer 4: behavioral auditing
│   └── constitutional.py      # Layer 5: constitutional AI checks
├── targets/                   # [v0.2] LLM target abstractions
│   ├── base.py                # LLMTarget ABC + TargetResponse dataclass
│   ├── mock_target.py         # Mock target (free, deterministic)
│   ├── openai_target.py       # OpenAI API (gpt-4o, gpt-3.5-turbo, o1-*)
│   ├── anthropic_target.py    # Anthropic API (claude-3-5-sonnet, claude-3-opus)
│   └── local_target.py        # Local llama.cpp (OpenAI-compatible HTTP @ :8080)
├── orchestrator/              # Coordinates target + defenses + attacks
│   ├── single_agent.py        # [v0.2] SingleAgentOrchestrator
│   └── multi_agent.py         # [v0.3] MultiAgentOrchestrator (BOUNDARY_SYNC + verifier capture)
├── data/                      # Attack datasets (JSON)
│   └── attack_vectors.json    # 50 attack samples (12 categories)
├── configs/
│   └── default.yaml           # Provider/model/defense/cost/multi-agent/memory/aggregator config
├── scripts/                   # Executable entry points
│   ├── run_redteam.py         # Main runner (--mode {single,multi}, --record default on)
│   ├── build_zipapp.sh        # Build self-contained agent-redteam.pyz (release asset)
│   ├── tradeoff.py            # [v0.3] Defense-autonomy sweep + run_tradeoff_recorded()
│   └── dashboard.py           # Streamlit dashboard (4 tabs: Sim / Tradeoff / Calibration / History)
├── tests/                     # pytest suite: 92 passing (1 skipped)
│   ├── test_targets.py        # 9 — v0.2 LLM target smoke tests
│   ├── test_v03.py            # 17 — v0.3 6-task additions
│   ├── test_v04.py            # 9 — v0.4 B1 pluggable embedders
│   └── test_v04_storage.py    # 11 — v0.4 B2 SQLite storage
├── storage/                   # [v0.4 B2] Runtime DB lives here
│   └── agent_redteam.db       # (gitignored) Created on first Recorder.__enter__
├── outputs/                   # Generated logs, traces, scores
└── docs/
    └── USAGE.md               # How to run + interpret results

Quickstart

# Install (Python 3.11+)
cd agent-redteam
uv venv .venv
uv pip install -r requirements.txt

# Optional: copy .env.example to .env and fill in API keys
cp .env.example .env

# Run smoke tests (no API keys required)
python tests/test_targets.py

# Run the red-vs-blue simulation (default: mock target)
python scripts/run_redteam.py --target mock --rounds 100

# Run with real LLM (needs API key)
python scripts/run_redteam.py --target openai --model gpt-4o-mini --rounds 20 --max-cost-usd 5.00
python scripts/run_redteam.py --target anthropic --rounds 20

# Launch the Streamlit dashboard
python -m streamlit run scripts/dashboard.py

# CI regression gate (strict — exit 1 if any suite < baseline)
python scripts/ci_regression.py --strict

# Multi-Python matrix (3.10 / 3.11 / 3.12)
python scripts/ci_multi_python.py --quick

Install from release (no clone)

A self-contained zipapp is attached to each release — mock target and all 5 defense layers run fully offline inside it:

curl -L -o agent-redteam.pyz https://github.com/aidless/agent-redteam/releases/download/v0.4.0/agent-redteam.pyz
python3 agent-redteam.pyz --target mock --rounds 100 --no-record

Built by scripts/build_zipapp.sh. Verified 2026-11-10: --target mock --rounds 3 --no-record inside the pyz (run outside any checkout) executes 3 attacks through all 5 defense layers and writes outputs/redteam_results.json. Scope note: --config and data/attack_vectors.json resolve from the CWD, so for custom configs or the full 50-sample dataset run from a checkout (Quickstart above).

Limits and Scope

What this platform is: an offline, reproducible evaluation harness. The 12 attack categories, 5 defense layers, 7 aggregators, and 3 memory architectures are instrumented components you run and measure — the mock target makes every red-vs-blue round deterministic and free.

What it is not (and does not claim to be):

  • Not an offensive toolkit. Attack payloads are synthetic fixtures defined in attacks/v01..v12 and data/attack_vectors.json; the harness simulates their delivery against an in-process target. It performs no real-world exploitation, scanning, or delivery against any external system.
  • Not a deployable defense product. The 5 defense layers are evaluation instruments inside the harness. Their block/flag decisions are real code paths, but they are not hardened, audited production guards — do not deploy them as your security boundary.
  • Mock numbers do not transfer. Block rates on --target mock characterize the defense layers, not real models. openai/anthropic targets are nondeterministic, cost money, and are subject to provider-side behavior; no cross-provider equivalence is claimed.
  • Not a benchmark. There is no reference leaderboard or gold result set; all metrics (block rate, calibration ECE/JSD, tradeoff Pareto points) are relative to your own runs and seeds.
  • Single machine. Persistence is a local WAL-mode SQLite file (storage/agent_redteam.db); no distributed orchestration, no concurrency guarantees.
  • Out of scope attack surfaces: streaming/multimodal attacks, sandboxed real tool execution, and persistence attacks beyond the three simulated memory architectures are not modeled.

Relationship to Survey

Survey Section Platform Component
§8.2 Attack Taxonomy (12 vectors) attacks/ (12 categories, 50 samples in JSON)
§8.3 Defense Layers (5 layers) defenses/ (5 modules)
§8.4 Calibration as a Security Property v0.2: tracked in target_response.tokens
§8.5 Compliance Context (EU AI Act) v0.3: configs/compliance_profiles.yaml
§8.6 Compositional Attacks data/attack_vectors.json::V12_compositional
§8.7 Defense–Autonomy Tradeoff v0.2: cost / latency tracked per defense

What is proven / what is not

Proven (reproducible in this repo):

  1. 12 attack-vector classes in attacks/v01..v12_*.py round-trip through load_attack_vectors(), and data/attack_vectors.json holds exactly 50 samples whose per-vector counts sum to 50 (asserted in tests + verified by the reproduction command below).
  2. 5 defense layers (input_separation, tool_whitelist, output_filter, behavior_audit, constitutional) each implement the Defense interface and each produces a concrete action (BLOCK / MODIFY / FLAG / ALLOW) on the demo matrix.
  3. 7 aggregators collapse a VerifierOutput stream into an AggregatedDecision with label ∈ {0,1}, score ∈ [0,1], and per-verifier length matching (asserted in tests/test_v04_b6.py).
  4. Calibration metrics (ECE, JSD, entropy, CV, γ_temporal, γ) are computed from a deterministic synthetic score stream (synth_score_stream(n, seed, bias)), so they are reproducible given the seed.
  5. Defense–autonomy tradeoff is measurable: scripts/tradeoff.py sweeps defense subsets and reports Pareto-optimal points per subset.

Not proven (honest boundaries):

  1. No real-world attack success rate. Attacks are synthetic payloads in data/attack_vectors.json; the demo matrix reports what defense actions fire, not whether a real attacker succeeds against a production system.
  2. No LLM quality claims. The mock target is deterministic and free; the OpenAI / Anthropic targets are thin API adapters with no benchmark results. tests/test_targets.py only smoke-tests dry_run(); no end-to-end real-LLM evaluation is included.
  3. No production-readiness claim. The SQLite recorder and Streamlit dashboard are demonstration-grade; no load testing, crash recovery, or multi-instance concurrency guarantees are asserted.
  4. The demo matrix is layer-level over synthetic payloads. A BLOCK on layer 2 does not imply the whole defense stack blocks that attack; only the leave-one-layer-out table estimates marginal contribution.

Reproduce (numbers used above)

All counts below are recomputed from this repo; nothing is copied from memory.

# Full suite (authoritative pass count)
python3 -m pytest tests/ -q
# → 93 passed (0 skipped) when sentence-transformers installed

# CI regression gate (strict — exit 1 if any suite < baseline)
python3 scripts/ci_regression.py --strict
# → 93/93 passing across 8 suites (exit 0)

# Attack dataset counts (12 vectors, 50 samples, sums to 50)
python3 - <<'EOF'
import json
from collections import Counter
d = json.load(open("data/attack_vectors.json"))
av = d["attack_vectors"]
assert len(av) == 12
assert sum(len(v["samples"]) for v in av) == 50
sev = Counter(v["severity"] for v in av for _ in v["samples"])
cat = Counter(v["category"] for v in av for _ in v["samples"])
print("vectors:", len(av), "| samples:", sum(len(v["samples"]) for v in av))
print("by_severity:", dict(sev))
print("by_category:", dict(cat))
EOF
# → vectors: 12 | samples: 50
#   by_severity: {'high': 24, 'medium': 17, 'critical': 9}
#   by_category: {'input_attack': 19, 'tool_attack': 5, 'memory_attack': 4,
#                 'output_attack': 9, 'aggregation_attack': 5,
#                 'denial_of_service': 4, 'advanced_attack': 4}

# Demo matrix (no API key)
python3 demo/cli.py

License

Apache 2.0 — open-source companion to the TMLR survey.

Authors

Anonymous Authors (TMLR double-blind compliant). Full author list to be added in the camera-ready version.

Changelog

  • v0.4 B6 (2026-07-24): Coverage sprint + Historical Calibration tab + CI multi-Python matrix + version control init. 4 polish tasks delivered (A: coverage; B: git; C: dashboard trend; D: multi-Python matrix):
    • B6-A coverage sprint (25 tests in tests/test_v04_b6.py):
      • A1 factories (6 tests): create_target(mock/raises/overrides) + create_defenses(default/disable/none) — covers both happy and edge paths in targets/factory.py and defenses/factory.py.
      • A2 aggregators (9 tests): every one of the 7 aggregators/ classes gets a smoke test that constructs a small VerifierOutput stream and checks the AggregatedDecision invariants (label is 0/1, score in [0,1], method name, per_verifier length). Plus make_aggregator() factory round-trip for all 7 class names.
      • A3 attacks (5 tests): all 12 attack classes round-trip through load_attack_vectors(); V01/V03/V06/V10 each tested for the "no orchestrator in system_state" error path that any user- facing caller hits first.
      • A4 Recorder (3 tests): finalize() writes finished_at only when record_attack was called at least once; finalize() without any record call is a no-op (no spurious empty row); current_recorder() defaults to None outside a with context.
      • A5 MultiAgentOrchestrator (2 tests): T_rounds=0 returns an empty EpisodeResult (degenerate input); single-agent config with no verifier agent produces no AggregatedDecision (so aggregated is None, not a fake zero).
    • B6-B git version control init: project was 0-tracked under F:/Research parent repo. Now committed as standalone repo: 69 files (68 source
      • .gitattributes). .gitattributes enforces eol=lf for all tracked text files (Python / YAML / TOML / .tex / .bib / etc.) and eol=crlf for Windows-only .cmd shims. core.autocrlf=false set locally so future git add doesn't re-introduce CRLF. Eliminates the cross-platform diff noise that bit us earlier.
    • B6-C Dashboard Historical Calibration tab (5th tab):
      • C1 New storage.list_calibration_reports(source, limit) and storage.count_calibration_reports_by_source() read helpers (join calibration_reports with runs for started_at / target_* / mode context).
      • C2 6-metric trend chart (st.line_chart over ECE / JSD / entropy / CV / γ_temporal / γ) with a multi-select for which metrics to plot.
      • C3 Source filter: checkboxes for each known source tag (calibration_runner, cli_calibration_runner, dashboard_run, post_run_multi) + any source that actually has rows in the DB; counts come from a single GROUP BY query.
      • C4 "Impossible-triangle" fraction panel + raw-row dataframe
        • JSON dump (capped at 50 for perf).
      • 6 new tests in tests/test_v04_b6_calib_history.py covering list/filter/limit/groups/source_tags/dashboard-wiring.
    • B6-D CI multi-Python matrix (3.10 / 3.11 / 3.12):
      • D1 scripts/ci_multi_python.py — for each Python in the matrix, create a throw-away venv, pip-install requirements.txt
        • pytest, then run the same direct-runner suite pipeline as ci_regression.py. Outputs a Markdown matrix table; supports --py 3.12 (restrict), --quick (B3/B4/B6 suites only), --json (machine-readable).
      • D2 .github/workflows/ci_multi_python.yml — companion to ci.yml. Triggers only on workflow_dispatch (manual) + weekly Monday 06:00 UTC cron. Runs each suite per Python with fail-fast: false so every cell of the matrix is informational.
      • D3 compat fix: requirements.txt previously pinned pandas==3.0.5 and numpy==2.4.6, which do not exist on PyPI (pandas 3.x is unreleased; numpy 2.4.x is unreleased). Replaced with realistic ranges pandas>=2.2,<3 and numpy>=2.0,<3. After fix, all 3 Pythons green.
      • Local matrix result (after D3): 3.10: 9+6+25+6=46 PASS | 3.11: 46 PASS | 3.12: 46 PASS.
    • Final state:
      • pytest tests/ → 92 passed, 1 skipped (exit 0)
      • python scripts/ci_regression.py --strict → 93/93 passed across 7 suites (exit 0)
      • python scripts/ci_multi_python.py --quick → 3 Python × 4 suites × all-PASS (3.10 / 3.11 / 3.12)
      • git: git log --oneline | head shows the new feat(agent_redteam): v0.4 B5 + version control init commit (69 files).
  • v0.4 B5 (2026-07-24): Polish + requirements.txt + 4 latent-bug fixes + test_v04.py recovery.
    • P1 CI workflow indent unified to 2-space throughout.
    • P2 Dashboard import: deliberately kept inside the button handler (consistent with all other tabs — tab_tradeoff, tab_history, tab_calib metrics all use lazy imports to avoid heavy deps on reruns when the user is on a different tab).
    • P3 local pytest: pytest==9.0.2 installed into the project venv via uv pip install. Local python -m pytest tests/ --collect-only -q collects all 61 tests in 0.25s.
    • P4 requirements.txt: 7 runtime + 1 dev dep pinned (openai, anthropic, streamlit, pandas, numpy, python-dotenv, PyYAML, pytest). CI workflow now pip install -r requirements.txt instead of duplicating the list inline. Drift risk between local venv and CI is gone.
    • Latent fix #1 (skip→failure): tests/test_v04.py and tests/test_v04_b3.py runners now catch BaseException (not just Exception) and use _pytest.skip() so pytest and direct runner agree on skip semantics. 1 formerly-failing skip now correctly classified.
    • Latent fix #2 (recursion guard): ci_regression.evaluate() reads _CI_REGRESSION_INVOKED_BY_SUITE env var (set by run_suite) and short-circuits when a suite calls itself recursively via subprocess. Prevents test_v04_b3.py::test_d4_* from infinite-looping under pytest.
    • Latent fix #3 (clean-baseline logic): test_d4_evaluate_clean rewritten to monkey-patch run_suite (synthetic returns = baseline) rather than setting baseline=0 (which always regressed).
    • Latent fix #4 (recurse-skip): test_d4_evaluate_regression_detection now skips itself via _pytest.skip() when the recursion guard is active (cannot meaningfully exercise evaluate() from inside the loop that triggered it).
    • Recovery: tests/test_v04.py (9 tests + runner) was deleted by an external process between sessions. Reconstructed from git history
      • my earlier Edit content; aligned with current memory/embeddings.py API.
    • Final state:
      • pytest tests/ → 61 passed, 1 skipped in 27.47s (exit 0)
      • python scripts/ci_regression.py --strict → 62/62 passed across 6 suites (exit 0)
      • All 6 direct runners (python tests/test_*.py) → pass.
  • v0.4 B4 (2026-07-23): load_config(None) auto-resolves + CI workflow file + Dashboard calibration button. 4 tasks delivered:
    • D6 load_config auto-resolution: when path is None, now auto-loads <repo>/configs/default.yaml (pre-existing B3 hack removed). Falls back to hardcoded defaults only if that file is missing. Eliminates the need to pass config_path="configs/default.yaml" explicitly from Python API callers. Test: test_d6_load_config_none_loads_default_yaml.
    • D7 GitHub Actions CI workflow: new .github/workflows/ci.yml runs pytest tests/ -q then python scripts/ci_regression.py --strict --json. Triggers on push to main and on PRs. Strict mode so regressions fail the build.
    • D8 Dashboard "Run calibration" tab: tab_calib now has a 4-column interactive panel (seed, n, bias, Persist to SQLite checkbox) + "▶ Run calibration" button. Calls scripts.calibration_runner.run_calibration_recorded() (when checkbox on) or run_calibration() (when off) and renders the 6 metrics as st.metric cards + the notes dict in an expander.
    • D9 6 new tests in tests/test_v04_b4.py (D6 × 3: auto-resolve + explicit path + missing path; D7 × 2: workflow file exists + runs --strict; D8 × 1: dashboard button present + 6 metrics wired). Total now 61/61 (no regression). CI gate BASELINE dict bumped to include test_v04_b3.py:9 + test_v04_b4.py:6 so future regressions of new tests are caught.
  • v0.4 B3 (2026-07-23): CI regression gate + post-run calibration + --no-record flag. 5 tasks delivered:
    • D1 --no-record flag: argparse tri-state group (--record/--no-record/default=true) on scripts/run_redteam.py. Use --no-record to skip SQLite persistence (e.g. for smoke / exploratory runs).
    • D2 calibration_runner: new scripts/calibration_runner.py with synth_score_stream(n, seed, bias) (deterministic synthetic biased-vs-clean score pairs), run_calibration() returning a full CalibrationReport (6 metrics), and run_calibration_recorded() wrapper that writes a calibration_reports row. CLI: --seed/--n/--bias/--record.
    • D3 auto record_calibration after multi-mode: scripts/run_redteam.py collects per-episode aggregated verifier scores during the multi-agent loop and, when ≥ 2 episodes produce aggregated decisions, fires the post-loop calibration block (record_calibration("post_run_multi", report)). Result appears in results["calibration_report_recorded"] = True and is visible in the Historical Runs tab.
    • D4 CI regression gate: new scripts/ci_regression.py with baseline counts (test_targets.py:9, test_v03.py:17, test_v04.py:9, test_v04_storage.py:11) and --strict / --soft (default) / --update-baseline / --json modes. --soft mode emits WARNs on regressions but exits 0; --strict exits 1 on any regression. CI workflow: pytest first, then python scripts/ci_regression.py (soft) → ✅.
    • D5 tests + docs: 9 new tests in tests/test_v04_b3.py (D1 flag resolution × 3, D2 calibration runner × 3, D3 post-run calibration wiring, D4 evaluate × 2). Total now 55/55 (9 + 17 + 9 + 11 + 9, no regression).
  • v0.4 B2 (2026-07-23): SQLite trace persistence + Historical Runs tab. New storage/ package with 5-table WAL-mode SQLite schema (runs, attack_results, tradeoff_points, calibration_reports, schema_version), a Recorder context manager (enter creates the run row, exit stamps finished_at), and read helpers (list_runs, get_run_aggregates, list_attack_results, list_tradeoff_points). scripts/run_redteam.py now records one attack_results row per round/episode for both --mode single and --mode multi. scripts/tradeoff.py gains run_tradeoff_recorded() which writes one tradeoff_points row per subset with the Pareto flag. Dashboard gets a 4th tab 🗄️ Historical Runs with a run-selector, per-run aggregates, and drill-down into per-attack rows or per-tradeoff points. 11 new tests in tests/test_v04_storage.py (all 11 pass). Total now 46/46 (9 + 17 + 9 + 11, no regression). New API: from storage import Recorder, list_runs, get_run_aggregates, list_attack_results, list_tradeoff_points, DEFAULT_DB_PATH.
  • v0.4 B1 (2026-07-22): Pluggable embedders for RAG memory. New memory/embeddings.py with Embedder ABC, TokenOverlapEmbedder (v0.3 baseline, preserved), CharNgramEmbedder (pure-Python char n-gram + IDF, captures sub-word morphology), and SentenceTransformerEmbedder (real embeddings, optional). default_embedder() factory auto-falls back to char_ngram when sentence-transformers is not installed. RAGMemory.__init__ gains an embedder= kwarg; default still uses token overlap so v0.3 call sites are unchanged. 9 new tests in tests/test_v04.py (all 9 pass; 1 skip if ST not installed). v0.2 9/9 + v0.3 17/17 still pass (no regression).
  • v0.3 (2026-07-22): Multi-agent + calibration + memory + aggregators + tradeoff. Added metrics/calibration.py (6 metrics from survey §2.2), memory/ (3 architectures from §6: append-only / summarization / RAG), aggregators/ (7 implementations from §7.2), attacks/v01..v12_*.py (12 vector classes
    • loader.py), orchestrator/multi_agent.py (BOUNDARY_SYNC protocol, verifier capture, 7-aggregator pipeline), scripts/tradeoff.py (defense-autonomy sweep with GSM8K-style proxy), new tab in scripts/dashboard.py. run_redteam.py gains --mode single|multi, --memory, --aggregator flags. tests/test_v03.py adds 17 smoke tests (17/17 passing). v0.2 9/9 still passing (no regression).
  • v0.2 (2026-07-22): LLM target abstraction. Added targets/ (4 providers), orchestrator/ (single-agent), configs/default.yaml, .env.example, .gitignore, tests/test_targets.py (9 smoke tests). Updated run_redteam.py
    • dashboard.py with --target parameter and cost panel.
  • v0.1 (2026-07-22): Initial scaffold. 5 defense layers, 12 attack vectors in JSON, mock run_redteam.py, Streamlit dashboard.

验收标准与复算

验收判据:本仓判为合格的条件是 python3 eval/run_cases.py 通过率必须为 100%(20/20),且 make eval 退出码必须为 0;任何一条失败即判为不合格。

题库与 axis

eval/cases.jsonl 共 20 条七键用例(id/axis/question/expected/mode/verify/source)。18 条 expr/raises 真调本仓 API 或真解析 data/attack_vectors.json,2 条 doc 查本文档;期望值全部由当前实现现算,不是抄文档。

axis 条数 覆盖对象
metrics 3 metrics/calibration.py:ece / jsd / cv 的边界行为
dataset 1 data/attack_vectors.json:metadata 与各 vector 的 samples 自洽(50 条 / 12 类)
attacks 3 attacks/loader.py + attacks/_base_v03.py:装载、未知 id、缺 orchestrator
aggregators 3 aggregators/:MajorityVote 票型、registry 7 个、未知名字
defenses 3 input_separation / tool_whitelist / output_filter 的动作
memory 2 append-only 截断与 RAG top-1
storage 1 storage/recorder.py:临时库 run 落盘与聚合
demo 2 demo/harness.py:12×5 覆盖矩阵 summary 与 verification
docs 2 本文档的验收与归因小节

三条命令

python3 eval/run_cases.py             # 期望 20/20 cases passed,exit 0
python3 eval/run_cases.py --selftest  # 期望 19/20(第一条被篡改后必须变红),exit 0
make eval                             # 期望 rc=0(先跑题库,再跑既有 pytest 套件)

复算说明

每题 verify 字段就是复算代码:它真调本仓 API 或真解析本仓数据文件,期望值由当前实现现算;改一行实现就可能让它变红,这正是这套题的意义。python3 eval/run_cases.py --save 会重建 eval/run-report.json(逐题 PASS/FAIL 与按 axis 的 failed_by_axis 汇总),所以落盘结果随时可从题库重算,不依赖任何手工记录。

失败归因

失败不报一个总分就完事:--json / --save 里的 failed_by_axis 按 axis 汇总失败用例 id。示例(假设 C01、C13 变红):

失败归因(按 axis):
  metrics: 1 条 —— C01
  defenses: 1 条 —— C13

定位顺序:先看 axis 落在哪个模块,再读该题 verify 里的 API 调用,最后对本仓源码核对该行为是否被本意改动(验收层只报告源码问题,不改实现)。

About

Red-team harness for LLM agents: multi-attacker evaluation with an OWASP-aligned playbook and a documented fake-version-pin incident

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages