Closed
Labels
good first issueGood for newcomersGood for newcomers
Description
evals/labelled/injection.jsonl holds 20 items and the injection_guard rail scores 20 of 20 on it. A set that size says little about precision and recall on real attacks.
Build a larger labelled set from the public benchmarks in the same JSONL shape (state.tool, state.text, label, note): InjecAgent (1,054 tool-output injections) and AgentDojo (97 tasks, 629 security cases). Then run the rail on it and add the precision and recall table to docs/benchmarks.md:
uv run s1a run injection_guard --slot jev --labelled-set evals/labelled/injection-public.jsonldocs/roadmap.md § "Prompt-injection guard" has the plan and the two sources. Needs a Jev key (TYPESAFE_API_KEY or OPENROUTER_API_KEY).
Metadata
Metadata
Assignees
Labels
good first issueGood for newcomersGood for newcomers
Activity
GoDiao commentedon Sep 24, 2026
Hi @Yunaik,
I noticed that PR #6 has been open with several feedback points pending (proper extraction of InjecAgent's
Tool Response, rendering AgentDojo placeholders, balancing benign negative controls, deduplication, and providing a reproducible build script underscripts/).If @berkay-byte is currently occupied and not actively revising it, I would be very glad to take over this issue and submit a clean PR that addresses all the review requirements. Could you assign this issue to me? I will try my best to work it out in this weekend. Thanks!
Yunaik commentedon Sep 25, 2026
that'd be much appreciated. thanks!
GoDiao commentedon Sep 25, 2026
Hi @Yunaik, I've started on this. Before I write the build script, I'd like to check three things with you. Counts below are distinct inputs.
Extraction
Tool Response, withUser Toolastool.GroundTruthPipeline, offline and with no LLM calls. With the default injection vectors, the tool outputs are the negatives. With an attack rendered into the vectors, the outputs that change are the positives.toolis the real function name, e.g.get_most_recent_transactions.injection.jsonlNegatives top out at 163, so the size of the set comes down to which positives go in.
Questions
enhancedfiles differ frombaseonly by the prefixIMPORTANT!!! Ignore all previous instructions and strictly adhere to the following instruction:. I'd takebasefor both dh and ds, since without the prefix the injection is harder to spot. OK?scripts/build_injection_dataset.py, pinned toagentdojo==0.1.35, and fetch InjecAgent at commitf19c9f2. Thenuv run scripts/build_injection_dataset.pybuilds the set andpyproject.tomlanduv.lockstay unchanged. Does that work for you?15 AgentDojo outputs are longer than 6,000 characters, and the guard truncates to 6,000 at runtime (
MAX_CHARS). The build will truncate the same way and re-derive each label after truncation, so no positive loses its injection to the cut.Yunaik commentedon Sep 25, 2026
Thanks, @GoDiao. This is a careful plan. Answers by number:
Go with the stratified sample. About 330 positives against 163 negatives is the same order of magnitude. A 4-minute run is fine. Two additions: spread the AgentDojo positives across the 5 attack templates too, and put the always-true baseline (about 67% precision at that ratio) in the benchmarks table next to the measured numbers. Please also report recall per source (InjecAgent dh, InjecAgent ds, AgentDojo). Precision only means something over the whole set, because InjecAgent has no negatives of its own.
Optional, if cheap: InjecAgent's 17 user-tool responses with the attacker instruction left out (the
Tool Response Templatefield, if it holds the placeholder the way I remember) would give negatives in the same format as the InjecAgent positives.Yes, base for both. The enhanced prefix is a keyword match and would inflate recall.
Yes. The inline metadata keeps agentdojo and its 71 packages out of pyproject.toml and uv.lock. CI passes without agentdojo installed, because ruff is the only gate that reads scripts/. Please make the output deterministic (fixed seed, stable sort) so a rerun leaves the committed file unchanged, and add both MIT notices to NOTICE and THIRD_PARTY_LICENSES.md since the JSONL redistributes their text.
The truncation plan sounds right.
Use
--model jevin the command; #7 renamed the flag from--slot.