Skip to content

About

Research-backed rules, skills, and spec templates for building LLM agent systems -- distilled from arXiv papers, queries and reports.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Repository files navigation

Agentic Engineering

Research-backed rules, skills, and spec templates for building LLM agent systems -- distilled from 395+ arXiv papers and counting.

What this is: A curated knowledge base of transferable procedures, design rules, and decision frameworks for agent engineering. Every rule cites its evidence. Every skill describes a reusable procedure you can drop into your agent workflows.

What this isn't: A framework, library, or SDK. This is knowledge, not code. Load it into your AI coding assistant's context (rules, skills) or use the spec templates when starting a new project.

License: CC BY 4.0


Quick Start

For AI coding assistants (Antigravity, Claude Code, Cursor, etc.):

  1. Copy rules/ into your project's .agents/rules/ (or equivalent) -- they'll be active during every session
  2. Copy skills/ into your project's .agents/skills/ -- they'll be retrieved on-demand when relevant
  3. Fill in a specs/ template when starting a new project in a covered domain

For humans: Browse the rules for decision criteria and pitfalls, read the skills for step-by-step procedures, or start from a spec template to map out your design space.


What's Inside

Rules (Always Active)

Concise, evidence-backed guardrails. Load these so your agent applies them automatically during every coding session.

Rules File Domain
agent-sandbox-safety Sandbox design, command classification, network policy, capability gateway, defense composition, checkpoint integrity, deployment context, adversarial testing, tool output safety, GitOps span editing, harness tampering audit, blind CoT monitoring, hook security, multi-turn refusal variance, dependency-scoped plan lineage, dialogue authentication decoupling, semantic patch validation, input tree perturbation, constructive agent host obligations, guard-agent topologies, deterministic safety constraint gates, guardrail repetition instability, dynamic resource acquisition bounds, pre-execution action auditing, agent-tool boundary contracts, universal tool anomaly filtering, trajectory-level multi-turn safety, historical safety-constraint restoration, per-channel refusal testing
skill-system-design Skill authoring, selection, library management, skill evolution, training data quality, evidence triage, library integrity, hallucination detection, prefix-preserving context assembly, protocol-aware context trimming, persistent agent architecture, procedural families, operational know-how distillation, speculative macro commit, code embedding functional retrieval gap, standardized semantic envelopes
recursive-improvement Self-modification architecture, change classification, evaluation, instruction refinement, targeted reflection, verification, multi-day autonomous loops, joint harness-weight optimization, reference trajectory evolution, error-structured prompt optimization (ESPO), counterexample-guided repair, rubric artifact bias, neural invariant diagnosis, belief-calibrated scaffold optimization, swarm commons governance, debate consensus calibration, surrogate-guided bilevel rubric evolution, autonomous environment exploration and frozen causal memory, sequential stopping for self-evolution loops
adk-workflow-architecture Graph DAG orchestration, deterministic runtime gates, zero-token policy routing, default fallback edges, lifecycle callback interceptors, Memory Bank vs RAG separation, long-running tool receipts, universal resumption
adk-security-and-evaluation Model Armor prompt injection guards, Sensitive Data Protection (SDP) PII de-identification, A2A mutual agent authentication & Agent Card verification, AP2/UCP cryptographic commerce tokens, CI/CD golden dataset trajectory validation, multi-criteria LLM-as-a-Judge rubrics
agent-evaluation-quality Output evaluation, benchmarking, quality gates, pass/fail rubrics, trajectory verification
agent-human-interaction Work presentation, feedback solicitation, cognitive load management, structured reviews
multi-agent-coordination Multi-agent topology, role delegation, communication protocols, coordination failure recovery, intent-regularized bus communication, intra-family model candidate selection, protocol overhead sizing, differential sub-agent auditing

Skills (On-Demand)

Transferable, subtask-level procedures. Each skill has a SKILL.md with when-to-use criteria, a step-by-step procedure, environment caveats, and failure modes.

Agent Sandboxing & Security

Skill Source Paper
browser-agent-http-sandbox ceLLMate (arXiv:2512.12594)
speculative-sandbox-scheduler SpecBox (arXiv:2607.23933)
transactional-coding-sandbox Fault-Tolerant Sandboxing (arXiv:2512.12806)
unified-capability-gateway CrabOS (arXiv:2608.28165)
layered-defense-ensemble Layered LLM Defenses (arXiv:2608.28327)
covert-tool-injection-defense Covert Indirect Prompt Injection (arXiv:2608.30362)
skill-evolution-defense EvoSkill Injection (arXiv:2608.30429)
self-improving-red-team SIR Red-teaming (arXiv:2608.30207)
deterministic-span-editing Minimal-Diff GitOps Remediation (arXiv:2609.00227)
harness-tampering-audit Auditing Harness Tampering (arXiv:2609.00069)
dependency-scoped-plan-validation PlanFence (arXiv:2609.03340)
black-box-trajectory-risk-monitoring Web Agent Key-Step Monitoring (arXiv:2609.02057)
nlip-agent-message-envelope NLIP Agent Protocol Standard (arXiv:2609.04135)
auth-revocation-quiescence Auth Revocation for Long-Running Agents (arXiv:2609.21284)
auto-formalization-safety-guarantee MAGS Auto-Formalization Safety (arXiv:2609.19391)
residual-auth-state-preservation ResidualAuth (arXiv:2609.08062)
prime-power-federation-governance PRIMUS (arXiv:2609.07910)
taxonomy-driven-red-teaming Black-Box Red Teaming of Agentic AI (arXiv:2609.09647)
multi-agent-federation-governance PRIMUS Federation Identity (arXiv:2609.07910)
description-only-injection-detection No-Box Vulnerability Analysis (arXiv:2609.10854)
high-fanout-sandbox-memory-compression High-Fanout Sandbox Memory (arXiv:2609.11294)
adk-model-armor-interceptor Google Cloud Model Armor & SDP Interceptors
runtime-resource-authorization-bounds AcquireBound (arXiv:2609.14744)
pre-execution-action-auditing ActGuard (arXiv:2609.14987)
agentic-prompt-injection-search Test-Time Injection Search (arXiv:2609.04495)
security-context-composition CONTINUITY Security-Context Contracts (arXiv:2609.05269)
cve-history-executable-detection The History Is the Detector (arXiv:2609.05335)
macos-reverse-engineering Jonathan Levin (OS Internals) & Patrick Wardle (TAOMM)
universal-tool-defense Universal Tool Defenses (arXiv:2609.16098)
mcp-tool-hijacking-defense A2M MCP Hijacking (arXiv:2609.26761)
selection-invariant-agent-communication SICC Privacy-Aware MAS Comms (arXiv:2609.26076)
tainted-message-clean-room-recovery ESC-CR Tainted Message Recovery (arXiv:2609.26072)
mcp-zero-trust-tool-authorization Zero-Trust Authorization for Enterprise MCP (arXiv:2609.22573)
verified-policy-action-governance ActGov (arXiv:2609.24446)
closed-loop-adaptive-red-teaming CART (arXiv:2609.27336)
pentest-harness-assurance Pentest Harness Assurance (arXiv:2609.22664)
authorization-closure-repair Authorization Closure Graph (arXiv:2609.32428)
provenance-capability-grants ToolFence (arXiv:2609.37196)
vault-mediated-credential-injection API Secrets Threat Analysis (arXiv:2609.33371)
poisoned-skill-tool-call-authorization ActionGuard (arXiv:2609.39450)
evidence-guided-skill-audit SKILLLITE (arXiv:2609.36879)
rubric-first-safety-probing SCOUT (arXiv:2609.36201)
historical-constraint-restoration GHOST / STAR-Guard (arXiv:2610.02664)

Self-Improvement & Evaluation

Skill Source Paper
hyperagent-self-improvement HyperAgents (arXiv:2603.19461)
algorithmic-design-evaluation AI4AI-Bench (arXiv:2608.20318)
targeted-failure-attribution DoCtOR (arXiv:2608.28264)
behavior-aware-verification HarnessLens (arXiv:2608.27311)
agent-working-memory-eval Measure Before You Manage (arXiv:2608.31057)
reference-trajectory-harness-evolution HarnessEvolve (arXiv:2609.00829)
trajectory-aware-eval-pruning PTA-IRT Benchmarking (arXiv:2609.01603)
error-structured-prompt-optimization ESPO (arXiv:2609.04197)
counterexample-guided-repair A-CEGIS (arXiv:2609.02892)
neural-invariant-failure-diagnosis AgentScope (arXiv:2609.02371)
belief-calibrated-scaffold-optimization BCO (arXiv:2609.01861)
debate-consensus-memory-calibration R^2-MAD (arXiv:2609.03619)
reward-hacking-immunization HackProbe (arXiv:2609.04665)
intervention-guided-mas-prompt-optimization AgentGrad (arXiv:2609.08572)
temporal-workflow-graph-compilation ReActNet (arXiv:2609.05774)
offline-trajectory-tool-use-learning AgentBrew (arXiv:2609.05837)
debate-layer-disagreement-analysis Layered Disagreement Analysis (arXiv:2609.08016)
self-verification-elicitation Self-Verification via RL (arXiv:2609.08025)
static-dynamic-verification-gap-measurement Beyond Static Guarantees (arXiv:2609.10762)
bayesian-backward-disagreement-anchor Bayesian Backward Reasoning (arXiv:2609.11709)
adk-eval-golden-dataset-ci Google ADK Golden Dataset & Trajectory CI/CD
dense-rubric-skill-evolution SkillLift (arXiv:2609.15396)
agentic-review-deploy-loop AI-Native SDLC Playbook (Review, deploy, and maintain stages)
autonomous-research-to-launch-harness AutoLR Research-to-Launch Harness (arXiv:2609.04871)
autonomous-environment-exploration RSIAgent (arXiv:2609.15364)
fast-tree-search-self-improvement SIFT (arXiv:2609.19526)
recursive-self-improvement-loop AIDE² (arXiv:2609.26457)
corpus-scale-prompt-distillation CASD (arXiv:2609.26261)
world-model-trust-gating Dual-Frontier (arXiv:2609.26293)
serving-stack-eval-checklist Serving Stack Confounds (arXiv:2609.26693)
regularized-harness-evolution RRSI (arXiv:2609.24972)
staged-capability-registration MedRSI (arXiv:2609.24838)
learned-teamwork-strategy-bank Self-Organizing Agent Teams (arXiv:2609.22682)
decision-only-sequential-acceptance Reliable Self-Improvement under Benchmark Reuse (arXiv:2609.33180)
source-disjoint-proposer-feedback False Frontiers / CrossFit (arXiv:2609.39102)
execution-driven-environment-repair WEFT (arXiv:2609.36887)
pairwise-action-verification Mid-Harness (arXiv:2609.39982)
hypergraph-augmented-mcts HyperMCTS (arXiv:2609.33920)
self-evolution-stopping-rule When Is Enough Enough (arXiv:2610.04756)

Skill Evolution & Knowledge

Skill Source Paper
knowledge-compounding-loop WikiSkill (arXiv:2608.27454)
iterative-instruction-refinement NPO (arXiv:2608.27266)
skill-design-methodology Break It Down, Pass It On (arXiv:2608.20274)
capability-aware-skill-selection Optimal Skill Selection (arXiv:2608.19993)
policy-centroid-routing Policy-Centroid Routing (arXiv:2608.30757)
governed-knowledge-graph MAGG Governed KGs (arXiv:2608.28642)
agentic-data-cracking Token-Efficient Data Reasoning (arXiv:2608.31082)
attribution-guided-skill-graph-update SkillAA (arXiv:2609.20455)
controlled-skill-lifecycle-management FINSKILLOPS (arXiv:2609.19680)
semantic-aware-multi-agent-delegation SAIGE (arXiv:2609.19759)
prefix-preserving-context-assembly ContextPipe (arXiv:2609.00749)
protocol-preserving-context-trimming Protocol-Preserving Context Trimming (arXiv:2609.16461)
persistent-agent-migration Enoch Persistent Agents (arXiv:2609.00546)
requirements-driven-code-generation WiseSpec (arXiv:2609.00568)
grill-spec Multiple-choice interview → spec, ADRs, milestone roadmap, AGENTS.md for maintained projects; example by alexziskind1 + WiseSpec, ProSE
procedural-family-skill-consolidation SkillGLoW (arXiv:2609.02217)
speculative-macro-commit Speculative Macro Commit (arXiv:2609.03236)
procedural-graph-evolution Procedural Graphs (arXiv:2609.09153)
graph-of-skills-scaling SE-GoS (arXiv:2609.08228)
stable-skill-evolution SkillAdam (arXiv:2609.08944)
ledger-orchestrated-coding-loop Zero-Shot Self-Orchestration (arXiv:2608.26480)
adk2-agent-orchestration-patterns Google Cloud Tech / ADK 2 Orchestration
adk-async-long-running-tool-resumption Google ADK VibeStudio (Async Tool Resumption & Call ID Matching)
adk-lifecycle-callback-interceptors Google ADK VibeStudio (Lifecycle Callbacks & Interceptors)
adk-a2a-agent-federation Google ADK & Agent Runtime (A2A Protocol & Agent Cards)
adk-a2ui-dynamic-widgets Google ADK & A2UI (Declarative Widgets & Action Dispatch)
adk-eventarc-reactive-trigger Google Cloud Eventarc & ADK (Event-Driven Reactive Pipelines)
adk-mcp-multimodal-tool-interception Google ADK & MCP (Multimodal Tool Interceptors & Callbacks)
intent-driven-sdlc-planning AI-Native SDLC Playbook (Intent → Spec → Plan upstream pipeline)
mcp-server-design MCP Server Best Practices & Progressive Discovery
agentic-requirements-reverse-engineering Code to Requirements (arXiv:2609.22719)

Retrieval & Evidence

Skill Source Paper
rag-evidence-triage Knowing Before Answering (arXiv:2608.27661)
hallucination-mean-shift-probe Hallucination Mean Shift (arXiv:2608.28930)
rag-hallucination-repair RAG Hallucination Repair (arXiv:2608.29307)
cost-effective-repo-exploration Cost-Effective Repo Exploration (arXiv:2608.29675)
necessary-tool-evidence-path NTEP (arXiv:2609.03493)
cost-aware-hierarchical-analysis Cost-Aware Hierarchical Analysis (arXiv:2609.04820)
evidence-gated-adversarial-deliberation LabourCrew (arXiv:2609.27814)
visual-evidence-workspace VLM-in-Sandbox (arXiv:2609.24362)

Spec Templates (Project Kickoff)

Structured decision frameworks to fill in when starting a new project. Each template surfaces research-backed decision points with evidence notes.

Template Domain
agent-sandbox.spec Designing agent execution sandboxes
skill-library.spec Designing skill memory systems for LLM agents
self-improving-agent.spec Designing self-modifying agent systems
adk-agentic-workflow.spec Designing multi-agent graph workflows with Google ADK 2
adk-multi-agent-a2a.spec Designing distributed multi-agent systems with A2A Protocol
adk-enterprise-deployment-gke.spec Production containerized ADK deployment on GKE & Eventarc
adk-eval-quality-gate.spec Designing golden datasets, trajectory assertions & CI/CD gates
adk-a2ui-declarative-interface.spec Designing rich agentic interfaces with A2UI dynamic widgets

Research Briefs

Papers that provide valuable context but don't produce standalone skills:

Brief Why It's Here
ai-agency-typology Legal vs moral agency framework for AI accountability
agentic-data-quality-framework ACE lens for assessing agentic training data quality
planner-controller-decoupling Architectural pattern for real-time agent systems
agent-control-cycle-benchmark Evaluation taxonomy for iterative agent control
agentic-os-interface-design Interface principles from String and CrabOS
knowledge-conditioned-topology Dynamic multi-agent collaboration topologies
llm-agents-security-survey Comprehensive reference for agent security domains
adversarial-probe-verification Model integrity verification via adversarial probes
scaling-lrms-beyond-supervision L0–L4 ladder for scaling LRMs beyond human oversight
entropy-space-theory Information-theoretic framework for deep learning
science-sandbox-benchmark Sandboxed evaluation of AI scientific reasoning
baitbench-reward-hacking Benchmark for agent reward hacking with planted shortcuts
apiflow-dependent-workflows Agent survival on long dependent API chains
agentlogs-cloud-agent-traces Real-world traces from GitHub's cloud coding agent
knowledge-gated-agent-tasks Separating knowledge gaps from execution gaps
ideation-arena-research-eval Elo-style evaluation of LLM-generated research ideas
content-ecosystem-ranking-effects How ranking optimization degrades content ecosystems
ai-text-detection-token-filtering When token filtering helps/fails AI text detection
collapsed-sequence-score-recovery Recovering evidence from collapsed LLM sequence scores
hidden-state-divergence-tracking Detecting reasoning drift via hidden-state trajectories
multi-solver-disagreement-rewards Disagreement-based rewards for self-evolving curricula
judge-panel-deliberation Multi-reward panel deliberation for compact LLM judges
deployment-dependent-safety Safety behavior changes with deployment context
rollback-attack-continuity Breaking agent execution via checkpoint manipulation
mcts-coding-agent MCTS-based action selection for coding agents
ontology-learning-scale When bigger models help (and don't) for ontology learning
searchwiki-active-seeking Wiki-structured knowledge building during information seeking
test-time-prompt-calibration Calibration-aware test-time prompt tuning
harness-of-harness-autonomous-development Multi-day autonomous development with continual improvement
agent-factory-workflow-optimization Automated optimization of multi-agent topologies and models
checklist-aggregation-eval-pipeline Decomposed checklist aggregation for reliable LLM evals
drift-aware-llm-routing Routing across model portfolios under shared budgets and drift
quantum-federated-learning-aggregation Stable aggregation for quantum neural network parameters
assistant-ideal-self-concept Post-training self-concept elicitation and value stability
cot-monitoring-answer-bias Measuring bias when oversight monitors have answer access
recursive-self-improvement-criticality Criticality thresholds in dynamical self-improvement systems
verbal-reinforcement-learning-taxonomy Taxonomy of natural language feedback across agent lifecycles
byzantine-placement-decentralized-fl Adversarial node placement in decentralized federated learning
embodied-vla-skill-orchestration Proposal-validation-recovery runtime for embodied VLA agents
predictive-coding-boundary-optimization Boundary-first inference schedules in predictive coding
rank-heterogeneous-federated-lora Federated LoRA under client rank heterogeneity
activation-matched-finetuning-detection Unsupervised dormant backdoor detection via activation residuals
joint-harness-weight-optimization Alternating harness search and weight updates (WHALE)
workload-aware-column-imprint-joins Real-time edge query processing with workload-aware column imprints
attention-sensitivity-dissociation Dissociating attention proxies from behavioral in-context learning
hookpry-lifecycle-hook-vulnerabilities Exploiting and sandboxing agent lifecycle hook update vectors
repo-to-skill-distillation Distilling GitHub repos into operational skills for autonomous agents
rubric-artifact-bias-in-llm-judges Demonstrating rubric-only prediction and counterfactual judge failures
door-in-the-face-model-refusals Sequential request retreat effects across model provider families
reliable-enterprise-agent-deployment Scale AI framework for enterprise agent reliability and guardrails
flowbalance-verifier-grounded-self-improvement Verifier-grounded trajectory balance preventing mode collapse
potential-guided-policy-optimization Anchor-state potential differences for multi-turn agent credit
conflict-driven-model-merging Resolving parameter conflicts via preference optimization on merge defects
evaluability-aware-assistance Balancing proposal acceptance against latent preference learning
apex-procedural-experience-distillation Hierarchical experience and test-time RL for deep research agents
post-training-ternarization-qwen3 Capability retention and kernel overheads of 1.58-bit model conversion
growpage-dynamic-kv-budgeting On-demand dynamic page budgeting for long-output reasoning serving
paper-code-discrepancy-detection Dual-detection multi-agent verification for paper-code discrepancies
hybrid-micro-level-agent-personalization Prompt conditioning on Bloom cognitive complexity and learner profiles
civilization-framework-sovereign-agent-communication Sovereign-anchored asynchronous store-and-forward agent collaboration
reflect-sql-multi-stage-reflection Multi-stage decoupled reflection loops for enterprise text-to-SQL
patchbench-vulnerability-patching-evaluation Diagnostic benchmark revealing 1.83× inflation in PoC-only vulnerability repair evals
swarm-emergent-cheating-and-whistleblowing Ostrom commons governance and spontaneous whistleblowing in 100-agent research swarms
conversational-false-authentication Identifying model-issued pseudo-credentials (MIPC) and conversational false authentication
alcatraz-anchored-tree-rule-defense Rule-tree input perturbation achieving superior security across 33 open-weight models
environment-evolution-terminal-agents Off-policy difficulty scaling along three axes yielding +14.4 to +18.0 pts on Terminal-Bench 2.1
dalek-constructive-agent-machine Von Neumann constructive machine architecture and host contracts for self-evolving agents
value-preserving-agentic-architectures Architectural topologies and dedicated guard-agent patterns for value-preserving MAS
safety-constrained-evaluation-protocol FLY-EVAL++ protocol revealing 28-point safety divergence under identical prediction accuracy
multilingual-persuasive-jailbreak-evaluation IndicSafeEval benchmark examining multilingual transfer and persuasive framing vulnerabilities
creative-ai-model-governance Closing the governance gap between data storage, circulation, and federated learning
execretrieval-code-embedding-functional-gap Diagnostic benchmark revealing dense code retrievers prefer near-clone buggy code 91.5–99.4% of the time
collective-preference-inference-evaluation Collective-centric evaluation of preference inference preserving consensus and minority voice
when-llm-decompilers-recompile-more-and-prese Behavioral divergence in LLM decompilers despite high recompilability
refuse-without-refusal-a-structural-analysis Rationale-only safety tuning reduces false refusals without harming safety
repeat-after-me-black-box-adaptive-visual-pro Black-box visual prompt injection achieving 80%+ ASR on open-weight VLMs
dcfa-dual-view-causal-inspired-attribution-fo Dual-view causal dependency graphs for multi-agent failure attribution
how-a-chatbots-response-style-shapes-a-classr Multi-agent classroom simulation of chatbot response style effects
mabpd-multi-agent-bias-probing-detection-via Training-free multi-agent bias detection via structured argument debate
building-a-research-software-catalog-with-a-c Silent failures in coding-agent-built research software catalogs
simulated-deliberation-representation-challen Challenges in AI-simulated democratic deliberation representation
style-over-substance-safety-judge-wrappers Content-invariant wrappers flipping LLM safety-judge verdicts
multimodal-resource-exhaustion-vlm-attacks Cross-modal resource-exhaustion attacks on vision-language models
pipeline-dependent-cybersecurity-benchmarks Pipeline choices swinging cybersecurity benchmark scores by 80+ points
autofyn-non-parametric-expert-iteration Non-parametric expert iteration updating persistent state, not weights
gvs5h-zero-shot-self-orchestration Zero-shot ledger scaffold matching frontier models via test execution
ai-paper-review-arms-race Adversarial co-evolution dynamics in AI scholarly publishing
threat-model-coverage-gap-safety-eval Threat-model coverage gap in automated vs human safety evaluation
cs-guard-guardrail-benchmark Guardrail weakness for code generation security
watermarks-without-verification AI text watermarking gap under EU AI Act
active-adaptation-preventative-steering Temporal dynamics of active safety adaptation
residual-auth-revocable-delegation Authorization state preservation under delegation revocation
proof-carrying-cognition Reality-settled verification gap framework
arbitrary-cipher-attacks Cipher-encoded jailbreaks without fine-tuning
multimodal-prompt-injection-eval Cross-modal prompt injection on agentic frameworks
self-evolving-consistency-gap Goal drift in long-horizon self-evolving agents
reliable-self-evolution-two-gate Two-Gate theory of reliable harness self-evolution: allowed change on solved tasks, admission threshold
agentxploit-defensive-lessons Defensive lessons from repository-to-runtime agent audits: validate what you execute, bind credentials to destinations
tool-mediated-refusal-gap Refusal drops 15.5–84.4 points when the same request comes as a tool call
agenttell-behavioural-side-channel Browser agents leak carried secrets through their choices despite instructions
cosec-community-authorization Multi-community authorization depends on the harness, not just the model
pictionary-render-untrusted-content Rendering untrusted text as images cuts injection success, per model
codel-latent-injection-coevolution Latent injections break explicit-trained defenses; static suites saturate
countersteer-tool-result-steering Activation steering on tool results; parameter manipulation still gets through
fragmented-injection-defensive-lessons Agents reconstruct injections split across fragments: defensive lessons
jailbreak-context-safety-routing Jailbreak context lingers and diverts safety routing in tool agents
gradient-jailbreak-detector-multiturn Gradient jailbreak detectors calibrated on synthetic benign traffic fail on real chats
skill-scanner-evasion-defensive-lessons Malicious-skill scanners fall to iterative evasion: defensive lessons
malicious-retriever-backdoor-defensive-lessons Backdoored retriever checkpoints in agentic search: defensive lessons
semantic-cache-poisoning-defense Embedding similarity is not answer validity for semantic caches
codemimicry-defensive-lessons Safety alignment lags in the code domain: defensive lessons
unsanctioned-supply-chain-eval-defensive-lessons In-prompt scope cut severe unsanctioned actions but not to zero
redherring-verification-decoys Unreachable decoys exploit the cost of verification in vulnerability-hunting agents
rag-vulnerability-detection-dissection Component attribution for retrieval-augmented vulnerability detectors
secprobe-adaptive-security-eval Adaptive evaluation of vulnerability repair; test-passing patches can be incomplete
madbench-debate-security Debate absorbs wrong answers but amplifies unauthorized actions
minirep-reputation-debate-aggregation Reputation-weighted debate aggregation under adversarial debaters
er-audit-debate-distillation Aggregate accuracy hides local reliability loss in debate-trained verifiers
cheap-verifier-rl-rewards Cheap open-weight reward verifiers come within points of the best for RL
coevo-policy-aware-context-evolution Co-evolving prompts and weights beats fixed-prompt RL
r2flow-recursive-skill-evolution Evidence-gated recursive skill evolution
collabflow-evidence-gated-collaboration Evidence-gated messages in agent teams
tocomas-topological-coherence Topological coherence for self-evolving multi-agent systems
gameboyworlds-self-improvement-testbed Self-improvement methods fail without expert guidance; contamination masks it
smart-self-evolving-subtitle-translation Evolve prompts on part of a long job, then freeze
frontier-autolab-org-simulation-pitfalls Hindsight leakage and uncalibrated critics in simulated agent firms
multi-agent-preference-learning Preference fine-tuning of small agent teams; a stronger comparator hurt
heterofold-cross-family-kv-transfer Cross-family KV transfer is fast but loses accuracy vs text
htn-mcp-coordination-layer HTN planning as a coordination layer for multi-server MCP
cua-sandbox-state-capsules Shared runtimes with private state capsules for computer-use RL
robust-sgpo-harness-evolution Search-space control for agent harness evolution
verifier-survey-no-free-checker Coverage-cost-soundness trade-offs in policy verifiers
agent-audit-lifecycle-trust Full-lifecycle trust evaluation framework
inference-time-governance-taxonomy Feasibility taxonomy for inference-time AI governance
sae-scientist-bench Autonomous SAE interpretability research benchmark
trace-causal-exploration Synthesized rewards for causal reasoning agents
driftnet-prompt-injection-detection Dual-head trajectory transformer for injection detection
terminal-agent-rl-long-horizon RL for terminal agents on long-horizon tasks
scaffolding-mas-clinical-training Multi-agent scaffolding for clinical interview training
debate-to-skill-process-supervision Capability-bound process supervision via debate
semverbench-version-constraint LLM semantic versioning comprehension benchmark
orch-collective-intelligence Organizational principles for embodied AI collective intelligence
opendiscoverytrace-ai-scientist Process traces for evaluating AI scientist workflows
data-efficient-language-modeling Survey of data efficiency techniques for LM training
adk2-orchestration-three-pillars Architectural pillars and decision matrix for ADK 2 graph, collaborative, and dynamic workflows
overflip-guardrail-repetition-instability Repetition-induced label flips in guardrail models
agent-tool-boundary-anomalies Structural failures and semantic divergence when tool calls succeed
a-structured-debate-mixture-of-agents-framewo Structured debate MoA framework for clinical diagnostic support
a-verifier-guided-explainable-reasoning-frame Verifier-guided explainable reasoning with gold-anchored QLoRA and RLVR
aria---an-agentic-framework-for-autonomous-te Autonomous agentic testing framework for infotainment systems
cabal-multi-agent-simulacra-for-tracing-the-e Multi-agent simulacra modeling collusive bidding cartels in peer review
from-language-models-to-world-acting-systems Progress and boundary limits of world-acting agentic AI
rise-recursive-improvement-via-self-extrapola Recursive improvement via self-extrapolating policy distillation
blindspot-long-horizon-agent-safety Trajectory-level safety benchmark evaluating delayed emergence of failures across 2,500+ multi-turn runs
sciencebuddy-recursive-harness-evolution Recursive-in-recursive self-improvement coupling inner harness evolution with outer model RL
algoevo-agentic-algorithm-discovery Self-evolving agentic search with design skill hubs and hierarchical experience trees
busma-multi-agent-bus-substrate Shared bus communication substrate with 4 explicit communicative intents and chair convergence
stellar-colosseum-agent-harness Many-agent harness for long-horizon mathematical proofs integrated into Antigravity Teamwork
evoontology-self-evolving-ontology MCP-encapsulated ontology layer with attribution-guided typed edits for data agents
world-model-science-metastable-dynamics Dynamical diagnostics, error avalanches, and metastable belief basins in long-horizon agents
mas-model-pool-selection Empirical model pool selection showing intra-family candidate pools outperform heterogeneous mixes
thinkflow-latent-conversational-memory Self-evolving probabilistic continuous latent memory overcoming explicit text bottlenecks
pentestchain-cost-aware-mcp-pentesting Cost-aware, MCP-orchestrated penetration testing cascade with local SLM and MCP threat model
mist-mid-training-cybersecurity-llms Domain adaptation via synthetic mid-training flows preserving general reasoning
trusting-trust-revisited-poisoned-benchmarks Thompson's "Trusting Trust" applied to self-modifying coding agents via poisoned benchmarks
red-teaming-auto-mode-blocking-monitors Red-teaming production pre-execution action blocking monitors (Auto Mode in Claude Code, Codex Guardian)
inference-engine-fingerprinting-and-escape Practical inference engine fingerprinting and to-the-bare-metal sandbox escapes via output tokens
contagion-multi-agent-trading-systems Adversarial signal propagation and topology damping across multi-agent financial stacks
he-guardrail-encrypted-jailbreak-defense HE-based guardrail feasibility for encrypted LLM inference
sol-pi-recursive-harness-scaling Recursive self-improvement at the harness layer with 44-49% token reduction
harness-value-planning-vs-verification Quantifying planning vs verification value in agent harnesses
vehicle-voice-command-authorization Safety-critical LLM authorization benchmark with 2-3 False Executes per 161 scenarios
foundation-model-operating-system Vision for FM virtualization analogous to VM abstraction
closed-world-tool-hallucination 5-class tool hallucination taxonomy with MCP-specific attack surfaces
micro-collaborative-rag-poisoning Distributed false claims across multiple retrieved documents
llm-group-consensus-overstatement Agent groups 34-44pp more consensual than humans, mostly on wrong answers
empirical-harness-design-study 176-setting component-level harness comparison across 4 models
ddpo-jailbreak-defense Dynamic deep prompt optimization for input-adaptive jailbreak defense
metrics-failure-vuln-repair Standard metrics fail to capture repair quality in vulnerability repair
coding-agents-kernel-exploits Evaluating coding agent capability on kernel exploit generation
emergent-collusion-long-horizon 94% collusion rate in long-horizon multi-agent peer verification
silent-sabotage-state-triggered-backdoors Internal state triggered backdoor attacks on LLM-powered robotic systems
vacs-value-aligned-shielding Four-layer value-aligned compositional shielding for multi-agent reasoning
zerogate-trust-fast-paths Trust-preserving fast paths with ActionPass revalidation contracts
akasicmem-governed-enterprise-memory Authorization continuity via transitive lineage in enterprise agent memory
harness-zero-distillation Harness distillation via agent-as-harness into model weights
growing-harness-specialist-agents Failure-guided harness growth reducing LLM calls by 76-92%
indirect-tipping-social-attack Stepping-stone equilibria as social attack surface in agent populations
contrastive-epistemic-decoding Zero-shot sycophancy mitigation via dual forward-pass conformity isolation
monitor-jailbreaking-cot-evasion RL against a CoT monitor teaches readable-but-benign-scored reasoning; paraphrasing defense
instrumental-monitor-evasion Agents route around blocking action monitors under ordinary task pressure (up to 88%)
memory-defense-benign-cost Benign-case cost of memory-poisoning defenses; read-time reranking quarantines 33.6%
encoded-prompt-benign-arm Encoded-prompt refusal evals need a matched benign arm; refusing everything looks safe
ajar-open-privilege-measurement Open privilege as a third axis for tool-call defenses beyond ASR and utility
xai-guided-guardrail-evasion Confidence-guided substitutions flip compact prompt-injection classifiers
multi-agent-prompt-injection-defense-architecture 14 multi-agent injection vectors mapped to a 4-layer defense
adversarial-influence-scaling-multi-agent Adversarial influence scales with deceiver share, not group size
wisdom-deliberative-crowds Deliberation gains come from model diversity; clone groups show none
process-level-self-evolution-evaluation EvoPathBench: acceptance gates trade harmful commits against missed improvements
evolutionary-safety-rsi-taxonomy Taxonomy of RSI risks; updater, evaluator and security tests stay outside the editable surface
role-scoped-progressive-skill-discovery Tool hiding and role prompts are not access control; server-side limits are
kernel-level-agent-preemption Syscall-level containment and cgroup freeze for rogue agent execution
sdc-mcp-dry-run-gateway Read-only plus dry-run MCP gateway pattern for safety-critical domains
mcp-granite-tool-granularity Tool granularity benchmark for MCP agents; monolithic tools hurt
mobilecybench-executable-probes Executable probes for agent vulnerability discovery; refusal cost in authorized pentests
counterfactual-multi-turn-safety-attribution Counterfactually anchored attribution of multi-turn safety failures
multi-turn-consistency-survival-analysis Survival analysis and failure-rationale taxonomy for multi-turn consistency
selfop-security-skill-optimization Self-optimizing security agent; batch failure clustering without a regression gate
delay-gratification-survival-benchmark Mandatory per-step self-questioning worsened long-horizon self-control
reasoning-topology-cybersecurity Graph-structured reasoning beats tree and chain for security classification (cost unmeasured)
nlip-a2a-hop-latency-benchmark Per-hop protocol and client overhead for NLIP vs A2A
embodied-agent-guardrail-benchmark No embodied guardrail wins on both bypass rate and benign task completion
sentinel-intent-matching-jailbreak-defense Intention-aware input-output matching kept HarmBench jailbreak ASR under 5%
emotion-induced-over-refusal Emotional framing drives benign over-refusal; mitigation without raising harmful compliance
topology-conditioned-backdoors Defensive lessons: backdoors that fire only in multi-agent framings
mlcommons-jailbreak-benchmark-v1 Standardized jailbreak benchmark; unsafe rate 11.08% to 18.65% under attack
safe-role-internalization-alignment Safe-role fine-tuning beat token-matched SFT on attack success for two of three models
refusal-steering-dimensionality Safety refusal steers along one direction; general refusal does not
visual-grounding-safety-gap VLMs refuse in text but still ground harmful targets
sigma-self-improving-alignment Self-improving alignment from a model spec; transfers to AgentHarm and agentic misalignment
subgoal-transition-authorization Runtime authorization of self-generated subgoals blocked drift with no benign loss
moe-safety-sensitive-experts Router-gradient selection finds the MoE experts that carry refusal
recursive-harness-data-synthesis Recursive harness that makes synthesized reasoning tasks harder each round

How This Was Made

Every artifact traces back to a specific paper and was produced through a systematic pipeline:

  1. Papers ingested and indexed via Praxis
  2. Each paper analyzed through the skill-design-methodology checklist
  3. Papers with transferable subtask-level procedures became skills
  4. Decision criteria and quantitative pitfalls became rules
  5. Design spaces with multiple valid configurations became spec templates
  6. Papers without actionable procedures became research briefs

The selection criteria: a paper produces a skill only if it describes a procedure that (a) operates at the subtask level, (b) could appear in 3+ different project types, and (c) is written as natural language, not executable code.


Contributing

Found a paper that should be here? Open an issue with the arXiv link and a brief note on what transferable procedure you see in it.

Changes follow the standard in AGENTS.md: numbers checked against the paper text, a Scope: line and a source on every rule entry, and linked Tension with notes where two entries point different ways. Before opening a PR:

python scripts/kb_lint.py --base origin/master

CI runs the same check on every pull request. It catches broken links, encoding damage, leftover template text, rule entries without a Scope or Source line, cross-references to headings that don't exist, and figures that look copied between papers. It also warns about new rule entries that point the opposite way to an existing one. Contradictions phrased in different words still need a periodic read-through audit; see the deconfliction-report-*.md files.


License

This work is licensed under CC BY 4.0.

About

Research-backed rules, skills, and spec templates for building LLM agent systems -- distilled from arXiv papers, queries and reports.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages