Agentic Evidence is an OWASP open-source reference project exploring how to collect, correlate, and evaluate security evidence for agentic AI execution.
Agent observability can show that an event was logged. A security investigation needs to establish what happened, which identities and permissions were involved, what inputs influenced the action, which policy decisions were made, and what conclusions the available evidence actually supports.
Agentic Evidence focuses on evidence sufficiency, execution reconstruction, and security investigation. It complements agent security taxonomies and testing projects rather than duplicating them.
Project status: The project is in incubation, and the initial implementation is being planned. The architecture and commands described in this document represent the intended MVP and may change as implementation decisions are finalized.
- Reconstruct agent executions from correlated execution and security events.
- Connect actions to identities, inputs, retrieved content, tool calls, authorization decisions, and outcomes.
- Make evidence provenance and evidence gaps visible to investigators.
- Demonstrate what can and cannot be concluded from collected evidence.
- Provide deterministic, repeatable scenarios that contributors can run locally.
- Publish an extensible evidence taxonomy and practical guidance for security practitioners.
- Benign execution: Establish a baseline for a normal agent workflow.
- Retrieval poisoning: Demonstrate how retrieved content can influence an agent action and whether the evidence captures that relationship.
- Tool descriptor modification: Evaluate whether the evidence can establish which tool descriptor was observed or used and whether modifications are detectable.
Scenarios should be deterministic and use synthetic data. The project must not claim that an attack is detectable unless the implementation and tests support that claim.
The first release is planned as a local reference application featuring:
- A deterministic scenario runner.
- A versioned JSON evidence schema.
- OpenTelemetry execution telemetry and correlation.
- An evidence recorder and local evidence store.
- A security investigation interface with execution timelines, event details, and evidence gaps.
- Exportable evidence bundles.
- Automated tests, Docker Compose configuration, and contributor documentation.
| Component | Proposed Technology |
|---|---|
| Application/API | Python with FastAPI, or Go where appropriate |
| Evidence format | Versioned JSON with JSON Schema |
| Telemetry | OpenTelemetry SDK and OTLP |
| Local storage | SQLite and/or JSON fixtures |
| Investigation UI | Streamlit or an equivalent lightweight UI |
| Local deployment | Docker Compose |
| Testing | pytest and schema validation |
| Continuous integration | GitHub Actions |
These are proposed implementation choices, not a guarantee that every component is available. Architecture decisions should be documented in the repository and changed only when there is a clear technical justification.
The initial release is not intended to be:
- A production SIEM or general-purpose observability platform.
- An agent runtime, model provider, or autonomous response engine.
- A guarantee that an agent is safe or that recorded evidence is truthful.
- A replacement for authorization enforcement, runtime security controls, or incident response.
- Dependent on a live LLM, Kubernetes, or cloud infrastructure.
Telemetry is not proof by itself. Events may be missing, incorrectly correlated, or generated by a compromised component. The application must distinguish observed records, component assertions, independently verified evidence, and unknowns.
- Run a deterministic scenario.
- Generate execution and security events.
- Correlate events with an agent run and trace context.
- Validate event structures and record provenance.
- Inspect execution timelines, authorization decisions, outcomes, and evidence gaps.
- Export the evidence bundle and compare it with expected scenario findings.
- Getting Started: Local setup and first-run workflow.
- Resources: Standards, specifications, and related security projects.
- FAQ: Project scope, evidence trust, telemetry limitations, and contribution questions.
- Evidence before claims: Findings must be supported by explicit evidence.
- Make uncertainty visible: Missing evidence is a result, not a reason to invent a conclusion.
- Repeatability: Scenarios must have deterministic inputs and testable expected outcomes.
- Least data: Use synthetic inputs by default and avoid collecting secrets or unnecessary personal data.
- Interoperability: Prefer documented schemas and established telemetry conventions where appropriate.
- Security by design: Treat evidence ingestion, storage, exports, and the investigation interface as security-sensitive components.
Contributions are welcome in the following areas:
- Evidence schema design and validation.
- Agent security scenarios and reproducible test cases.
- Telemetry collection and event correlation.
- Evidence provenance, integrity, and analysis.
- Investigation interface improvements.
- Automated testing and documentation.
Before starting substantial work:
- Review existing issues and the project roadmap.
- Open or comment on an issue describing the proposed change.
- Keep pull requests focused and include tests or reproducible evidence where applicable.
- Never submit secrets, real customer data, or sensitive production traces.
Follow the repository's contribution guidelines for current requirements.
The project is intended to use the Apache License 2.0, subject to the license file published in the repository. Consult the repository's LICENSE file for the authoritative terms.
Agentic Evidence is an OWASP project. Refer to the official OWASP project listing and repository metadata for current leadership, contact details, and project status. References to third-party tools and services do not imply OWASP endorsement.