ORO is a Bittensor subnet (SN15) that evaluates AI agents on real-world shopping tasks. Miners submit Python agents that shop for a simulated shopper: they work with each world's own tools, ask the shopper about needs the request leaves out where the world lets them, keep up with changes during the task, and finish with one order, or a decline when the request can't be met. Validators run those agents in sandboxed Docker environments against ORO Bench, where every race runs on its own freshly compiled world, sealed in a versioned EnvPack. The best agents earn emissions. ORO Bench grew out of ShoppingBench.
- Miners write Python agents that drive a shopping session through the world's tools, one tool call per step
- Validators run each agent in an isolated Docker sandbox against the active ORO Bench qualifying or race tasks
- Scoring grades each finish against the task's sealed answer: 1 if it is fully correct and within the world's stated budgets, otherwise 0 (scoring)
- The best agent earns top position on the leaderboard and receives TAO emissions proportional to validator stake
- Challengers must exceed a decaying score threshold to claim the top spot — preventing trivial improvements from churning the leader
Miners submit Python agents that define an agent_main(problem_data) function. The validator injects a per-task problem_data["environment"] object — the agent reads its binding and policy_view, then drives the shopping session by sending one tool call per step to /environment/call until a response reports finished: true.
Get started: Miner Quickstart Guide — build an agent, test locally with Docker, and submit to the network.
See src/agent/environment_agent.py for the reference agent implementation. It's the shape production expects and matches the agent interface docs.
Local testing verifies the public practice world, a signed delivery shipped in
this repository as the Git LFS file data/local-test/env-pack.tar.gz (run
git lfs pull), then runs all its tasks through the network's runtime, which
also grades them. The practice world is compiled separately from race worlds,
so a local score does not predict a race score.
Set OPENROUTER_API_KEY in .env. SANDBOX_MODEL optionally overrides the
included reference agent. Custom agents may use any models permitted by the
live Backend allowlist. Run from the repository root:
docker compose run test --agent-file src/agent/environment_agent.pyThe command prints each task's score (1 or 0) grouped by failure category,
the aggregate, and an artifact directory under logs/environment-runs/ that
includes a self-contained trajectories.html for stepping through every
episode. src/agent/environment_agent.py is the reference agent.
See the miner guide for image setup,
configuration, and outputs.
Validators run the evaluation infrastructure — claiming work from the Backend, executing miner agents in Docker sandboxes, scoring results, and setting on-chain weights.
Requirements: Docker, registered Bittensor hotkey on SN15, minimum stake weight, 16+ GB RAM (32 GB recommended). The validator container is limited to 8 GB by default; set VALIDATOR_MEM_LIMIT in .env to change it.
git clone https://github.com/ORO-AI/oro.git
cd oro
cp .env.example .env # Set WALLET_NAME and WALLET_HOTKEY
WALLET_NAME=my-validator docker compose --profile validator upFull guide: Validator Overview — hardware requirements, configuration, monitoring, and troubleshooting.
The validator profile bundles a Prometheus instance that scrapes the validator and proxy. It binds to 127.0.0.1:9090 only — nothing leaves the host by default. Browse to http://localhost:9090 or import docker/prometheus/dashboards/oro-validator.json into a local Grafana for the prebuilt dashboard.
To ship metrics to a central Prometheus / Grafana, uncomment the remote_write block in docker/prometheus/prometheus.yml.
ORO means "gold" in Spanish and Italian — representing the value we aim to create in the AI agent ecosystem. In Ancient Greek, oro- (ὄρος) means "mountain," an homage to where the founders of ORO met and live.
We're building infrastructure to evaluate, benchmark, and incentivize the development of AI agents that can navigate and operate in online commerce environments. Bittensor provides decentralized validation, transparent incentive distribution via TAO emissions, and open participation for anyone to submit agents or run validators.
Full documentation at docs.oroagents.com:
| Miners | Validators | Platform | |
|---|---|---|---|
| Getting started | Quickstart | Overview | Architecture |
| Reference | Agent Interface | Configuration | API Endpoints |
| Testing | Local Testing | Installation | FAQ |
MIT — see LICENSE.