Add an RL evaluation example

CONTRIBUTING.md

@@ -22,7 +22,7 @@ Every subject has one owner. Most owners are the page whose title names the subj
 | Website: landing page, bilingual documentation maintenance, documentation site build and GitHub Pages publication | [Website guide](website/README.md) |
 | Installer lifecycle, locking, generated state and downloads | [Deployment](deploy/README.md) and [Node installer](deploy/node/README.md) |
 | Web page behavior and visual rules | [Web product](apps/web/PRODUCT.md), [Web design](apps/web/DESIGN.md) |
-| Application example behavior and local operation | [Application example](example/parsar/README.md) |
+| Application example behavior and local operation | [Parsar](example/parsar/README.md), [RL evaluation](example/rl-eval/README.md) |
 
 ## Repository boundary
 
@@ -39,6 +39,8 @@ Core builds, deploys and runs independently of product services, frontends and d
 - Product resources use `/app/` and never become Core API or database conventions.
 - It is excluded from Core distributions and cannot become a service dependency.
 
+`example/rl-eval/` is an optional script that runs batch rollouts for RL and evaluation. Its [README](example/rl-eval/README.md) owns its behavior. It calls only public `/v1` APIs, holds only a Project key, uses only Core-managed hosted environments and writes results only to a local JSONL file.
+
 ## Workflow and review
 
 ### Before you start

Makefile

@@ -142,6 +142,7 @@ check-example: example-deps
 	pnpm --filter @oac/parsar-example test
 	pnpm --filter @oac/parsar-example build
 	pnpm --filter @oac/parsar-example test:e2e
+	PYTHONDONTWRITEBYTECODE=1 python3 example/rl-eval/rl_eval_test.py
 
 check-mcode-harness:
 	node --test packages/mcode-harness/*.test.mjs

docs/examples.md

@@ -28,4 +28,23 @@ Each feature maps to one part of the API. Read the code next to the guide sectio
 | History | Paging Turns and Items | [Pagination](./api/public-agent-api.md#pagination) | [`src/lib/api.ts`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/parsar/src/lib/api.ts) |
 | Skills and versions | `/skills`, default version | [Skills](./api/public-agent-api.md#skills) | [`src/Skills.tsx`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/parsar/src/Skills.tsx) |
 
+## RL and evaluation rollouts
+
+A Python script that runs batches of tasks as hosted Sessions, with any Harness and model, and grades each answer outside the sandbox. It shows Core as a rollout engine for RL training and evaluation. Each rollout is one Session with no network, task files as initial files and the answer as an Artifact.
+
+Source: [`example/rl-eval`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/rl-eval/README.md).
+
+Run and configure it with the [example README](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/rl-eval/README.md#run), which also owns its output format, task format and training setup.
+
+### What to look at
+
+| Feature | API used | Guide | Code |
+| --- | --- | --- | --- |
+| One Session per rollout | Inline `input`, `metadata` and `openai_hosted` | [Create a Session](./api/public-agent-api.md#create-a-session) | [`rl_eval.py`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/rl-eval/rl_eval.py) |
+| Task files without network | Inline `files` and `network.access: disabled` | [Initial files](../contracts/agents-api/environments.md#initial-files) | [`rl_eval.py`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/rl-eval/rl_eval.py) |
+| Harness and model matrix | `x_agents_core.harness`, `model`, `model_provider` | [Core extensions](./api/public-agent-api.md#core-extensions) | [`rl_eval.py`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/rl-eval/rl_eval.py) |
+| Wait, time out and cancel | Turn list polling, `agent.session.input.cancel` | [Cancel](./api/public-agent-api.md#cancel) | [`rl_eval.py`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/rl-eval/rl_eval.py) |
+| Answer outside the sandbox | Artifacts under `outputs/` | [Artifacts](./api/public-agent-api.md#artifacts) | [`rl_eval.py`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/rl-eval/rl_eval.py) |
+| Trajectories and usage | Items and Turn `usage` | [Turns and Items](./api/public-agent-api.md#turns-and-items) | [`rl_eval.py`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/rl-eval/rl_eval.py) |
+
 To contribute an example, follow the [application example boundary](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/CONTRIBUTING.md#optional-application-example).

docs/zh/examples.md

@@ -1,7 +1,7 @@
 ---
 title: "示例"
 source: docs/examples.md
-source_hash: 99d9e955b20c81fea35ba28915e0bd96619afa94b6fc224b8b7a2a1047cce4a4
+source_hash: 4bd0d6d3d382f7a6e3ef666c204528cb99c21172b8cebb8694c83a16d48456c6
 ---
 
 基于 [Agents API 指南](api/public-agent-api.md)构建的完整应用，每个都使用项目 API 密钥连接真实的 Core 安装。
@@ -30,4 +30,23 @@ source_hash: 99d9e955b20c81fea35ba28915e0bd96619afa94b6fc224b8b7a2a1047cce4a4
 | 历史 | 分页读取 Turns 和 Items | [分页](api/public-agent-api.md#pagination) | [`src/lib/api.ts`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/parsar/src/lib/api.ts) |
 | Skills 和版本 | `/skills`、默认版本 | [Skills](api/public-agent-api.md#skills) | [`src/Skills.tsx`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/parsar/src/Skills.tsx) |
 
+## RL 与评测 rollout {#rl-and-evaluation-rollouts}
+
+一个 Python 脚本，把任务作为托管 Session 批量运行，可使用任意 Harness 和模型，并在沙箱外为每个答案评分。它展示 Core 作为 RL 训练与评测的 rollout 引擎。每次 rollout 是一个无网络的 Session，任务文件作为初始文件，答案作为 Artifact。
+
+源码：[`example/rl-eval`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/rl-eval/README.md)。
+
+按照[示例 README](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/rl-eval/README.md#run)运行和配置；该文档也定义其输出格式、任务格式和训练接入方式。
+
+### 阅读重点 {#what-to-look-at-1}
+
+| 功能 | 使用的 API | 指南 | 代码 |
+| --- | --- | --- | --- |
+| 每次 rollout 一个 Session | 内联 `input`、`metadata` 和 `openai_hosted` | [创建 Session](api/public-agent-api.md#create-a-session) | [`rl_eval.py`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/rl-eval/rl_eval.py) |
+| 无网络的任务文件 | 内联 `files` 和 `network.access: disabled` | [初始文件](../../contracts/agents-api/zh/environments.md#initial-files) | [`rl_eval.py`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/rl-eval/rl_eval.py) |
+| Harness 与模型矩阵 | `x_agents_core.harness`、`model`、`model_provider` | [Core 扩展](api/public-agent-api.md#core-extensions) | [`rl_eval.py`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/rl-eval/rl_eval.py) |
+| 等待、超时与取消 | 轮询 Turn 列表、`agent.session.input.cancel` | [取消](api/public-agent-api.md#cancel) | [`rl_eval.py`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/rl-eval/rl_eval.py) |
+| 沙箱外取得答案 | `outputs/` 下的 Artifacts | [Artifacts](api/public-agent-api.md#artifacts) | [`rl_eval.py`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/rl-eval/rl_eval.py) |
+| 轨迹与用量 | Items 和 Turn `usage` | [Turns 和 Items](api/public-agent-api.md#turns-and-items) | [`rl_eval.py`](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/example/rl-eval/rl_eval.py) |
+
 要贡献示例，请遵循[应用示例边界](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/CONTRIBUTING.md#optional-application-example)。

example/rl-eval/.gitignore

@@ -0,0 +1,2 @@
+runs/
+!tasks/*/files/**

example/rl-eval/README.md

@@ -0,0 +1,119 @@
+# RL and evaluation rollouts
+
+[简体中文](README.zh.md)
+
+A single Python script that uses OpenAgentCore's public `/v1` Agents API as a rollout engine. It runs batches of tasks in hosted sandboxes, with any Harness and model, and grades each result outside the sandbox. The output is a JSONL file with one rollout per line: reward, answer and the full trajectory. It shows that Core serves RL training and evaluation as well as chat products.
+
+## How a rollout works
+
+A rollout is one Session for one (task, Harness, model, sample). The script:
+
+1. Reads the task's `files/` and sends them as inline `files` of an `openai_hosted` Environment under `/workspace/`, with network `access` set to `disabled`, so the sandbox reaches only Core and the model provider.
+2. Creates the Session with the task's `instruction.md` as `input`. A new hosted Session reads `idle` while Core prepares the sandbox, and the input starts the first Turn when it is ready.
+3. Polls the Turn list until the Turn is `completed`, `failed` or `cancelled`. When the timeout passes, it cancels the Turn and records `timeout`.
+4. Reads the answer from the Artifact `/workspace/outputs/answer.json`, which Core captures when the Turn completes.
+5. Grades the answer, reads the Items and Turn usage, and deletes the Session.
+
+Grading runs in the script. `expected.json` never enters the sandbox, so the agent cannot read the expected answer. The grade is binary: the answer must be a JSON object with every expected key and a matching value. Numbers match within a relative tolerance of 1e-6, strings after trimming, and extra keys are ignored.
+
+The [Agents API guide](../../docs/api/public-agent-api.md) describes each call: [Create a Session](../../docs/api/public-agent-api.md#create-a-session), [Cancel](../../docs/api/public-agent-api.md#cancel), [Turns and Items](../../docs/api/public-agent-api.md#turns-and-items), [Artifacts](../../docs/api/public-agent-api.md#artifacts) and [Core extensions](../../docs/api/public-agent-api.md#core-extensions). The [Environment contract](../../contracts/agents-api/environments.md) owns the initial file limits and the restricted network.
+
+## Run
+
+Use Python 3.9+. Configure [Core](../../docs/getting-started/install.md), a model, and a Project API key as in the [quickstart](../../docs/getting-started/quickstart.md):
+
+```sh
+cd example/rl-eval
+python3 -m venv .venv
+. .venv/bin/activate
+pip install -r requirements.txt
+export OPENAI_BASE_URL=https://core.example/v1
+read -rs OPENAI_API_KEY && export OPENAI_API_KEY
+python3 rl_eval.py --harness codex --harness claude_sdk --samples 4
+```
+
+Without `--harness` and `--model`, rollouts use the deployment default Harness and the installation default model, labelled `default`.
+
+| Option | Meaning |
+| --- | --- |
+| `--harness`, `--model` | Repeatable. Every combination runs |
+| `--samples K` | Rollouts per task, Harness and model. Default 1 |
+| `--concurrency N` | Parallel rollouts. Default 4 |
+| `--tasks DIR`, `--task ID` | Task directory (default `tasks/`) and a repeatable filter on task ids |
+| `--timeout SECONDS` | Per rollout, including sandbox preparation. Default 900 |
+| `--out PATH` | JSONL file. Default `runs/<UTC timestamp>.jsonl` |
+| `--model-base-url`, `--model-protocol` | Your own model endpoint and its protocol (`anthropic`, `responses` or `chat_completions`). The key comes only from `MODEL_API_KEY`. Requires `--model` |
+
+The run ends with one Markdown table per Harness and model: rollouts, mean reward, pass@k (the share of tasks with at least one reward-1 sample), errors and timeouts, and mean duration. The exit code is 0 even when every reward is 0.
+
+## Output
+
+Each finished rollout appends one JSON line to the output file:
+
+| Field | Meaning |
+| --- | --- |
+| `rollout_id`, `task`, `harness`, `model`, `sample` | Identity of the rollout. `harness` and `model` read `default` when not set |
+| `session_id` | The Core Session, deleted after the rollout |
+| `status` | `completed`, `failed`, `cancelled`, `timeout` or `error` |
+| `reward` | `1.0` or `0.0` |
+| `answer` | The parsed `answer.json`, or `null` |
+| `duration_s`, `usage` | Wall time and the last Turn's token usage. Usage is `null` for Harnesses that report none, see [Usage](../../contracts/agents-api/sessions-events.md#usage) |
+| `items` | The trajectory: every Session Item in order, as returned by the API |
+| `error` | The exception message when `status` is `error` |
+
+## Tasks
+
+| Task | The agent must |
+| --- | --- |
+| `sales-by-region` | Aggregate a CSV with refunds and quoted fields by region and quarter |
+| `failed-logins` | Count failed SSH password attempts per address in a syslog file |
+| `inventory-bug` | Fix a pricing bug against `PRICING.md`, run the report and read its total |
+| `orders-json` | Query nested JSON for customers and spend on one SKU |
+| `salaries-sqlite` | Compare yearly average salaries per department in a SQLite database |
+
+They need only bash, git, python3, node and ripgrep from the managed sandbox image, and no network.
+
+To add a task, create `tasks/<id>/` with:
+
+- `instruction.md`, which states the question precisely and ends with the output contract: write `/workspace/outputs/answer.json` with the given keys.
+- `expected.json`, a JSON object with the expected values. It stays outside the sandbox.
+- `files/`, copied recursively to `/workspace/` with relative paths kept. Core accepts at most 50 inline files, 5 MiB each and 10 MiB in total.
+
+## Connect training
+
+Each Harness calls the model natively, so a training-side inference server sees every request and records token ids and logprobs. Core records none of them. Point rollouts at that server:
+
+```sh
+export MODEL_API_KEY=<key-of-the-inference-server>
+python3 rl_eval.py --model my-policy --model-protocol responses \
+  --model-base-url 'https://infer.example/rollouts/{rollout_id}/v1' --samples 8
+```
+
+`{rollout_id}` is replaced per rollout with the `rollout_id` in the output, so the server can attribute each request to its rollout. The K samples of a task form a GRPO group, and `reward` is the group's outcome reward. With network `disabled`, the sandbox still reaches the model provider on its `base_url` port at public addresses. A private inference host needs the deployment's [network cap](../../contracts/agents-api/environments.md#restricted-network) to cover its name.
+
+## Scope
+
+The script covers tasks whose answer is graded outside the sandbox. Tasks graded inside their own environment, such as SWE-bench and Terminal-Bench, need a container you own with a `self_hosted` Environment, and are not covered.
+
+## Results
+
+Recorded on 2026-10-29 on a test deployment built from `main`, with microsandbox nodes and MiniMax-M2.7 as the default model of both Harnesses:
+
+```sh
+python3 rl_eval.py --harness codex --harness claude_sdk --samples 2 --concurrency 2
+```
+
+| harness | model | rollouts | mean reward | pass@k | errors/timeouts | mean duration (s) |
+| --- | --- | --- | --- | --- | --- | --- |
+| claude_sdk | default | 10 | 1.000 | 1.000 | 0 | 25.7 |
+| codex | default | 10 | 1.000 | 1.000 | 0 | 17.6 |
+
+Both Harnesses solve every bundled task, so the tasks show the loop, not a training signal. A GRPO group teaches only when its samples disagree: train on tasks your policy solves some of the time.
+
+## Validation
+
+```sh
+python3 rl_eval_test.py
+```
+
+The tests need no network and no `openai` package. They cover grading, the pass@k summary and loading every bundled task.