docs: add intro video and make the README agent-agnostic - #157
Conversation
Add the YouTube intro video and rework the README so it reads as a harness-agnostic framework rather than a Claude Code tool: - New tagline: "Playwright for coding agents". - Title drops "Claude Code skills" for "their skills". - Intro names all four harnesses and states that changing harness is one field (agent.type) with tasks/criteria/scoring unchanged. - Quick Start prerequisites become a per-agent runtime table (claude-code, codex, antigravity, opencode) instead of the Claude CLI only. - Task Definition section explains swapping agent.type and links Run-Limit Parity. - Known limits: bring your own agent runtime as well as model credentials; mention OpenRouter for open-weight models via OpenCode. - Acknowledgments credit every harness the framework drives. The plugin section stays Claude Code specific — it is, factually — but says so and notes the suites it authors run on every harness. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016v9L8gXbq4MzH6V9JZPqcb
Bring the GitHub Pages stub in line with the reworked README: drop "Claude Code skills" from the title, name all four harnesses (OpenCode was missing entirely), and lead with the "Playwright for coding agents" tagline in both the meta description and the visible lead paragraph. The old lead also read "agents and their Claude Code skills", which parsed wrong even for a Claude-only framing. The stub's own comment says its description mirrors the package metadata, so pyproject's description and keywords gain OpenCode too — otherwise that comment stops being true the moment the stub changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016v9L8gXbq4MzH6V9JZPqcb
|
Claude finished @uipreliga's task in 1m 43s —— View job Code Review in Progress
|
bai-uipath
left a comment
There was a problem hiding this comment.
Fix what you agree with, otherwise lgtm. The agent-agnostic rework is the right call and the video placement is clean. The notes below are all about the framing that's left.
-
The skills-only read survives, and that's the one costing us. A prospect benchmarking agents on their own domain task bounced off this page believing the framework was skills-only. The title still leads with skills, the audience line still says "built for CLI and skill builders", and the "not an agentic coding benchmark" disclaimer reads as "we don't do domain benchmarks". Fix: keep the SWE-bench contrast but recast it as "not a fixed leaderboard: you bring the tasks and the scoring", and widen the audience line to include benchmark authors.
-
The docs site still says "Claude Code skills".
docs/index.md(title, description, H1),docs/llms.txt,docs/comparison.md, andmkdocs.yml'ssite_descriptionlive in this repo, sync to coder-eval.com/docs, and outrank both surfaces this PR touches. Cheap to fold in here. -
The Quick Start asks for the contributor toolchain.
--extra devpulls pytest, ruff, pyright, and pre-commit onto someone who just wants one run, and thepip installpath is parked under a heading that reads CI-only.uvitself is the right call given the 3.13 floor, so this is about which extras and which ordering, not about uv. -
Minor: the prerequisites table hands users
pip installthree lines aboveuv sync(should be--extra codex/--extra antigravity); "installs the harness adapters, never the agents themselves" isn't true for codex or antigravity, whose extras ship the runtime; and the agent roster is now restated in five places with nothing holding it toAgentType, which is how OpenCode went missing in the first place.

Summary
Two things, both docs/copy only — no source or behavior changes.
1. Add the intro video. New
## Watch the introsection in the README with a clickable YouTube thumbnail and the linked title: Coder Eval: UiPath open-source framework to test AI Coding Agents. Placed after the feature list so it does not orphan those bullets under the heading.2. Make the framing agent-agnostic. The README and the Pages stub both read as a Claude Code tool, even though the framework has driven four harnesses for a while. OpenCode was missing from several surfaces entirely. New tagline throughout: "Playwright for coding agents".
README
Title: "…AI coding agents and their skills" (was "Claude Code skills").
Tagline under the badges: Playwright for coding agents — one declarative test file, any agent runtime, a real sandbox, and a pass/fail gate in CI.
Intro states the switch cost explicitly: changing harness is one field (
agent.type); tasks, criteria, scoring, telemetry, and reports stay the same.Quick Start prerequisites were "the Claude CLI" only. Now a table of all four agents with
agent.type, install command, and a link to each guide:agent.typeclaude-codebrew install claudecodexpip install 'coder-eval[codex]'antigravitypip install 'coder-eval[antigravity]'opencodenpm install -g opencode-aiTask Definition notes swapping
agent.typeor-D agent.type=opencode, and links Run-Limit Parity.Known limits: "bring your own agent runtime and model credentials", including an OpenRouter key for open-weight models via OpenCode.
Acknowledgments credit all four harnesses, not just the Claude Agent SDK.
Left Claude Code specific on purpose: the "Use inside Claude Code" plugin marketplace section — it genuinely is — but it now says so, and notes the suites authored there run on every harness.
Pages stub + package metadata
.github/pages-stub/index.html:<title>,<meta name="description">, and the visible lead paragraph. OpenCode was absent from all three. The old lead also read "agents and their Claude Code skills", which parsed wrong even under the Claude-only framing.pyproject.toml: the stub's own comment says its description mirrors the package metadata, sodescriptionandkeywordsgain OpenCode — otherwise that comment stops being true the moment the stub changes. This ships to PyPI on the next release; happy to drop it if you would rather keep metadata out of a docs PR.Verification
make lint(388 passed) andmake checkgreen. No doc-surface rule tripped — CE026's Action snippet, its prerequisite steps, and the CE028-generated docs index tables are all untouched. Both YouTube thumbnail URLs verified 200.Not covered
The coder-eval.com website source is not in this repo, so its copy still carries the old Claude-Code-first framing and needs a separate change.
🤖 Generated with Claude Code
https://claude.ai/code/session_016v9L8gXbq4MzH6V9JZPqcb