Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,8 @@ cases with known totals and a local program as the agent, then runs the real
loop — only the model is replaced — and ends with:

```
held-out reward: 1.000 outcomes: {'committed': 1}
what it learned: rules.md -> 'COLUMN: amount'
held-out reward: 0.000 -> 1.000 outcomes: {'committed': 1}
what it learned: rules.md -> 'COLUMN: amount'
```

Skill evolution here means a **multi-file skill library**, not a single prompt:
Expand Down
5 changes: 5 additions & 0 deletions changelog.d/1146-first-run-transcript.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Fixed
The first-screen demo transcripts (`README.md`, `docs/start.md`,
`docs/plugin-quickstart.md`, `docs/testing-guide.md`) now show the seed baseline
the CLI prints, `held-out reward: 0.000 -> 1.000`, instead of a bare `1.000`, and
a docs guard keeps those four blocks on the command's output (#1146).
2 changes: 1 addition & 1 deletion docs/plugin-quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ plan: 12 tasks, 4 rounds x 2 workers, up to 68 agent calls
running (20260905-134121-1c695b)...
round 0 reward=1.000 +1/-0

held-out reward: 1.000 outcomes: {'committed': 1}
held-out reward: 0.000 -> 1.000 outcomes: {'committed': 1}
what it learned: rules.md -> 'COLUMN: amount'
```

Expand Down
4 changes: 2 additions & 2 deletions docs/start.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ cases with known totals and a local program as the agent, then runs the real
loop — only the model is replaced — and ends with:

```
held-out reward: 1.000 outcomes: {'committed': 1}
what it learned: rules.md -> 'COLUMN: amount'
held-out reward: 0.000 -> 1.000 outcomes: {'committed': 1}
what it learned: rules.md -> 'COLUMN: amount'
```

Skill evolution means a **multi-file skill library**, not a single prompt:
Expand Down
2 changes: 1 addition & 1 deletion docs/testing-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ plan: 12 tasks, 4 rounds x 2 workers, up to 68 agent calls
running (20260905-134121-1c695b)...
round 0 reward=1.000 +1/-0

held-out reward: 1.000 outcomes: {'committed': 1}
held-out reward: 0.000 -> 1.000 outcomes: {'committed': 1}
what it learned: rules.md -> 'COLUMN: amount'
```

Expand Down
22 changes: 22 additions & 0 deletions tests/test_docs_examples.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,3 +75,25 @@ def test_snapshot_docs_spec_examples_are_json():
for name in re.findall(r'load_spec\("([^"]+)"\)', text):
assert name.endswith(".json"), (
f'docs/snapshot.md load_spec("{name}") is not a file load_spec can read')


FIRST_SCREEN_DEMO_PAGES = (
"README.md",
"docs/start.md",
"docs/plugin-quickstart.md",
"docs/testing-guide.md",
)


def test_first_screen_demo_transcript_shows_the_seed_baseline():
"""#1146: the CLI prints the seed and the final reward on one line.

`agentdescent demo` ends with `held-out reward: 0.000 -> 1.000`, and these
four pages are the transcript a new user compares a working install against;
a bare `1.000` makes the first screen disagree with the command it documents.
"""
for page in FIRST_SCREEN_DEMO_PAGES:
text = (ROOT / page).read_text()
assert "held-out reward: 0.000 -> 1.000" in text, page
assert "held-out reward: 1.000 outcomes" not in text, page
assert "held-out reward: 1.000 outcomes" not in text, page
Loading