Skip to content

docs: show the seed baseline in the first-screen demo transcripts - #1160

Merged
Birfy merged 2 commits into
mainfrom
docs/1146-first-run-transcript
Nov 18, 2026
Merged

Birfy merged 2 commits into
mainfrom
docs/1146-first-run-transcript

Conversation

@HugoXK

@HugoXK HugoXK commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1146.

What changed

The four first-screen demo transcripts now carry the line the offline demo actually prints.

README.md, docs/start.md, docs/plugin-quickstart.md §2 and docs/testing-guide.md §2 showed a bare held-out reward: 1.000, which the CLI has not printed since the seed baseline landed (#708), so a working install looked different from the docs on the first screen. Each block now reads held-out reward: 0.000 -> 1.000 outcomes: {'committed': 1}, with the CLI's spacing before outcomes and after what it learned:; the plan and what it learned lines are otherwise unchanged.

docs/algorithms.md is untouched: its 1.000 -> 1.000 seed-already-solved line is correct.

A docs guard, test_first_screen_demo_transcript_shows_the_seed_baseline in tests/test_docs_examples.py, pins those four pages to the seed-baseline line and rejects the bare form, so the transcripts cannot drift from the command again. One Fixed changelog fragment.

Six files, +33/−6.

Verification

Tip SHA: 85267778bedf44c6026f210d250b9fc15d082905

Red first, base 081af544 with only the new test rows: 1 failed, 5 passed — AssertionError: README.md, assert 'held-out reward: 0.000 -> 1.000' in ....

Offline demo at the tip, default skill dir (verbatim tail):

plan: 12 tasks, 4 rounds x 2 workers, up to 68 agent calls

running (20261118-171904-f38e13)...
  round  0  reward=1.000  +1/-0

held-out reward: 0.000 -> 1.000   outcomes: {'committed': 1}
what it learned:  rules.md -> 'COLUMN: amount'

Ran:

  • pytest -q tests/test_changelog_fragments.py tests/test_docs_examples.py tests/test_initial_reward.py tests/test_seed_already_solved.py -> 62 passed
  • pytest -q tests/test_demo.py -> 37 passed
  • python -m tools.changelog_fragments check -> 76 changelog fragment(s) OK
  • mkdocs build --strict -> rc 0 in 12.97 s and no strict warnings, with the social plugin's card rendering excluded. With the unmodified config the build hangs on this host at the first page's material/social post_page (both a read-only and a writable HOME), a host limitation rather than a docs error; the build CI job runs the unmodified config.
  • agentdescent demo (offline, AGENTDESCENT_HOME in a scratch dir) -> rc 0, transcript above

Not run:

  • full pytest -q -> not achievable on this host in one pass; an 8-way split of tests/test_*.py finished six chunks (17 failed, 5523 passed, 46 skipped) before the two docker/network chunks stalled with no result. All 17 failures are the host's environment set (test_apply_noop OSError, docker sandbox setup, wheel smoke, era_srbench, skill-library installs, shadowing cwd), the same set as at the merge base; none touches a changed file. The CI matrix is the authoritative suite.
  • wheel smoke -> no packaging change

No UI surface: docs text only

@HugoXK HugoXK added documentation Improvements or additions to documentation area:docs Documentation site (docs/, mkdocs.yml) and README labels Nov 18, 2026
@HugoXK
HugoXK marked this pull request as ready for review November 18, 2026 17:00
@HugoXK

HugoXK commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator Author

Advisory read at 85267778bedf44c6026f210d250b9fc15d082905 — state in the same API read: open, merged: false, non-draft, base 081af544, 6 files +33/−6, labels documentation area:docs. No blocking finding.

Checked against the code, not the issue text:

  • I ran the offline demo at the tip (AGENTDESCENT_HOME in a scratch dir, rc 0). It prints held-out reward: 0.000 -> 1.000 outcomes: {'committed': 1} (three spaces before outcomes) and what it learned: rules.md -> 'COLUMN: amount' (two after the colon), from agentdescent/cli.py:912 and :917 at this tip. All four pages carry both lines byte-for-byte — README.md:48, docs/start.md:26, docs/plugin-quickstart.md:81, docs/testing-guide.md:80 — and their plan:/round 0 lines match the same run.
  • The guard is non-vacuous: at base 081af544 with only the new test file, 1 failed, 5 passed (AssertionError: README.md, tests/test_docs_examples.py:97, positive assert). On the base pages directly, held-out reward: 1.000 outcomes is still present in README.md/docs/start.md and held-out reward: 1.000 outcomes in the other two, so both rejection asserts have a red pre-state. At the tip, tests/test_docs_examples.py is 6 passed.
  • Scope is as stated: docs/algorithms.md untouched and its 1.000 -> 1.000 seed-already-solved line is correct; one Fixed fragment (changelog.d/1146-first-run-transcript.md), no Python code.
  • python -m tools.gen_api_docs --check and python -m tools.changelog_fragments check pass at this tip; tests/test_docs_links.py 22 passed; the body's focused sets reproduce (62 passed; tests/test_demo.py 37 passed).
  • No UI surface: docs text only. — screenshot pair not due.

Not run: mkdocs build --strict (no mkdocs in this environment) and the full pytest -q (docs-only diff, no Python path changed; the docs tests above are the affected set).

Non-blocking: the body's Ran: says 76 changelog fragment(s) OK; at this tip the check reports 77 (76 at base plus the new fragment). The check itself passes.

The four first-screen pages still show a bare `held-out reward: 1.000`, which the CLI has not printed since the seed baseline landed. The guard names the four pages and the line the command prints now (#1146).
The four first-screen pages now carry the line the offline demo prints, `held-out reward: 0.000 -> 1.000`, with the CLI's spacing before `outcomes`, so a working install matches the docs (#1146).
@HugoXK

HugoXK commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator Author

Verification complete at 85267778bedf44c6026f210d250b9fc15d082905 (base 081af544) — both seats PASS on this exact tip.

  • Reviewer: PASS. The two demo lines match the four pages byte-for-byte (README.md:48, docs/start.md:26, docs/plugin-quickstart.md:81, docs/testing-guide.md:80); guard red at base (1 failed, 5 passed, AssertionError: README.md at tests/test_docs_examples.py:97) and green at the tip (6 passed); diff is the six declared files, +33/−6, with docs/algorithms.md untouched; No UI surface: docs text only.
  • Test: PASS. Same red-first result (1 failed, 5 passed); offline agentdescent demo rc 0, spaces counted on the bytes (od -c): three before outcomes, two after what it learned:; both lines match as whole lines in all four pages; python -m tools.changelog_fragments check → 77 OK.
  • CI at this tip: build, otel-wire, test (3.9), test (3.10), test (3.11), test (3.12), test (3.13), reviewed tip, risk tier all success; deploy skipped.
  • Nit, not a blocker: the body's 76 changelog fragment(s) OK is one behind this tip — 76 at base plus the new fragment makes 77; the check passes.

Next: the counting PASS, then the owner's click. Fixes #1146.

Written-by: AgentDescent Lead

@github-actions github-actions Bot added risk:T1 Risk tier T1 (tests, examples, changelog.d), set by the controller from the diff agent-pr Pull request opened by an agent labels Nov 18, 2026
@Birfy

Birfy commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator

Review (Birfy)

Tip SHA: 8526777
Ran:

  • .github/scripts/risk_gate.py --pr 1160 --repo ThinkFlowLab/agentdescent → T1 (39 lines / 6 files)
  • pytest -q tests/test_docs_examples.py tests/test_changelog_fragments.py tests/test_initial_reward.py tests/test_seed_already_solved.py → 62 passed at this tip
  • pytest -q tests/test_demo.py → 37 passed
  • python -m tools.changelog_fragments check → 77 changelog fragment(s) OK
  • red-first: base 081af54 with only the new test rows → 1 failed, 5 passed (AssertionError: README.md at tests/test_docs_examples.py:97)
  • agentdescent demo (offline, AGENTDESCENT_HOME in a scratch dir) → rc 0; prints held-out reward: 0.000 -> 1.000 outcomes: {'committed': 1} and what it learned: rules.md -> 'COLUMN: amount'; both lines match README.md, docs/start.md, docs/plugin-quickstart.md and docs/testing-guide.md byte-for-byte; plan line is unchanged (12 tasks, 4 rounds x 2 workers, up to 68 agent calls)
  • docs/algorithms.md is not in the diff
  • CI at this tip: build, otel-wire, test (3.9–3.13), reviewed tip, risk tier SUCCESS; deploy skipped
    Not run:
  • mkdocs build --strict locally — the CI build job is green at this tip
  • full pytest -q — no code path changed; the CI test matrix is the line
  • Ark real-model run — no model path changed
    Findings:
  • none blocking. Nit already noted by the author: the body's 76 changelog fragment(s) OK is one behind this tip (76 at the base plus the new fragment is 77); the check passes.
    Verdict: PASS

@Birfy
Birfy merged commit f2eda2e into main Nov 18, 2026
10 checks passed
@Birfy
Birfy deleted the docs/1146-first-run-transcript branch November 18, 2026 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-pr Pull request opened by an agent area:docs Documentation site (docs/, mkdocs.yml) and README documentation Improvements or additions to documentation risk:T1 Risk tier T1 (tests, examples, changelog.d), set by the controller from the diff

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: first-run demo transcript still shows held-out reward: 1.000 without the seed baseline

2 participants