Skip to content

feat(harness): world actors with permitted actions and an action log - #220

Closed
KarthikAvinashFI wants to merge 24 commits into
feat/TH-8651-sub-actors-harnessfrom
feat/TH-8875-action-log
Closed

KarthikAvinashFI wants to merge 24 commits into
feat/TH-8651-sub-actors-harnessfrom
feat/TH-8875-action-log

Conversation

@KarthikAvinashFI

@KarthikAvinashFI KarthikAvinashFI commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

Summary

World actors: people and systems in the world who act only through the world's own actions, each with permitted actions checked by the world's tool proxy, and an action log of every attempted action with who made it and what came of it. Scheduled per scenario at triggers tied to what the agent does. Part of TH-8875.

Status

What How it was verified
✅ Permitted actions enforced at the world's door; an action an actor is not permitted is refused and logged as its attempt Live run ac722b88 (1 Nov): a coworker's approval refused, the manager's committed, both tied to the agent's request; 537 tests
✅ Every action logged with its actor and outcome; the agent's own calls unchanged Tests; live runs
✅ A world actor's actions never count as the agent's: the sealed tool trace keeps only the agent's rows, so grading, checks and a sub-actor's tool trigger see only what the agent did (found in review: the trace query did not read the actor column) Test on the real row shape, fails with the fix reverted
✅ An actor's identity is matched in the same encoding the arguments use, so a name with accents or quotes still reaches its actor; removing an actor a scenario includes is refused, as removing a sub-actor already is; the actor token is emptied when the trial ends; a world database error is logged as a fault, not as an unknown actor Tests
✅ A world actor added after the build runs Fixed in #221 (53718c6e); live run 80dbdf46 (3 Nov)
❌ World actors on chat calls, and the trial view of who did what Rest of TH-8875 and TH-8921

1. Why

A scenario's other people (a manager who approves a discount, a coworker who claims to) are currently only voices in the conversation. Nothing stops one of them from "approving" something they have no authority for, and a check cannot tell an attempted action from a committed one. This adds world actors: people and systems that act in the world through its own actions, each limited to the actions it is permitted, with every attempt recorded in the world's action log under who made it.

Stacked on #207 (sub-actors on the simulated user side). Everything here is behind HARNESS_SUB_ACTORS; with it off, builds and runs are unchanged.

This is a partial delivery of TH-8875. Its done-when is a trial in which the coworker's attempt is refused, the manager's approval commits, and the trial shows the two apart. This PR delivers the world side: permissions enforced at the world door and every attempt in the action log. Still to come for TH-8875: actors acting on chat calls (the dispatcher runs in the voice call runner only), a live discount-approval trial, and the trial view of who did what.

2. What changed

File Change Preserved
harness/actors.py Actor, ActorList, Trigger, ScheduledAction, ScenarioActor; load_actors / save_actors (actors.json); actors_seed, the world's own table of who may do what Sub-actor models from #207 unchanged
harness/tool_trace_proxy.py The world door checks permission per actor: a request naming an actor needs this trial's actor token, the actor must be listed and hold the action, else it is refused before the world with a refusal_code. Every attempt is logged with outcome, refusal_code, actor_id, parent_event_id, on_behalf_of. Idempotency-Key: a retry reaches the world once and gets the first result; a reused key with other arguments is refused Agent tool calls are logged as before, now under actor_id = agent
harness/actor_dispatch.py ActorDispatcher plays a scenario's scheduled actor actions at their triggers (after an agent tool call, after a delay, when the agent reaches an actor's identity, or drawn once by chance) through the world door
harness/call_runner.py, harness/prove.py Actor rows are sealed with their outcome; the agent's sealed calls carry their action ids, so an actor action can name its cause; proofs play permitted actor actions and keep them out of the agent's calls Agent call evidence unchanged
harness/scenario_tools.py, scenario.py, scenarios.py, folder.py, build.py, builder.py, bundle_author_v2.py, process_runtime.py, world/{handle,tools}.py, source_data_invariants.py add_actor / drop_actor for the builder; a scenario naming an actor the world does not list is not kept; the seed carries the actors table Unchanged when the switch is off
skills/modules/world-actors.md When and how to write world actors; checks described, not quoted

3. Behavior callouts

  • Off by default: HARNESS_SUB_ACTORS unset means no actor fields, tools or prompts.
  • An attempted action and a committed one are separate rows; checks read who did what from the action log, never from the transcript.
  • A refused actor is told only why it was refused, never what the world holds.

4. Tests

File Locks
test_tool_trace_proxy.py (9) Agent actions logged with the world's outcome; idempotent retries; reused key refused; refused key may retry; worlds seeded before the new columns still log; permitted actor commits under its id; unpermitted actor refused before the world; actor named without the trial token refused; unlisted actor refused
test_actor_dispatch.py (9) Actor acts under its id with the token; trigger after a tool call fires once and names its cause; delay; identity contact; chance drawn once and the same on rerun; unlisted actor sends nothing; agent id never an actor; bad trigger field refused; checks read who did what from the log
test_actor_done_when.py The coworker cannot authorise the discount, the manager can, and the trial shows both (real Postgres world)
test_call_runner.py, test_harness.py (additions) Sealing of actor rows, cause links, the switch, builder tools and prompts

The three test_harness.py failures (hosted_claude_session, skill_only_names_tools, system_prompt_budget) fail the same way on dev and main without this change.

5. Commands to run the tests

uv sync --extra harness-stores
uv run pytest tests/harness/test_tool_trace_proxy.py tests/harness/test_actor_dispatch.py tests/harness/test_actor_done_when.py tests/harness/test_call_runner.py tests/test_harness.py -q

Terminal output

537 passed; 3 failures that also fail on dev without this PR.

6. Steps to test on local

  1. Set HARNESS_SUB_ACTORS=1 for the harness.
  2. Build an environment for an agent that applies discounts, and add a Manager allowed approve_discount and a Coworker who is not.
  3. Run a scenario where the coworker says "approved" first and the manager approves later.
  4. In the trial's action log: the coworker's approve_discount is refused (not permitted), the manager's is committed.
  5. The discount is applied only after the manager's approval.

Video

Live run ac722b88 on the local stack (1 Nov): the coworker's approve_discount refused (not permitted for coworker), the sales manager's approve_discount committed, both tied to the agent's request_approval; the 20% discount applied after; all four sub-goals passed.

Tested by hand

Check Result
The done-when's world rule on a real Postgres world, without a call: coworker approval refused, manager approval committed, discount applied after Passed (1 Nov)
Combined branch with the actor editor and declared workers PRs: changed tests 1386 passed, the same 3 pre-existing failures
Live voice build and run on this code (2 Nov) Build and calls completed; the sub-actor spoke and was graded. That build had no world actors, so the live path for world actors is not shown by it

Not tested

  • A live call where a world actor acts during the call; covered only by the tests above.
  • Any chat call: world actors do not act on chat calls yet (see Why).

Merge note

call_runner.py conflicts with the declared workers PR (#226). The resolution keeps both: each sealed row is trace_row(call) with this PR's marks merged in, then the actor rows are appended.

7. Scope in / scope out

In: world actors, permission at the world door, the action log, scheduled actor play, idempotency keys.

Out:

  • Editing actors through the platform API: the platform actor editor PR.
  • Live model-driven actors: actors here act on scheduled data per scenario.

8. Design choices

  • Permission is enforced at the world door, not in a prompt, so an actor cannot talk its way past it.
  • Actors are named by world action, never by the agent's tool names, so the same cast works for any agent.
  • A per-trial token stops the agent from posing as an actor.

Linear

Fix TH-8875

@KarthikAvinashFI
KarthikAvinashFI deleted the feat/TH-8875-action-log branch November 4, 2026 03:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant