Repository navigation
feat(harness): world actors with permitted actions and an action log - #220
Closed
KarthikAvinashFI wants to merge 24 commits into
Closed
KarthikAvinashFI wants to merge 24 commits into
KarthikAvinashFI wants to merge 24 commits into
Conversation
…coding, once for the run and the proof
…s with sub-actors
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
World actors: people and systems in the world who act only through the world's own actions, each with permitted actions checked by the world's tool proxy, and an action log of every attempted action with who made it and what came of it. Scheduled per scenario at triggers tied to what the agent does. Part of TH-8875.
Status
ac722b88(1 Nov): a coworker's approval refused, the manager's committed, both tied to the agent's request; 537 tests53718c6e); live run80dbdf46(3 Nov)1. Why
A scenario's other people (a manager who approves a discount, a coworker who claims to) are currently only voices in the conversation. Nothing stops one of them from "approving" something they have no authority for, and a check cannot tell an attempted action from a committed one. This adds world actors: people and systems that act in the world through its own actions, each limited to the actions it is permitted, with every attempt recorded in the world's action log under who made it.
Stacked on #207 (sub-actors on the simulated user side). Everything here is behind
HARNESS_SUB_ACTORS; with it off, builds and runs are unchanged.This is a partial delivery of TH-8875. Its done-when is a trial in which the coworker's attempt is refused, the manager's approval commits, and the trial shows the two apart. This PR delivers the world side: permissions enforced at the world door and every attempt in the action log. Still to come for TH-8875: actors acting on chat calls (the dispatcher runs in the voice call runner only), a live discount-approval trial, and the trial view of who did what.
2. What changed
harness/actors.pyActor,ActorList,Trigger,ScheduledAction,ScenarioActor;load_actors/save_actors(actors.json);actors_seed, the world's own table of who may do whatharness/tool_trace_proxy.pyrefusal_code. Every attempt is logged withoutcome,refusal_code,actor_id,parent_event_id,on_behalf_of.Idempotency-Key: a retry reaches the world once and gets the first result; a reused key with other arguments is refusedactor_id = agentharness/actor_dispatch.pyActorDispatcherplays a scenario's scheduled actor actions at their triggers (after an agent tool call, after a delay, when the agent reaches an actor's identity, or drawn once by chance) through the world doorharness/call_runner.py,harness/prove.pyharness/scenario_tools.py,scenario.py,scenarios.py,folder.py,build.py,builder.py,bundle_author_v2.py,process_runtime.py,world/{handle,tools}.py,source_data_invariants.pyadd_actor/drop_actorfor the builder; a scenario naming an actor the world does not list is not kept; the seed carries the actors tableskills/modules/world-actors.md3. Behavior callouts
HARNESS_SUB_ACTORSunset means no actor fields, tools or prompts.4. Tests
test_tool_trace_proxy.py(9)test_actor_dispatch.py(9)test_actor_done_when.pytest_call_runner.py,test_harness.py(additions)The three
test_harness.pyfailures (hosted_claude_session,skill_only_names_tools,system_prompt_budget) fail the same way ondevandmainwithout this change.5. Commands to run the tests
Terminal output
537 passed; 3 failures that also fail on
devwithout this PR.6. Steps to test on local
HARNESS_SUB_ACTORS=1for the harness.approve_discountand a Coworker who is not.approve_discountisrefused(not permitted), the manager's iscommitted.Video
Live run
ac722b88on the local stack (1 Nov): the coworker'sapprove_discountrefused (not permitted for coworker), the sales manager'sapprove_discountcommitted, both tied to the agent'srequest_approval; the 20% discount applied after; all four sub-goals passed.Tested by hand
Not tested
Merge note
call_runner.pyconflicts with the declared workers PR (#226). The resolution keeps both: each sealed row istrace_row(call)with this PR's marks merged in, then the actor rows are appended.7. Scope in / scope out
In: world actors, permission at the world door, the action log, scheduled actor play, idempotency keys.
Out:
8. Design choices
Linear
Fix TH-8875