[bagel786]

Resets every CPU environment containing a swapped opponent policy, locating that policy by physical agent slots and policy_layout. Environment tags contain only the maximum policy ID and cannot identify all participating policies. Clears recurrent/reward/terminal state for all agents in each affected environment and uploads refreshed buffers.

Fixes #620. Continues #672.

Validation: a host fixture compiles and executes the production reset function across three policies and two buffers; it fails on the original code and passes with this fix. Full Python suite attempted: 13 collection errors from unavailable torch/pufferlib/CUDA dependencies. CUDA compilation and GPU runtime validation remain outstanding; no claim of CUDA validation is made.

[Copilot]

🔵 Needs a closer look

Environment matching must account for physical policy slots in multi-policy environments.

Pull request overview

Resets CPU environments and recurrent state when historical opponent policies are swapped.

Changes:

• Resets environment boundaries, rewards, terminals, and recurrent state.
• Uploads refreshed buffers before the next rollout.
• Applies resets after historical checkpoint swaps.

File summaries

File	Summary
src/pufferl.cu	Implements policy-specific environment resets; matching can fail for multi-policy environments because env->tag reflects only the maximum policy ID.

Review details

Suppressed comments (1)

src/pufferl.cu:1241

• env->tag is assigned as the maximum effective policy ID for an environment in env_setup, not as an identity for every policy used by its agents. Thus an environment containing policies [0, 1, 2] has tag 2 and is skipped when policy 1 is swapped, leaving that agent's recurrent state and environment trajectory on the old opponent. Determine whether any agent's physical slot maps to policy_idx via policy_layout, and reset the whole environment when it does.

        if (env->tag != policy_idx) {
            continue;

• Files reviewed: 1/1 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.