Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,10 @@ breaking changes may land in a minor release.
place (#865).
- Add the `hooks.events-incomplete` validate warning: a profile hook event whose relay is
not registered, with the `init` that registers it or the CLI version it needs.
- Add opt-in `[limits] usage_limit_action = "relaunch"`: a session that ends on a usage
limit relaunches the story at once without charging an attempt, up to
`max_usage_relaunches` per story (default 3), then pauses as before. The relaunched
session gets `BMAD_LOOP_USAGE_RELAUNCH` for account-rotation wrappers (#959).

### Changed

Expand Down Expand Up @@ -207,6 +211,8 @@ breaking changes may land in a minor release.
- Name a displaced psmux registry that belongs to another project (or this one before a
rename) as such in `cleanup` and the TUI, instead of calling its sessions `not migrated`
(#906).
- Pause a Claude session uncharged, instead of charging a dev attempt for a crash, when
its window dies after a usage-limit or account StopFailure or during a quota wait (#959).

### Security

Expand Down
7 changes: 5 additions & 2 deletions docs/FEATURES.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,10 +108,12 @@ bmad-loop drives a BMAD sprint unattended: a deterministic Python loop launches
- When a Claude Code turn ends on an API error (its `StopFailure` hook), the adapter acts at once and never counts it as a completion: a transient error gets the stall nudge right away and is charged if never recovered, and an account error (billing, authentication, an account hold) pauses the run uncharged with a reason naming the error.
- `validate --probes` launches each role's CLI once with no prompt and fails when its first screen shows a startup dialog that would park an unattended session.
- While the pane shows the CLI waiting out a usage-limit reset, the session timeout and stall nudge are held for at most `[limits] quota_wait_max_min` (default 300; 0 = off) minutes; past that the session pauses with a usage-limit reason.
- With `[limits] usage_limit_action = "relaunch"`, a dev, fix or review session that ends on a usage limit or a Claude Code account error is relaunched at once without charging an attempt or review cycle, at most `[limits] max_usage_relaunches` (default 3) times per story; past that it pauses uncharged as in the default `"pause"` mode.
- In relaunch mode the session ends at the limit instead of waiting for its reset, a Claude Code `rate_limit` error naming no known limit counts as one, and the relaunched session gets `BMAD_LOOP_USAGE_RELAUNCH=<n>` so a wrapper can switch accounts.
- Before an attempt is reset, its commits go to an `attempt-preserve/<run>-<tip8>` branch and uncommitted work to a `refs/attempt-preserve-dirty/*` snapshot.
- **Config:** `[limits] max_dev_attempts`, `[limits] max_review_cycles`, `[limits] session_timeout_min`, `[limits] dev_stall_grace_s`, `[limits] dev_stall_nudges`, `[limits] dev_stall_nudges_cap`, `[limits] quota_wait_max_min`, `[scm] preserve_keep`
- **Config:** `[limits] max_dev_attempts`, `[limits] max_review_cycles`, `[limits] session_timeout_min`, `[limits] dev_stall_grace_s`, `[limits] dev_stall_nudges`, `[limits] dev_stall_nudges_cap`, `[limits] quota_wait_max_min`, `[limits] usage_limit_action`, `[limits] max_usage_relaunches`, `[scm] preserve_keep`
- **Tested by:** `tests/test_escalation.py`, `tests/test_engine.py`, `tests/test_generic_tmux.py`, `tests/test_panematch.py`, `tests/test_sweep.py`, `tests/test_verify.py`, `tests/test_recovery_flow.py`, `tests/test_stories_e2e.py`
- **Docs:** [Preserved attempts](escalations-guide.md#preserved-attempts), [StopFailure signals](adapter-authoring-guide.md#stopfailure-signals), [[limits] reference](reference/configuration.md#limits), [[scm] reference](reference/configuration.md#scm)
- **Docs:** [Preserved attempts](escalations-guide.md#preserved-attempts), [Usage-limit pauses](escalations-guide.md#usage-limit-pauses), [Account rotation](plugin-authoring-guide.md#account-rotation), [StopFailure signals](adapter-authoring-guide.md#stopfailure-signals), [[limits] reference](reference/configuration.md#limits), [[scm] reference](reference/configuration.md#scm)

### Plateau-defer

Expand Down Expand Up @@ -175,6 +177,7 @@ bmad-loop drives a BMAD sprint unattended: a deterministic Python loop launches
- A pattern match is overruled when the session's tracked token usage is nonzero; that session is charged like any failure.
- A session that times out, stalls or crashes after a live usage-limit sighting (its profile's `usage_limit_patterns` or `usage_wait_patterns` on the pane, or an opencode 2.x quota error) pauses with a `usage limit:` reason and is never charged; the reason names the reset time when the line gives one.
- A Claude Code `StopFailure` reporting `rate_limit` is matched by its message against the same usage lists: a blocking limit pauses as a usage limit, and a waiting limit can start the quota wait without a pane capture.
- A session whose window dies after a Claude Code usage-limit or account `StopFailure`, or during a quota wait (including at its admission or cap), ends `crashed` but pauses uncharged like the live case; a crash after an unclassified error is still charged.
- `bmad-loop resolve --at-reset` waits until a usage-limit pause's reset plus `--margin-min` (default 2), then re-arms and resumes; credit exhaustion and an unknown or too-distant reset are refused.
- **Config:** `[environment] probes`, `[environment] probe_timeout_s`, `[verify] env_fault_rc`, `[limits] quota_wait_max_min`; profile keys `env_fault_patterns`, `usage_limit_patterns`, `usage_wait_patterns`
- **Tested by:** `tests/test_engine.py`, `tests/test_cli.py`, `tests/test_escalation.py`, `tests/test_env_fault_patterns.py`, `tests/test_env_fault.py`, `tests/test_generic_tmux.py`, `tests/test_opencode_http.py`, `tests/test_usagereset.py`
Expand Down
49 changes: 45 additions & 4 deletions docs/adapter-authoring-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -532,13 +532,44 @@ The generic adapter acts on it when it arrives, by action:
offers none, so a backend that cannot capture its pane can still admit the wait. No
match leaves `stop-failure-rate-limit-unmatched` and the usual stall timing. A match
that blows the per-pattern timeout leaves `stop-failure-probe-failed` and counts as no
match.
match. In usage-relaunch mode (see below) a wait match and an unmatched value end the
session at once instead.
- **`ignore`**: nothing beyond the crumb.

Every event leaves a `stop-failure` lifecycle crumb (error, action, details, a message
excerpt); an unmapped value also leaves `stop-failure-unmapped` and keeps the usual stall
timing. A later `Stop` clears the latch. `SessionResult.stop_failure` carries the last
uncleared error, as does the heartbeat and the `session-end` journal entry.
uncleared error, as does the heartbeat and the `session-end` journal entry, and
`SessionResult.stop_failure_action` its mapped action.

**A window that dies after the event.** An operator's own `StopFailure` hook may kill the
window, for example to switch accounts. The session then ends `crashed`, but an `account`
error, or a `rate_limit` that matched a usage list, keeps its classification: the result
is labeled `parked` (account) or `usage_limited` (blocked or wait class), and the run gets
the same uncharged pause as the live exit. The adapter leaves a `crash-relabeled`
lifecycle crumb (`kind`, `error`). A `Stop` clears the classification, and so does a
transcript write after the event, since the CLI recovered by itself. A crash whose result
file exists is completed as before and never relabeled. An unmatched `rate_limit` stays an
unlabeled, charged crash outside usage-relaunch mode.

**Usage-relaunch mode.** With `[limits] usage_limit_action = "relaunch"`, primary dev and
review sessions are launched with two flags on `SessionSpec`: `usage_relaunch_mode`, and
`usage_relaunch_left` while the story has relaunches left under
`limits.max_usage_relaunches`. Then:

- a `rate_limit` that matches neither usage list counts as a wait-class usage limit. The
session ends `stalled` + `parked` + `usage_limited` at once, with the evidence
`StopFailure(<error>): <message, else "no message">` and no reset. The
`stop-failure-rate-limit-unmatched` crumb carries `relaunch_mode=True`. A window that
dies after it is relabeled `usage_limited` the same way (`crash-relabeled`,
`kind="unmatched"`). This holds at the cap too, where the engine pauses uncharged at
once instead of after a stall grace.
- with relaunches left, a `rate_limit` that matched `usage_wait_patterns` ends the session
at the event instead of waiting for the quota wait (`quota-wait-skipped`,
`reason="usage-relaunch"`).

The engine relaunches such a session without charging an attempt (see
[Usage-limit pauses](escalations-guide.md#usage-limit-pauses)).

The shipped `claude` profile maps the documented values: `overloaded`, `server_error`,
`unknown`, `max_output_tokens` and `invalid_request` (a context-shape failure that a fresh
Expand Down Expand Up @@ -691,8 +722,9 @@ matched evidence. **When the session timeout fires**, one capture against both l
`usage_limited` on the `timeout` result. Neither gate takes a usage match from a session
that is visibly working: a transcript written within the last 90 s (as of the last
heartbeat) or an unanswered tool call means the line is left over from a limit it
recovered from, and the stall or timeout is an ordinary one. A dead window is never
usage-limited.
recovered from, and the stall or timeout is an ordinary one. A dead window is
usage-limited only when it died during a quota wait or after a usage `StopFailure` (see
[StopFailure signals](#stopfailure-signals)).

A `usage_limited` result is classified as an environment fault with no token-usage
overrule and no tail scan, and pauses with a `usage limit:` reason (see
Expand Down Expand Up @@ -727,6 +759,15 @@ line whose parsed reset lies beyond the remaining cap is not waited on: the sess
the same way at once (`quota-wait-skipped`, `reason="reset-beyond-cap"`). An unresolved
reset is admitted, and the cap bounds it.

A window that dies during the wait ends
`crashed` with the wait's evidence and reset as a wait-class `usage_limited`
(`crash-relabeled`, `kind="quota-wait"`).

In usage-relaunch mode with relaunches left (`SessionSpec.usage_relaunch_left`), no wait
is admitted: where it would open, the session ends `stalled` + `parked` + `usage_limited`
at once (`quota-wait-skipped`, `reason="usage-relaunch"`), so the engine can relaunch the
story uncharged. At the cap the wait admits as usual.

The `opencode-http` adapter runs its own wait loop with no pane, so neither the usage gate
nor the quota wait applies there.

Expand Down
12 changes: 11 additions & 1 deletion docs/escalations-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,7 @@ A session the adapter saw blocked by a coding-CLI usage limit (out of credits, o
usage limit: dev session stalled (<pane line>; resets <time>; the coding CLI is blocked by a usage limit — add credits, switch model or wait for the reset, then bmad-loop resolve <run>)
```

The attempt is not charged, whatever tokens the session used before the limit hit: the adapter saw the limit live, on screen when the session ended or, for opencode 2.x, as a provider-error event during the session. `<pane line>` is the matched line from the CLI's screen, or opencode's `provider-error …` line. `; resets <time>` appears only when that line names a reset: an ISO-8601 time when it resolves, otherwise the text as printed (for example when the host has no time-zone data for the zone the CLI named).
The attempt is not charged, whatever tokens the session used before the limit hit: the adapter saw the limit live, on screen when the session ended, in a Claude Code `StopFailure` event, or, for opencode 2.x, as a provider-error event during the session. The same holds when the CLI's window died after such an event or during a quota wait, including at its admission or cap (for example when your own `StopFailure` hook kills it): the session ends `crashed` but pauses uncharged, not as a charged crash. A dead window after a Claude Code account error (billing, authentication) pauses the same way, with a `parked:` reason naming the error. `<pane line>` is the matched line from the CLI's screen, or opencode's `provider-error …` line. `; resets <time>` appears only when that line names a reset: an ISO-8601 time when it resolves, otherwise the text as printed (for example when the host has no time-zone data for the zone the CLI named).

To recover:

Expand All @@ -290,6 +290,16 @@ To wait out a session-window reset instead of watching the clock, run `bmad-loop

A session-window limit the CLI waits out by itself does not pause at all at first. While the pane shows that wait and the transcript is idle, the session timeout stands still and no stall nudge is sent, so the session finishes after the CLI continues. `[limits] quota_wait_max_min` (default 300, one five-hour window; 0 = off) caps the minutes one session may spend waiting. Past the cap, or when the parsed reset already lies beyond what is left of it, the session pauses with the reason above. A heartbeat shows the wait as `quota_waiting` / `quota_wait_s` / `quota_resets_at`, and the TUI header as `quota wait`.

**Relaunch instead of pausing.** With `[limits] usage_limit_action = "relaunch"` (default `"pause"`), a dev, fix or review session that ends usage-limited, or on a Claude Code account `StopFailure`, does not pause: the story is relaunched at once and the attempt or review cycle is not charged. Use it when something outside bmad-loop clears the limit between sessions, typically a wrapper that switches accounts (see [Account rotation](plugin-authoring-guide.md#account-rotation)). In this mode:

- the session ends at the limit instead of waiting out a session-window reset, and a Claude Code `rate_limit` `StopFailure` whose message names no known usage limit counts as one;
- a dev or fix relaunch rolls back like any retry (the preserve ref keeps the cut-off work), and a review relaunch re-runs the review;
- the relaunched session's environment carries `BMAD_LOOP_USAGE_RELAUNCH=<n>`, the story's relaunch count so far;
- the decision row reads `usage relaunch (<n>/<cap>): <evidence>` with `uncharged: true`, and the notification `dev relaunch: <story> (usage limit, relaunch <n>/<cap>)` (`review relaunch:` / `fix relaunch:` likewise);
- `[limits] max_usage_relaunches` (default 3) caps the relaunches per story. At the cap the session waits out a session-window reset as above, and a limit that remains pauses uncharged as in pause mode, recovered the same way. A `resolve` re-arm resets the count.

Relaunches skip the `[environment]` probes, and a relaunch cannot fix a Claude Code account error such as `model_not_found`: those fail again on the first turn until the cap stops them.

## Worktree merge-back failures

Under `[scm] isolation = "worktree"`, each story merges back into the target branch in the main checkout. When the merge fails, the unit escalates: its branch `bmad-loop/<run-id>[/<story>]` and worktree are kept, the diff is saved to `.bmad-loop/runs/<run-id>/failed/<unit>/changes.patch`, and the run pauses at `escalation`.
Expand Down
47 changes: 47 additions & 0 deletions docs/plugin-authoring-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -711,6 +711,53 @@ before the gate is classified, so it is never published.
| `pre_session` | before **every** session (after the role-specific `pre_*_session`, so it sees its mutations) | `proposed_prompt`, `proposed_env`, veto |
| `post_session` | after every session | — |

#### Account rotation

bmad-loop does not switch coding-CLI accounts. With `[limits] usage_limit_action =
"relaunch"` it relaunches a story whose session ended on a usage limit without charging
an attempt (see [Usage-limit pauses](escalations-guide.md#usage-limit-pauses)), and the
relaunched session's environment carries `BMAD_LOOP_USAGE_RELAUNCH=<n>`, the story's
relaunch count so far. It is unset until the story's first relaunch, and stays set on
every later dev, fix and review session of that story. Switching accounts is left to you.
Claude Code keeps one account per config directory, so point `CLAUDE_CONFIG_DIR` at a
different one:

- **A wrapper `binary`.** A project profile overlay
(`.bmad-loop/profiles/claude.toml`, a copy of the shipped profile) names your script as
`binary`. The script picks the config directory and `exec`s `claude`.
- **A `pre_session` plugin.** The hook sees the session environment in
`ctx.proposed_env`, `BMAD_LOOP_USAGE_RELAUNCH` included, and sets `CLAUDE_CONFIG_DIR`
there.

Rotate once per relaunch, not on every session that carries the variable: remember the
highest count you rotated on for each run and story (`BMAD_LOOP_RUN_DIR`,
`BMAD_LOOP_STORY_KEY`), and switch only when the count goes past it. A wrapper sketch:

```bash
#!/usr/bin/env bash
accounts=("$HOME/.claude-a" "$HOME/.claude-b") # each logged in once by hand
state="${XDG_STATE_HOME:-$HOME/.local/state}/claude-rotate"; mkdir -p "$state"
idx=$(cat "$state/index" 2>/dev/null || echo 0)
seen="$state/seen-$(basename "${BMAD_LOOP_RUN_DIR:-none}")-${BMAD_LOOP_STORY_KEY:-none}"
n="${BMAD_LOOP_USAGE_RELAUNCH:-0}"
if [ "$n" -gt "$(cat "$seen" 2>/dev/null || echo 0)" ]; then
idx=$(( (idx + 1) % ${#accounts[@]} ))
printf '%s' "$idx" > "$state/index"; printf '%s' "$n" > "$seen"
fi
CLAUDE_CONFIG_DIR="${accounts[$idx]}" exec claude "$@"
```

Before relying on it:

- Each config directory needs its own one-time setup: log in, trust the project
workspace, and accept the bypass-permissions prompt. A directory that skipped a step
parks the session on that dialog.
- Hooks you added to your own user settings (a `StopFailure` hook, for example) live in
one config directory; copy them to every directory you rotate through. bmad-loop's own
relay hooks are in the project's `.claude/settings.json` and work with any directory.
- Never commit, ship or log credentials. Keep the config directories outside the project
tree, and never echo their contents from a hook or wrapper.

### Sweep (deferred-work cycle — bundles inherit all per-story stages)

| Stage | When |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,8 @@ Cycle/attempt caps, timeout & token budget.
| `session_budget_mode` | string | `"warn"` | `enforce` / `off` / `warn` | mid-session guard on the weighted per-session spend, sampled every ~30s · off: no sampling · warn: one ATTENTION + breadcrumb · enforce: wrap-up nudge then over_budget termination (retry→defer) |
| `max_tokens_per_session` | integer | `4000000` | ≥ 1 | weighted (cache-discounted) token cap a single session may spend before the budget guard trips; healthy sessions run ~1-2.5M weighted, so the default trips only true runaways |
| `session_budget_grace_s` | integer | `240` | ≥ 0 | enforce mode: seconds a tripped session gets to wrap up after the nudge before it is terminated over_budget · 0 = terminate at trip, no nudge |
| `usage_limit_action` | string | `"pause"` | `pause` / `relaunch` | what a session that ends on a usage limit or an account StopFailure does · pause: the uncharged usage-limit pause `resolve` clears · relaunch: relaunch it at once as an uncharged retry (BMAD_LOOP_USAGE_RELAUNCH=&lt;n&gt; in the session env, for account rotation), capped by max_usage_relaunches |
| `max_usage_relaunches` | integer | `3` | ≥ 0 | relaunch mode: uncharged usage relaunches per story (dev, fix and review together) · at the cap the session takes the usage-limit pause instead · 0 = always pause |

### `[verify]`

Expand Down
Loading
Loading