Skip to content

feat: choose a model's thinking effort per session, on every backend (#417) - #419

Merged
saucam merged 14 commits into
mainfrom
feat/thinking-effort-417
Oct 24, 2026
Merged

saucam merged 14 commits into
mainfrom
feat/thinking-effort-417

Conversation

@saucam

@saucam saucam commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #417

What changes

You can choose how hard a session's model thinks, on every backend, from the web UI, the TUI, Telegram and the CLI.

  • One scale: Default, Off, Minimal, Low, Medium, High, Extra high, Max.
    Each backend maps it to its own setting; docs/thinking-effort.md has a table of what each level means where.
  • Default is the backend's own choice, as it always was.
    Codex and pi keep a level once set, so for them Default is sent as what they'd pick unasked: on Codex your model_reasoning_effort if the running model takes it (a project's only if you trust the project, as Codex itself reads it), else the running model's own default, on pi your own defaultThinkingLevel (else Medium).
  • Per session, like the model.
    It's set at creation or any time after, kept across a daemon restart and a backend switch, and inherited by forks.
    session.defaultEffort (or CODEOID_DEFAULT_EFFORT) applies to new sessions.
    Changing it needs session:send on your own session, and every change is in the audit log (one through a delegated token is marked as such).
  • Never cuts a running turn short.
    • Live: Claude (applyFlagSettings), Qwen (setEffort) and pi (set_thinking_level) take a change on the running backend.
    • Next turn: Codex, OpenAI, llama.cpp and Gemini take it with each request. So do Claude to or from Off, and Qwen back to Default, whose next turn starts a fresh query that resumes the conversation.
    • Restart before the next turn: the Gemini CLI reads thinking only from its settings at startup, so it restarts with the new value before the next turn and resumes the conversation.
  • Each model's own levels where the backend reports them.
    Claude, Codex and pi report them, so the pickers offer only those.
    Everywhere else every level is offered, marked "support unknown", and a level the server refuses fails the turn with a note saying so.
  • Where to set it:
    • web: an effort selector next to the model picker;
    • TUI: /effort opens a picker, or /effort high;
    • Telegram: /effort;
    • CLI: codeoid new --effort <level>.

Two backends run from a settings overlay

  • The Gemini CLI takes thinking only from its settings.
    It runs from an overlay of its usual home that links everything (logins and chats too) except settings.json.
    That copy redefines the two aliases its chat models think through.
  • pi saves any thinking-level change as your global default in ~/.pi/agent/settings.json.
    It runs from an overlay of its agent dir with a private copy of settings.json, so one session's effort never changes your own pi or other sessions.
    Your logins are linked: pi writes them in place, and its OAuth refresh locks the real file.
  • Anything either backend writes in its overlay while it runs (a first sign-in, say) is handed back to the home the overlay is of at a later start, as it would have been written there without codeoid; the newer copy wins.
    A save still in progress (a temp file, or anything changed in the last 10 seconds) waits, and only plain files and directories are moved.
  • Both kinds of overlay live under ~/.codeoid/accounts/ (covered by the secret-path guard) and are removed with their account.
Code detail
  • Protocol: EFFORT_LEVELS / EffortLevel, session.set_effort → EffortChangeResult { effort, applies: "now" | "next-turn" | "restart" }, effort on session.create and SessionInfo, efforts / defaultEffort on ModelInfo.
  • Daemon:
    • Session.setEffort, #applyEffortRestart, TurnOpts.effort at every runTurn;
    • sessions.effort (store) and session.defaultEffort (config);
    • SessionManager.#setEffort, effort on create and fork, _cacheModels keeping efforts.
  • Providers:
    • claude/ — claudeEffortOptions, applyEffort, claudeCatalogEntry;
    • qwen/ — qwenEffort, applyEffort;
    • pi/ — #setThinkingLevel, applyEffort, piThinkingLevels, overlay env;
    • codex/ — turn/start effort via #effortParam, catalog efforts;
    • openai/ — effortParams (default reasoning_effort); llamacpp/ — llamaCppEffortParams;
    • gemini/ — geminiThinkingConfig; acp/ — effortChange: "restart", applyGeminiCliEffort;
    • effort.ts — effortRejectionHint.
  • Overlays (account-apply.ts):
    • applyGeminiCliEffort / geminiEffortSettings;
    • applyPiSettingsOverlay / piDefaultThinkingLevel;
    • ensureOverlayHome now skips lock directories and tolerates concurrent builds;
    • accounts.ts #discardHome trashes an account's overlays.
  • Frontends: packages/core /effort + parseEffort + effortChangeSummary; web EffortPicker; TUI /effort + BackendPicker kind effort; Telegram /effort; CLI --effort.

Design decisions (made with you before implementation)

  • Per session, with a config default (no per-message override).
  • The Gemini CLI included, by restarting it.
  • Models with unknown levels: offer every level, and surface a server's refusal.

Verification

Live (isolated daemon on this branch):

  • Claude: created at Off, a turn had no thinking blocks; set to High, the next turn thought. A change from High to Low applied immediately. The model list carries Claude's per-model levels.
  • Qwen: a live change applied immediately.
  • Codex: turn/start accepts effort (and null). config/read returns your model_reasoning_effort, and thread/start the model and effort the config gives. config/read for an untrusted project ignores the project's .codex/config.toml effort; once the project is trusted, it returns it. Model turns themselves can't run on this machine's ChatGPT login.
  • pi:
    • A live level change was accepted, and the model list shows per-model levels.
    • After the fix, ~/.pi/agent/settings.json stayed unchanged while the overlay's copy took the write.
    • Turns produced no reply: pi has no credentials on this machine.
  • Not run live: the Gemini CLI (expired login), plain OpenAI, Gemini API, llama.cpp. Their mappings and request bodies are covered by tests.

Tests:

  • src/tests/thinking-effort.test.ts: each backend's mapping, the overlays, and the session path (create, set, persist, resume, apply modes, restart before the next turn, forks, scopes, tenancy, audit, catalog, config default).
  • Effort cases added to the Claude, Codex, OpenAI and pi provider suites, with their fakes extended (Codex turn/start effort + config/read; pi thinking level that writes its settings as real pi does).
  • Each audit fix was mutation-checked: reverting it makes a test fail. Survivors are checks covered twice (lock entries filtered in both passes; the file-versus-directory skip and the staged-copy cleanup, which fail together), and one equivalent change (renaming a directory over an empty one loses nothing).
  • The cross-filesystem copy is tested across two real filesystems (/tmp and /dev/shm; skipped where they're one).
  • bun run typecheck, bun run lint and web tsc --noEmit are clean; the full suite is 3744 pass, 0 fail.
  • No regression tier: codeoid has no highflame-regression coverage.

Audit rounds (correctness + security each round, until one came back clean):

  • Round 1 (1bdc917): both go-with-fixes.
    pi's level change wrote your global pi default; Codex Default didn't reset an override; Gemini 3 Pro levels.
    Fixed in cc8ec1e.
  • Round 2: both go-with-fixes.
    High: my Codex fix sent the catalog's default model's effort over your own config; pi overlay login linking.
    Fixed in da48f6b.
  • Round 3: no High.
    Four correctness Mediums (Codex config cached for the process, fork path model, Codex's own lock file no longer linked, EEXIST on pi's auth.json); security Lows only.
    Fixed in 82f10cb.
  • Round 4: no Critical or High.
    One Medium in both lanes: pi's Default read a project's .pi/settings.json, which pi honours only for a trusted project. Also a fork at Default taking the config default, and a lock-race edge.
  • Rounds 5–8: no Critical or High; Mediums and Lows on Default semantics per model (Codex's configured effort only where the model takes it), pending-state display, waitsForBackground, pickers dropping a Default the model can't take, pi's agent dir required absolute.
    Fixed through 38d27c5.
  • Round 9: no Critical, High or Medium. Lows: a stray test file, a relative GEMINI_CLI_HOME, a first Gemini sign-in stranded in an overlay, a restart left pending across a backend switch or a reverted change. Fixed in 751a46e.
  • Round 10: security go; correctness two Mediums in the new overlay hand-back (a temp file mid-save in another session; a pi login held by an earlier overlay). Fixed in 564d5ca.
  • Round 11: both go. Lows in the cross-filesystem copy (a FIFO could block, a planted link at a guessable staged name, a partial copy left behind). Fixed in 765ef4f.
  • Round 12: both go, no findings at any severity.
    The one open question (does Codex's config/read honour project trust?) was checked live and is answered above.

Known, accepted (in docs/thinking-effort.md):

  • A pi sign-in or sign-out made in codeoid's pi and in your own at the same moment isn't serialized; OAuth refresh is.
  • On Codex, a model its catalog doesn't list, with no effort configured, has no known default, so Default sends nothing there.
  • A project's .gemini/settings.json that redefines the chat aliases overrides the effort on the Gemini CLI.
  • An overlay built by an earlier commit of this branch that holds a pi login, when your own pi dir holds a different non-empty one, keeps only the newer file (no per-provider merge). Only someone who ran this branch can have one.
  • The Gemini CLI and pi overlays of your own home stay (each with a copy of that tool's settings); agents can't read them.

Found and filed separately: #418 (a Codex turn right after its backend restarts can get the old process's exit error).
Oracle: n/a, codeoid isn't in enabled_repos.

🤖 Generated with Claude Code

saucam and others added 13 commits October 24, 2026 14:54
…417)

One scale (Default, Off, Minimal, Low, Medium, High, XHigh, Max), mapped to
each backend's own setting:

- Claude: the Agent SDK's `effort` on adaptive thinking (Off disables
  thinking), changed live with applyFlagSettings; to or from Off rebuilds
  the query, which resumes the conversation.
- Codex: `effort` on every turn/start (none = null: codex keeps a turn's).
- OpenAI: `reasoning_effort`; llama.cpp: the chat template's
  `enable_thinking` / `reasoning_effort`; Gemini: `thinkingConfig` (a budget
  for 2.5, a level for 3.x).
- Qwen: `effort`, live with setEffort; pi: `set_thinking_level`, live.
- Gemini CLI: a settings overlay of its usual home redefining its two chat
  aliases' thinking; it takes it only at startup, so a change rebuilds it
  before the next turn.

Per session like the model: `session.set_effort` (session:send), effort on
`session.create` and SessionInfo, persisted (sessions.effort), and
`session.defaultEffort` / CODEOID_DEFAULT_EFFORT for new sessions. Never
cuts a running turn short. The model list carries each model's efforts where
the backend reports them (Claude, Codex, pi); elsewhere every level is
offered, and a refused one fails the turn with a note.

Web UI: an effort selector next to the model picker; TUI: /effort and a
picker; Telegram: /effort; CLI: `codeoid new --effort`.
docs/thinking-effort.md says what each level means on each backend.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s level writes stay out of the user's settings; Gemini levels each model takes

- pi saves a thinking-level change as the user's global default: it now runs
  from an overlay of its agent dir whose settings.json is a fresh copy at
  each start (logins and sessions linked); Default is the user's own
  defaultThinkingLevel, else Medium — not whatever a resumed session had.
- Codex keeps a turn's effort and reads null as "no override": Default now
  sends the model's own default effort from model/list (the default model
  marked too, so pickers find its levels).
- Gemini: 3 Pro takes only Low/High; the Gemini CLI's overlay uses values
  every model takes (2.5 budgets 512–24576, 3.x Low/High); its errors get
  the effort hint.
- Claude: a live change also reaches a retry later in the same turn.
- A pending Gemini CLI rebuild is dropped after a switch to another
  backend; the same level again while it's pending says so.
- Forks inherit the effort; creation and delegated changes are audited.
- Two sessions building the same overlay at once no longer fail.
- Docs: what Default means on each backend; per-model Gemini values; pi's
  overlay; project Gemini CLI settings outrank the effort.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…'s overlay links logins made later; overlays go with their account

- Codex: Default now sends what codex would use unasked — the user's
  model_reasoning_effort (config/read, project layers included), else the
  running thread's model's own default — instead of the catalog's default
  model's, which overrode the user's config. The catalog no longer marks a
  default model (the configured one may differ).
- pi: auth.json is made in the user's dir when missing (as pi does) so the
  overlay always links it, and an overlay holding an empty one of its own is
  relinked; the level is sent again after a model change (pi re-applies its
  own); a ~ in PI_CODING_AGENT_DIR is read as pi does.
- Overlays: lock entries are never linked; a link removed by a concurrent
  build is tolerated; an account's pi and Gemini CLI overlays are trashed
  with it.
- Docs: Codex's Default; pi's lock limit with a pi you run yourself.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ock directories kept out of overlays; pi Default reads project settings

- Codex: the configured effort is read again for each thread (start,
  resume, fork, inject), as codex reads its config; the fork path records
  the thread's model; a turn that names a model moves the baseline to it.
- Overlays skip only a lock DIRECTORY (proper-lockfile's, held right now);
  a lock file such as codex's .sqlite-maintenance.lock is shared again.
- pi: making auth.json tolerates a dangling link or one made meanwhile;
  Default reads the project's .pi/settings.json first, as pi layers it; an
  account overlay's source expands ~ too.
- Docs: pi's OAuth refresh does share the lock (only sign-in/out doesn't);
  Codex's Default for a model its catalog doesn't list.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…k at Default stays at Default

- pi: Default is the user's own defaultThinkingLevel; a project's
  .pi/settings.json is no longer read — pi honours it only for a project
  the user trusts, and a repo mustn't choose the level (and the cost).
- A fork of a session at Default stays at Default rather than taking
  session.defaultEffort (an explicit null at creation is kept).
- An overlay entry named .lock that vanishes between the listing and the
  check is a lock just released: not linked.
- Wording: the Gemini CLI restarts between turns once no work is running;
  pi's lock note in the code matches the docs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ings again; Codex marks the model it runs unasked

- Claude: back to Default rebuilds the query before the next turn instead
  of a live "no level", which the SDK takes as the model's default rather
  than the user's own Claude Code effort setting.
- Codex: the catalog marks the model codex runs when none is chosen (the
  configured one, else its own default), so a picker shows its levels.
- session.create takes effort: null for the backend's default (and
  `codeoid new --effort default` sends it), rather than falling to
  session.defaultEffort.
- The secret-path guard reads PI_CODING_AGENT_DIR's ~ as pi does.
- Claude's reported levels are kept to codeoid's scale.
- Wording: the restart message every frontend shows; docs on what each
  backend reads at Default (Qwen's project settings, Claude's own).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ackground work; Codex's catalog shows Default as it really is

- A change the backend can't take live (and so rebuilds it) waits while
  the session has background work running, so that work isn't stopped; it
  lands on the first turn with none. The same level set again while a
  change is still on its way says so.
- Codex catalog: a configured model it doesn't list leaves no row marked
  default (not even its own); a configured effort is every row's default,
  as Default sends it.
- pi's agent dir may be a file:// URL, as pi accepts.
- CLI help names --effort default; docs: Claude reads project settings too;
  a rebuild waits for background work.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ffort change is shown as pending and announced when it lands

- Codex: the session reports what Default means for it (the configured
  effort, project config included, on the model the thread runs) and that
  model; the pickers prefer these to the daemon-wide catalog, which stays
  the user's own config (no project can change what other sessions see).
  A configured effort shows as a model's default only where the model
  takes it. pi reports the user's default the same way.
- A change held back by background work: SessionInfo.effortPending, shown
  in the web, TUI and Telegram pickers; the message points at Stop; an
  info message when it lands.
- piAgentDirOf never throws on a malformed file:// URL (the secret-path
  guard calls it on every tool call).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…where it's shown or sent; pi's agent dir must be absolute

- Codex: a configured effort counts as Default only for a model that takes
  it (else the model's own) — the same rule for the catalog, the session's
  report, and what turn/start sends.
- The session's Default is shown only while it's for the model in use
  (stale after a model change until the next turn); pickers drop a Default
  the model's known levels don't include; the TUI picker follows the
  session while open; runningModel is bounded.
- /effort's reply says when a change waits for background work
  (EffortChangeResult.waitsForBackground).
- pi: an agent dir that isn't absolute (or an unparseable file:// URL) is
  refused rather than building pi's logins under the daemon's own dir.
- Doc comments back on their own members.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…oes back to the user's home; no stale restart

- Settings overlays adopt what the backend wrote in them (a first sign-in,
  a file saved by replacing our link): moved into the user's home before
  the next start, unless the user's own is newer.
- GEMINI_CLI_HOME must be absolute, as PI_CODING_AGENT_DIR already is.
- A Gemini CLI restart still to come is dropped by a backend switch, and
  by changing back to the level the backend runs at.
- Removed relative/pi/auth.json, a test leftover from before round 8's
  guard.
- Docs: Codex's Default takes the configured effort only for a model that
  takes it; what the picker shows before a first turn; overlay adoption
  and the default-home overlays that stay.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… earlier overlay held

- Adoption skips temp files (the Gemini CLI's projects.json.<uuid>.tmp,
  trustedFolders.json.tmp.<uuid>, codeoid's own) and anything changed in
  the last 10s: another session's CLI may be running in the same overlay
  and still writing it.
- A pi auth.json an earlier overlay kept as its own goes to the user's
  home when that has none, or only codeoid's empty stand-in, instead of
  being dropped for the stand-in.
- A tie in mtime goes to the backend's write; across filesystems the
  entry is copied into place rather than left hiding the source's; a move
  that fails is logged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…es, and never leaves a partial copy

- Only regular files and directories are handed back: copying a FIFO
  across filesystems would block the daemon.
- A file and a directory of one name never replace each other.
- The cross-filesystem copy is staged under a fresh name, created
  exclusively (never through a planted link), and removed if the copy
  or the rename into place fails.
- A move that keeps failing is logged once per entry, not at every start.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ted project, as Codex does

Checked live against codex 0.154: config/read with an untrusted cwd
returns the user's model_reasoning_effort and ignores the project's;
with the project trusted, it returns the project's.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@saucam
saucam merged commit 89d1676 into main Oct 24, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Choose a model's thinking effort per session, on every backend

2 participants