Skip to content

feat(catalog): persist, refresh, and report the display model catalog - #401

Merged
caigee-cmd merged 2 commits into
caigee-cmd:mainfrom
yoruuuchan:feat/catalog-auto-refresh
Oct 26, 2026
Merged

caigee-cmd merged 2 commits into
caigee-cmd:mainfrom
yoruuuchan:feat/catalog-auto-refresh

Conversation

@yoruuuchan

Copy link
Copy Markdown
Contributor

Summary

The /v1/models display catalog was in-memory only. That produced three user-visible gaps:

  • Empty list after a restart until the first upstream catalog request succeeded.
  • No scheduled reload — a model an upstream announced reached clients only when someone happened to open the model list after the 5-minute TTL expired.
  • Invisible failures — a failed refresh either returned an empty list (cold cache) or was silently swallowed (warm cache).

This change makes the display catalog durable and observable. Each provider still discovers its own upstream list exactly as before; nothing about model identity, routing, the pool catalog, or the per-account catalogs changes.

  • Persist and restore. The last successful merged catalog is stored in app_secrets under catalog_snapshot_v1 and loaded at boot, so /v1/models answers from disk before any upstream call.
  • Scheduled reload. control.Catalog.RunRefresher reloads the merged view on an interval, started from app.New on the existing shutdown channel. Default 10 minutes; CLI2API_CATALOG_REFRESH_INTERVAL accepts a Go duration (15m) or plain minutes, and 0 disables the loop. Invalid values fall back to the default.
  • Last-known answer survives a failure. A failed refresh no longer empties the catalog: the previously served list is returned and the failure is recorded. The console ?refresh=1 path still blocks for a live fetch.
  • Per-provider health. New control.CatalogHealth records last attempt, last success, model count, last error, and a stale flag for every provider request (CatalogSource reports them; no behavior change).
  • Sync state surfaces. GET /v1/models gains catalog.synced_at and catalog.stale beside data — freshness only, so a provider-scoped API key cannot read another provider's upstream error. The console gets GET /api/model-catalog (full per-provider state) and POST /api/model-catalog (force a refresh).

Positioning against existing work: #142 already keeps pool models through a catalog outage, and #104/#129 own the 5-minute cache and stale-while-revalidate. This change is about the display catalog: durability across restart, scheduled freshness, and sync observability. It does not add a second cache — it reuses control.Catalog and its TTL.

Test plan

  • go build ./...
  • go vet ./...
  • go test ./internal/control ./internal/gateway ./internal/app ./internal/console ./internal/config -count=1
  • New internal/control/catalog_refresh_test.go: persistence round-trip, restore-before-fetch, last-known-after-failure, status reporting, refresher tick, refresher disabled at intervalMs=0, per-provider health transitions.
  • python scripts/release-notes.py validate (changelog fragment catalog-auto-refresh.md, 4 EN / 4 ZH bullets)
  • Backend-only change: internal/webui/static untouched (git diff --stat shows no frontend files).
  • go test ./... — not green on a Windows host on main either; the 19 failing tests are pre-existing POSIX/clock/file-lock assumptions listed below, and CI runs them on Ubuntu.
  • go test -race ./... — no cgo toolchain on this Windows host; CI covers it.

Windows-host test status (pre-existing, unchanged by this PR)

go test ./internal/... fails identically on main and on this branch: internal/api (10: t.TempDir cannot unlink an open SQLite handle, plus a POSIX backup-path assertion), internal/executor (2: Windows clock granularity breaks a strict cooldown After), internal/providers/qoder + internal/runtime (3: POSIX path assertions), internal/store (4: file mode 0666 vs 0600 and cleanup locks).

Live acceptance (real accounts, locally built binary)

Ran the branch build against a copy of a populated data directory with one WorkBuddy CN, one WorkBuddy Global, one QwenWork, and one Codex account, plus two Qoder accounts whose workers cannot start on this host:

Check Result
Startup refresh synced_at set within 2 s of boot; codex 10 models, workbuddy 19, qwenwork 3
Provider failure recorded qoder row: stale=true, last_error = the worker connection refusal; other providers unaffected
Persistence catalog_snapshot_v1 present in qoder.db with the full model list and health rows
Restart with CLI2API_CATALOG_REFRESH_INTERVAL=0 /v1/models and /api/model-catalog served the pre-restart synced_at and models before any fetch — the snapshot came from disk
Manual refresh POST /api/model-catalog advanced synced_at, refreshed every per-provider row, kept the qoder failure
/v1/models metadata {"object":"list","data":[…],"catalog":{"synced_at":"…","stale":false}}
Model count WorkBuddy Global 26 entries with reasoning_options, max_output_tokens, supports_images

Known limitations

  • CLI2API_CATALOG_REFRESH_INTERVAL is process-scoped, like CLI2API_LOG_*. Making it console-editable would need a SystemSettings field and a resettable ticker; not done here.
  • A tick fans out per account: WorkBuddy, Trae, and QwenWork have no internal catalog TTL, so N accounts mean up to N upstream requests per interval. With the default 10-minute interval and a handful of accounts this is small, but a very large pool would want per-provider minimum intervals or skipping idle accounts.
  • The restored snapshot is display metadata only (no tokens); it shares app_secrets with other settings and is included in SQLite backups. Payload size is bounded in practice by the catalog itself (tens of KB).
  • GET /api/model-catalog is a new leaf under /api/; it does not shadow /api/models/{provider}/{model}.
  • Nothing here changes model identity, so it neither helps nor blocks the cross-provider pool work in 希望:增加模型池机制,实现跨 Provider 的模型统一调用与自动切换 #272.

The display catalog was in-memory only: a restart served an empty model
list until the first upstream request succeeded, nothing reloaded it on a
schedule, and a failed refresh was invisible.

- Persist the last successful merged catalog in app_secrets and restore it
  at boot, so /v1/models answers from disk before any upstream call.
- Reload it in the background every 10 minutes (env
  CLI2API_CATALOG_REFRESH_INTERVAL, Go duration or minutes, 0 disables)
  from app.New, reusing the existing shutdown channel.
- Serve the last known list when a forced refresh fails instead of
  returning an empty catalog.
- Record per-provider last attempt, last success, model count, and error
  in control.CatalogHealth so a half-fresh catalog is visible.
- Report sync state: catalog.synced_at / catalog.stale beside data on
  /v1/models (freshness only, no upstream error text for scoped keys) and
  GET/POST /api/model-catalog on the console for the full per-provider
  state plus a manual refresh.

Each provider still discovers its own upstream list; nothing about model
identity, routing, or the pool catalog changes. Lazy TTLs and the console
?refresh=1 path keep working.

@caigee-cmd caigee-cmd left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the right layer for the display catalog: persist the last successful merged view, restore it on boot, keep serving it after a failed refresh, and schedule a background reload. Pool catalogs, model identity, and routing are left alone. Public /v1/models only exposing synced_at / stale (errors stay on the console endpoint) is the correct API-key boundary.

Tests cover persist/restore, last-known-after-failure, the refresher tick, and interval=0. The live-acceptance notes are useful.

Nits, not blockers:

  • catalogRefreshInterval comments say invalid values log a line; the code silently falls back to the default.
  • Storing the snapshot in app_secrets is a reasonable way to avoid a migration. It is not a secret; a later numbered migration can move it if the payload ever grows.
  • The per-account fan-out on each tick is documented. Fine at the default 10 minutes for a small pool.

Please land this before #404 if both stay open — both touch internal/app/app.go.

@caigee-cmd
caigee-cmd merged commit 708e0ac into caigee-cmd:main Oct 26, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants