From e6ee4ace237731f4f2590518019b3e89bf812200 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 11:35:54 +0000 Subject: [PATCH 01/62] Plan M017 managed client credentials corrective --- ...anaged-client-credential-reconciliation.md | 103 ++++++++++++++++++ 1 file changed, 103 insertions(+) create mode 100644 plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md diff --git a/plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md b/plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md new file mode 100644 index 00000000..097a554f --- /dev/null +++ b/plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md @@ -0,0 +1,103 @@ +# On-Prem User Access M017 — Managed client credentials and integration reconciliation + +Status: ready + +Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, PR #7 merged). + +Source roadmap: `plans/subsystems/on-prem-user-access-roadmap.md#m017--managed-client-credential-reconciliation` + +Long-term requirements: `plans/000-long-term-specification.md#1-product-definition`, `plans/002-long-term-roadmap.md#phase-5--optional-on-prem-small-team-access-and-budgeting`. + +Applicable ADR: `plans/adrs/ADR-0002-on-prem-user-identity-and-policy-boundaries.md` (accepted). + +Primary class: capability + +## 1. Objective + +Restore a complete fresh-install and managed client-configuration path without relying on a retired shared server key. A solo developer's first inference and a small-team member's remote desktop must use valid individual credentials. This is a post-merge follow-up to M006/M007 and M015/M016; preserve those closure records. + +## 2. Readiness and dependencies + +Hard: none; M001–M016 code is on `main`. Interface: existing `epk1` personal tokens, secret-free `epc1` profiles, `EGGPOOL_API_KEY`, integration-profile endpoint, and `eggpool-client-config` projection are stable. Operational: run a synthetic end-to-end CLI-to-agent smoke test before closure. + +## 3. Current evidence + +`rust/src/runtime.rs::onboard` generates `[server].api_key` then bootstraps an admin personal token. `rust/src/operations/integrations.rs::{build_context,build_endpoint_context}` calls `config_mutation::resolve_server_key`, which manufactures a shared key when absent. `build_remote_context_from_config` rejects a keyless managed instance. `runtime.rs::getkey`, README First Request and client hints still direct operators to `eggpool getkey`. `rust/src/server/middleware.rs` rejects the shared key after local identity initialization. The M006/M007 qualification did not traverse this full post-bootstrap path. + +## 4. Invariants + +Provider accounts, upstream secrets and routing stay in TOML/env; human state stays in SQLite. No personal token becomes a web-admin session, no credentials in secret-free desktop profiles, no implicit shared-key fallback in managed mode. Keep legacy authentication for `legacy_unmigrated`, Chat/Responses/compact/Messages, user-filtered discovery, generation leases, single SQLite gate/WAL/NORMAL and budget/finalization ownership. + +## 5. Scope + +In: `onboard`, `getkey/newkey` compatibility, all `configsetup` targets, `configremote`, `eggpool-connect`, onboarding and README instructions. Out: storing recoverable personal token plaintext, putting member credentials in connection tokens, new token protocol, IAM/SSO, provider routing changes. + +## 6. Production changes + +Make integration credential resolution aware of access state. In managed/admin-initialized installations, require an explicit personal credential or a safe local operator-issued one-time token, rather than `resolve_server_key`. Shared-key CLI commands must clearly report the distinction between legacy credentials and personal tokens. Export of remote configuration must not depend on the legacy key's presence and must remain read-only. + +## 7. Ordered work packages + +### A — Freeze authentication matrix + +For legacy-unmigrated, fresh-uninitialized, admin-initialized and managed states, define which commands can generate, show and use which credential. Select a non-mutating mode authority. Reject stale server-key use without silently changing the credential. + +Acceptance: contract tests for every state and a zero-secret export. + +### B — Reconcile adapters + +Change `operations/integrations.rs` and `runtime.rs` clients; preserve `EGGPOOL_API_KEY` interpolation and existing Codex/OpenCode output shapes. Cover Claude Code and other supported generated configurations. Make `configremote` function without a global server key and keep the profile token credential-free. + +Acceptance: remote desktop bootstrap, personal-token discovery, profile revision and revocation all work. + +### C — Test full user journeys + +Isolated HOME/TOML/SQLite/fake upstream: fresh onboard → initial personal token → configsetup → successful model discovery/inference; separate member credential → configremote → desktop helper → authorized models; revoke token and deny next request. Compare legacy pre-bootstrap flow. + +Acceptance: no post-bootstrap 401 from following documented first-request steps. + +## 8. Failure, cancellation, restart and contention + +No export mutates TOML or regenerates the retired shared key. On missing/revoked token, fail with actionable error; no anonymous or admin fallback. A changed user policy revision invalidates discovery; concurrent token issuance cannot leak or cross-attribute a credential. No DB gate held during external client setup. + +## 9. Compatibility and migration + +No database migration anticipated. Preserve `epc1`, `EGGPOOL_API_KEY`, personal bearer/`x-api-key` wire, client config ownership/drift semantics and legacy installations. If changing auth authority beyond ADR-0002, stop and propose an ADR. + +## 10. Tests + +Extend `rust/tests/cli_contract.rs`, `rust/tests/on_prem_identity.rs`, `rust/tests/on_prem_sessions.rs`, `rust/tests/on_prem_model_policy.rs`, and client-config/connect crate contract suites. Add a synthetic cross-component CLI/inference/remote fixture. Cover keyless managed, stale configured server key, legacy key, two principals, profile filtering, token revocation and redacted stdout. + +## 11. Verification commands + +```bash +cargo fmt --manifest-path rust/Cargo.toml --all -- --check +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings +cargo check --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- -D warnings +cargo test --manifest-path rust/Cargo.toml --workspace --all-targets -- --test-threads=1 +cargo test --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- --test-threads=1 +uv run pytest tests/tooling/ -q --tb=short --maxfail=1 +uv run python scripts/validate_release_docs.py +git diff --check +``` + +## 12. Documentation + +Update `README.md`, `docs/agent-configuration.md`, `docs/cli-reference.md`, `docs/deployment.md`, `architecture/deep-dive-integrations.md` and embedded paste hints; add a regression guard for obsolete post-bootstrap getkey instructions. + +## 13. Acceptance criteria + +Fresh solo onboarding produces a usable agent credential; all local/remote client configuration survives the managed transition without regenerating or requiring a legacy shared key; two members remain isolated; legacy single-key upgrades remain functional until operator cutover. + +## 14. Stop conditions + +Stop on cross-user secret disclosure, implicit key resurrection, broadened browser/admin privileges, or an incompatible client-profile schema change without an accepted decision. + +## 15. Closure evidence + +Create `plans/closure/on-prem-user-access/017-status.md` with exact implementation SHAs, a requirement-to-evidence matrix, executed test/CI evidence, negative auth/revocation cases, compatibility and docs review, severity-tagged residuals and unblock audit. No closure on unit tests alone. + +## 16. Handoff + +P1 integration corrective. Ensure the single-user first request is tested as a user journey, not inferred from separately passing auth and CLI tests. From 4f3f6e0d4692c5f29ad5cc735ae10775a927abd8 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 11:36:37 +0000 Subject: [PATCH 02/62] Plan M018 local operator status corrective --- .../018-local-operator-status-corrective.md | 102 ++++++++++++++++++ 1 file changed, 102 insertions(+) create mode 100644 plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md diff --git a/plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md b/plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md new file mode 100644 index 00000000..031350c8 --- /dev/null +++ b/plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md @@ -0,0 +1,102 @@ +# On-Prem User Access M018 — Local operator status after identity cutover + +Status: ready + +Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). + +Source roadmap: `plans/subsystems/on-prem-user-access-roadmap.md#m018--local-operator-status-corrective` + +Long-term requirements: `plans/000-long-term-specification.md#2-end-state-invariants-normative`, `plans/002-long-term-roadmap.md#phase-4--operations-integrations-and-deployment`. + +Applicable ADR: `plans/adrs/ADR-0002-on-prem-user-identity-and-policy-boundaries.md` (accepted; new authorization boundary requires ADR evaluation). + +Primary class: capability + +## 1. Objective + +Restore `eggpool status` and `eggpool runtime-status` for the local operator after administrator bootstrap without relaxing browser-only privileged HTTP authorization. Preserve solo/headless usability and both text/JSON contracts. + +## 2. Readiness and dependencies + +Hard: none. Interface: existing owner-protected local control socket, process lifecycle and status aggregation. Inspect actual peer/permission authority before extending the socket; if the proposed privilege boundary differs from ADR-0002, obtain an ADR before implementing. + +## 3. Current evidence + +`rust/src/runtime.rs::{fetch_local_json,status,runtime_status}` authenticates HTTP `/api/status` and `/api/stats/runtime` using `Config::resolved_server_api_key()` or no credential. `rust/src/server/middleware.rs` demands a browser admin session after bootstrap, so a healthy managed daemon can return 401. `rust/src/operations/control.rs` already owns local privileged rehash; `rust/src/operations/status.rs` owns the status contract. M006/M016 did not test this exact mode transition. + +## 4. Invariants + +A personal agent token cannot access privileged HTTP status. No unauthenticated loopback managed-mode exemption, no TCP admin backdoor, new daemon, second DB writer, or shared key revival. Preserve JSON schema, readiness semantics, status classifications, redaction and single-gate SQLite. + +## 5. Scope + +In: a bounded authorized local operator status/readback path and CLI adapter. Out: web/admin role changes, unrestricted local HTTP privilege, remote operator shell, provider health redesign or any automatic mutation when reading status. + +## 6. Production changes + +Prefer a narrow read-only request on the existing OS-owned control channel, with explicit socket ownership/peer validation, protocol version and bounded payload, reusing current status/runtimestat projection. Keep HTTP endpoints unchanged. Local CLI should accurately distinguish unauthorized, absent socket, stale PID and not-ready daemon; static offline snapshots must be labeled offline. + +## 7. Ordered work packages + +### A — Inventory OS authority and current status owner + +Inspect control socket permissions, peer credentials, lifecycle/service users and transport reliability on Linux/macOS. Decide whether current OS-authorized local commands can obtain the status projection without new administrative reach. If new auth semantics require an ADR, block before code. + +Acceptance: documented socket authorization and threat cases. + +### B — Implement bounded local readback + +Extend control status interface and `runtime.rs` CLI as appropriate; cap response bytes and timeouts; preserve `--json` schema and nonzero error exits. Keep control request handlers thin and do not tie the control loop to provider I/O. + +Acceptance: local operator can query running admin_initialized/managed and legacy instances. + +### C — Integrate failure and restart matrix + +Exercise wrong-owner socket, missing/stale socket, malformed/truncated/oversized response, daemon down, restart, rehash, role isolation and readiness degradation. Include a systemd-like filesystem privilege fixture without touching a live deployment. + +Acceptance: no error is silently treated as readiness, no unmanaged auth fallback. + +## 8. Failure, cancellation, restart and contention + +Bound requests; avoid holding the SQLite writer gate while waiting for network/provider activity. Stale runtime identity and process restart must not make a prior socket appear authoritative. Fail closed on unexpected peer/protocol, not by sending a shared key over loopback. + +## 9. Compatibility and migration + +No schema change anticipated. Preserve existing HTTP `/api/status` role checks and prior legacy flow. A read-only local status endpoint is not a user-facing inference credential or authorization scope. Escalate to an ADR if current socket file permissions cannot safely serve as operator authority. + +## 10. Tests + +Extend `rust/tests/cli_contract.rs`, `rust/tests/on_prem_sessions.rs`, `rust/tests/server_transport.rs` and existing control/lifecycle tests; new end-to-end status CLI fixture covers pre/post-bootstrap, no configured server key, malformed socket and restart. Assert unauthorized user tokens cannot read privileged HTTP admin stats. + +## 11. Verification commands + +```bash +cargo fmt --manifest-path rust/Cargo.toml --all -- --check +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings +cargo check --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- -D warnings +cargo test --manifest-path rust/Cargo.toml --workspace --all-targets -- --test-threads=1 +cargo test --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- --test-threads=1 +uv run pytest tests/tooling/ -q --tb=short --maxfail=1 +git diff --check +``` + +## 12. Documentation + +Update `docs/cli-reference.md`, `docs/deployment.md`, `architecture/deep-dive-control.md` and `architecture/deep-dive-security.md`; explain local operator permissions and failure modes. + +## 13. Acceptance criteria + +A local authorized operator sees live status/runtime diagnostics after fresh bootstrap and managed transition. JSON schema and exit behavior remain compatible; HTTP member/API-key access to privileged status remains forbidden. + +## 14. Stop conditions + +Stop if local socket peer identity/permissions cannot be verified, if operation would grant network privilege, or if correctness requires redefining the accepted auth ADR without review. + +## 15. Closure evidence + +Record exact SHA, permission/threat review, serialized Rust/no-default CI, process restart/negative-path evidence, stability of CLI JSON and unresolved findings in `plans/closure/on-prem-user-access/018-status.md`; audit blocked-work promotions. + +## 16. Handoff + +Independent P1 corrective; may proceed parallel with M017. Do not solve this by granting personal inference tokens the admin dashboard/stats scope. From 9b35316e9f86bceb5c430ba33d4575fa4d9c5d8c Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 11:37:20 +0000 Subject: [PATCH 03/62] Plan M019 secure login GET and solo LAN corrective --- ...019-login-transport-solo-lan-corrective.md | 102 ++++++++++++++++++ 1 file changed, 102 insertions(+) create mode 100644 plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md diff --git a/plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md b/plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md new file mode 100644 index 00000000..26cbd1a5 --- /dev/null +++ b/plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md @@ -0,0 +1,102 @@ +# On-Prem User Access M019 — Login transport and solo LAN operability corrective + +Status: ready + +Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). + +Source roadmap: `plans/subsystems/on-prem-user-access-roadmap.md#m019--login-transport-and-solo-lan-operability` + +Long-term requirements: `plans/000-long-term-specification.md#1-product-definition`, `#2-end-state-invariants-normative`; `plans/002-long-term-roadmap.md#phase-5--optional-on-prem-small-team-access-and-budgeting`. + +Applicable ADR: `plans/adrs/ADR-0002-on-prem-user-identity-and-policy-boundaries.md` (accepted). + +Primary class: invariant + +## 1. Objective + +Prevent a remote plaintext HTTP GET from soliciting credentials that the POST handler later rejects, and document a secure, low-friction solo SBC browser path while retaining managed-team HTTPS requirements. + +## 2. Readiness and dependencies + +Hard: none. Interface: existing `web_auth_transport_allowed`, trusted proxy peer/scheme detection, same-origin/CSRF, and dashboard routes. No native TLS stack or certificate manager is authorized by this plan. + +## 3. Current evidence + +`rust/src/server/mod.rs::build_router` mounts `GET /login` at `rust/src/server/auth.rs::login_page`. The GET handler renders a form unconditionally; `server/middleware.rs::is_unconditionally_public` bypasses authentication. POST `login_submit` rejects remote HTTP before verifying passwords, but the browser has already sent the password unencrypted. Managed sessions also make the legacy `dashboard.public` default insufficient for ordinary LAN dashboard access; README and Pi instructions are currently misleading. + +## 4. Invariants + +Admin/password/session browser routes do not become public on LAN or loopback by accident; only explicit loopback or authenticated HTTPS-proxy transport may carry password forms. Never trust arbitrary Host, `Forwarded` or `X-Forwarded-Proto`. Keep HttpOnly/SameSite/Secure-cookie behavior, CSRF, separate agent credentials and no additional required daemon for solo use. + +## 5. Scope + +In: gate `GET /login`, safe transport explanatory page and accurate operator docs/onboarding hints. Out: implementing native TLS, enabling insecure HTTP by default, automatically generating or trusting CA roots, putting TLS or browser auth in the inference path. + +## 6. Production changes + +Reuse one canonical transport predicate for GET/POST using actual peer scheme and configured proxy trust. For rejected GET serve a static no-password no-username HTML explanation or redirect only to an operator-configured trusted HTTPS URL; never infer redirect destination from untrusted request headers. Render an explicit solo remote-headless option using SSH port forwarding to local loopback; ordinary teams use operator-managed TLS reverse proxy/VPN. + +## 7. Ordered work packages + +### A — Prevent credential solicitation on unsafe GET + +Change `login_page` signature to obtain connection/header/server context; gate forms using the same policy as POST. Ensure a rejected GET contains no password input and cannot set a browser session. + +Acceptance: remote direct HTTP GET and POST reject safely, loopback GET/POST work. + +### B — Test proxy/scheme and browser contracts + +Build negative tests for spoofed, duplicated, malformed forwarding headers, direct bypass around proxy, wrong Host, same-origin mismatch; positive tests for configured HTTPS peer, loopback, cookie security, CSRF, M011 dashboard shell/parity. + +Acceptance: non-spoofable transport checks with browser tests on the rendered HTML. + +### C — Give solo and server operators supported paths + +Update first-run help, README/Pi guide, firewall/deployment examples. Clearly distinguish LAN HTTP inference tokens from browser password safety, local loopback browser, `ssh -L` on a Pi, and TLS termination for on-prem teams. Preserve the development-only insecure override with a warning. + +Acceptance: a headless personal Pi can reach dashboard without installing any extra Eggpool component; secure ordinary-server team path explained. + +## 8. Failure, cancellation, restart and contention + +An unknown connection or untrusted proxy must not display a credential form. Avoid database access on rejected GET. Security decision must not depend on DNS reachability or cached forwarded headers. Browser sessions remain invalid after disable/revocation/restart as currently documented. + +## 9. Compatibility and migration + +No schema, server protocol or API key changes. Legacy public read-only dashboard rules before initialization remain as implemented; initialized pages remain private regardless of `dashboard.public`. An operator opting into the explicit insecure developer override retains warning/flag semantics. + +## 10. Tests + +Extend `rust/tests/on_prem_sessions.rs`, `rust/tests/server_transport.rs`, `rust/tests/on_prem_admin_ui.rs` and `tests/tooling/test_dashboard_parity_on_prem.py`. Assert remote HTTP GET produces zero password fields; POST rejects; loopback and trusted proxy positive flows work; cookie flags and CSRF unchanged. + +## 11. Verification commands + +```bash +cargo fmt --manifest-path rust/Cargo.toml --all -- --check +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings +cargo check --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- -D warnings +cargo test --manifest-path rust/Cargo.toml --workspace --all-targets -- --test-threads=1 +uv run pytest tests/tooling/test_dashboard_parity_on_prem.py -q +uv run python scripts/validate_release_docs.py +git diff --check +``` + +## 12. Documentation + +`README.md`, `docs/raspberry-pi.md`, `docs/deployment.md`, `docs/firewall.md`, `architecture/deep-dive-security.md`; avoid promising plain HTTP browser login on a non-loopback LAN address. + +## 13. Acceptance criteria + +Insecure remote GET never presents a password form; insecure POST rejects without authenticating; loopback and correctly proxied HTTPS browser login work; solo SBC has clear documented path without sacrificing secure defaults. + +## 14. Stop conditions + +Stop for an ADR if transport privilege or cookie security meaning changes, or if proposed solution enables browser authentication on arbitrary RFC1918 peers as an undocumented bypass. + +## 15. Closure evidence + +Add `plans/closure/on-prem-user-access/019-status.md` with HTML and transport negative matrix, browser evidence, exact SHA, serial/no-default and hosted test outcomes, docs/security review and blocked-work audit. + +## 16. Handoff + +P1 security + solo ergonomics correction. POST rejection alone does not protect a user who has already submitted an HTTP login form. From 1994fe3407c9f7ad29d947f18c0cfd82376b0884 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 11:38:11 +0000 Subject: [PATCH 04/62] Plan M020 managed state and headless administration --- .../020-managed-transition-headless-admin.md | 101 ++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md diff --git a/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md b/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md new file mode 100644 index 00000000..83a91d25 --- /dev/null +++ b/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md @@ -0,0 +1,101 @@ +# On-Prem User Access M020 — Managed transition consistency and headless administration + +Status: blocked + +Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). + +Source roadmap: `plans/subsystems/on-prem-user-access-roadmap.md#m020--managed-transition-and-headless-administration` + +Long-term requirements: `plans/000-long-term-specification.md#1-product-definition`, `#2-end-state-invariants-normative`; `plans/002-long-term-roadmap.md#phase-5--optional-on-prem-small-team-access-and-budgeting`. + +Applicable ADR: `plans/adrs/ADR-0002-on-prem-user-identity-and-policy-boundaries.md` (accepted). Escalate any changed security semantics for ADR review. + +Primary class: capability + +## 1. Objective + +Make first-member transition and small-team management consistent across CLI/browser while keeping one-admin solo installation canonical. Enable headless operators to create, disable/reactivate, inspect and provision members without enabling dashboard browser administration. + +## 2. Dependencies and readiness + +Hard: M017 credential semantics and M019 browser-security closure, so the transition cannot strand users with invalid tokens or unsafe management guidance. Interface: `identity.rs` DB transactions, `model_policy.rs`, CLI user operations, and session fences. No hardware dependency. + +## 3. Current evidence + +`rust/src/runtime.rs::users` refuses `users enable-managed` when a shared key is configured or `dashboard.public = true`. `rust/src/identity.rs::create_member_with_fence` transitions the DB to `managed` directly when a member is created without those prerequisites. All initialized modes already reject the legacy shared key through middleware. `rust/src/cli.rs::UsersCommand` includes bootstrap/list/token/revoke/reset-password/policy/budget but no `add`, `disable`, `enable` or `show` member commands. Browser admin can create/disable members. + +## 4. Invariants + +Managed users never authenticate using legacy shared key, even if stale configuration remains. Bootstrap remains local with non-default password and one initial admin. Browser writes preserve CSRF/session fencing; CLI writes require OS-owned data-directory authority and do not accept passwords in argv. Never create public registration, provider web secrets, hosted groups or a second DB worker. + +## 5. Scope + +In: reconcile managed state machine, CLI member lifecycle commands and consistent first-member provisioning semantics. Out: adding arbitrary remote administrators, public sign-up, changing role hierarchy, native TLS or new global policy DSL. + +## 6. Production changes + +Define one state-transition service used by CLI and browser. Distinguish a safe functional auth cutover from *configuration hygiene* (removing a now-dead shared key). Decide whether `admin_initialized` and `managed` are semantically distinct; retain compatible stored states and ensure any automatic transition is atomic with member creation/profile assignment/audit. Add local CLI `users add`, `users disable`, `users enable` and `users show` using existing identity services and actor validation; support TTY/password-stdin and non-TTY secret handling. Do not recreate identities on retries. + +## 7. Ordered work packages + +### A — State-table review and reconciliation + +Freeze actual startup/middleware transitions for legacy, fresh, initialized and managed modes. Unify `enable-managed` and first-member creation preconditions and status messages. Require a plan or ADR if changing a public auth promise. + +Acceptance: tests demonstrate same security gates irrespective of CLI/browser trigger, with stale config requiring explicit cleanup but never allowing old key. + +### B — Headless member lifecycle + +Add CLI member create/enable/disable/show with bounded username/role, optional profile, and protected temporary password via TTY/FD/stdin. Reuse transactional privilege/fencing and mutation audit; preserve last-admin protections. Avoid a CLI path that bypasses initial profile or password-change requirements. + +Acceptance: headless workflow creates member, logs in, issues personal token, restricts/disabled account, and records audit. + +### C — Transition/compat regression + +Test concurrent member-creation attempts, duplicate username, racing disable and token authentication, restart, backup/restore, stale config key, public-dashboard setting and no-dashboard mode. Prove no partial user without assigned profile after a transactional failure. + +Acceptance: consistent mode and audit state with no orphan identities or active tokens on failed transition. + +## 8. Failure, cancellation, restart and contention + +Transaction must commit user, initial profile, access-state and audit together; errors cannot half-enable managed mode. Disability revokes credentials/sessions and fences in-flight admin mutations. No external provider calls or blocking password hashing inside SQLite gate. + +## 9. Compatibility/migrations + +Prefer no migration and preserve stored access-state labels. If a state merge is necessary, use append-only versioned migration/checksum, tested v54→current restore and backup rollback; never silently change legacy daemon authentication. CLI command names are additions, not behavior changes for old users. + +## 10. Tests + +Extend `rust/tests/on_prem_identity.rs`, `rust/tests/on_prem_admin_ui.rs`, `rust/tests/on_prem_sessions.rs`, `rust/tests/cli_contract.rs`, `rust/tests/database_compatibility.rs`. Include dashboard-disabled/headless E2E, stale shared key and dashboard.public transition contradictions, first-member atomicity, cross-member isolation and concurrent admin races. + +## 11. Verification commands + +```bash +cargo fmt --manifest-path rust/Cargo.toml --all -- --check +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings +cargo check --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- -D warnings +cargo test --manifest-path rust/Cargo.toml --workspace --all-targets -- --test-threads=1 +uv run pytest tests/tooling/ -q --tb=short --maxfail=1 +git diff --check +``` + +## 12. Documentation + +Update `docs/cli-reference.md`, `docs/deployment.md`, `architecture/deep-dive-security.md`, and administrative onboarding help with exact transition and member CLI examples. + +## 13. Acceptance criteria + +Fresh solo owner stays operational without additional mode toggles; first member can be provisioned fully headlessly; CLI/browser use identical cutover rules; stale server key cannot authenticate; no session/credential lost on failed transition. + +## 14. Stop conditions + +Do not proceed until M017/M019 closed. Stop if change reintroduces legacy shared key access, unauthenticated admin setup, breaking migration or inconsistent transactional security. + +## 15. Closure evidence + +`plans/closure/on-prem-user-access/020-status.md`: implementation SHAs, state matrix, headless CLI and two-user/negative E2E, serial/no-default CI, migration and security review and unblock audit. + +## 16. Handoff + +P2 capability. Prefer minimizing persistent mode state rather than adding new legacy compatibility flags; accepted ADR still controls bootstrap authority. From 19268b2f6841256e3b1452082fad4c6bf92424b5 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 11:38:59 +0000 Subject: [PATCH 05/62] Plan M021 small-team default provisioning --- .../021-default-member-policy-provisioning.md | 101 ++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md diff --git a/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md b/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md new file mode 100644 index 00000000..c7b9f1bd --- /dev/null +++ b/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md @@ -0,0 +1,101 @@ +# On-Prem User Access M021 — Default member policy and provisioning UX + +Status: blocked + +Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). + +Source roadmap: `plans/subsystems/on-prem-user-access-roadmap.md#m021--default-member-policy-and-provisioning-ux` + +Long-term requirements: `plans/000-long-term-specification.md#1-product-definition`, `plans/002-long-term-roadmap.md#phase-5--optional-on-prem-small-team-access-and-budgeting`. + +Applicable ADR: `plans/adrs/ADR-0002-on-prem-user-identity-and-policy-boundaries.md` (accepted). + +Primary class: capability + +## 1. Objective + +Make creation of a member useful immediately without weakening default-deny policy enforcement. Offer explicit reusable default-profile and default-budget selection for small teams, while keeping the single-admin personal case zero-configuration. + +## 2. Dependencies/readiness + +Hard: M020 headless/member lifecycle closure. Interface: `model_policy.rs` snapshots and revision triggers, `identity.rs` atomic create with optional profile, `user_budget.rs` rules and browser admin renderer. No physical SBC performance dependency. + +## 3. Current evidence + +`rust/src/model_policy.rs::ModelPolicySnapshot::allows` defaults to deny for a member without assigned profile and allow for an unprofiled administrator. `rust/src/server/dashboard/render/admin.rs` offers `Initial profile: No profile` when creating members, which succeeds but can leave an active member unable to use any model. Profiles already carry named exact model allow/deny rules and a preferred model; daily/monthly budgets are separately stored in SQLite. + +## 4. Invariants + +Default deny remains the internal safe fallback; instance-deny always overrides profile/user allow. Never copy provider secrets into user data or infer a model allow from client-supplied identity. No cloud catalog dependency for provisioning. Preserve revision invalidation, model alias and provider-qualified eligibility and user/global budget intersection. + +## 5. Scope + +In: optional operator-configured default profile/budget template, explicit create-time profile confirmation, policy visibility/reason codes, admin+CLI consistent defaults. Out: arbitrary role hierarchy, orgs, broad wildcard rules, billing/invoices, prompt/content retention, model policy language redesign. + +## 6. Production changes + +Provide a bounded instance-local default member provisioning choice, stored in SQLite under admin authorization (never modifying provider TOML). Choose whether one existing named profile can be marked default; do not add duplicate policy semantics. Member creation without explicit profile must either use the configured default transactionally or clearly surface the resulting no-access state before/after creation. Optional default budget template should copy limits at creation with clear provenance/semantics, not silently enforce approximate estimates as hard ceilings. + +## 7. Ordered work packages + +### A — Default-profile authority + +Select and persist one administrator-managed default member profile using existing policy revisions and audit. Retain safe no-profile/deny fallback if no default is configured. Reassignment or removal must have explicit effect and no accidental retroactive grant. + +Acceptance: deterministic effective grants for admin, no-profile member and default-profile member. + +### B — Provisioning integration + +Use default policy during CLI and browser member creation with atomic user/profile/audit write. Show resolved profile and any budget rules before final confirmation. Provide explicit allow-all, restricted/model-specific and local-model-only examples as editable regular profiles; do not embed provider IDs or automatically grant models without consent. + +Acceptance: newly created member sees the intended models on `/v1/models`, cannot request a denied provider-qualified target or alias. + +### C — Member/operator visibility + +Make empty catalog and denied model/budget errors distinguishable to users without exposing upstream private account details. Update admin/member pages and negative policy tests. + +Acceptance: clear explanation of why an account has no models; no user can see another user's budgets or profile internals. + +## 8. Failure/restart/contention + +Creating a user and assigning default profile/budget must not partially commit; concurrent admin change of default must use a consistent transactional snapshot. Policy edits invalidate discovery on next request; in-flight policy snapshot semantics remain unchanged. Do not hold the DB writer across network calls. + +## 9. Compatibility/migration + +Append a minimal numbered/checksum SQLite migration only if new default metadata is needed; preserve restored v54 and schema62 data. No implicit migration of existing users to a new default or hard budgets; new defaults apply only to future member creation unless an explicit admin action requests otherwise. + +## 10. Tests + +Extend `rust/tests/on_prem_model_policy.rs`, `rust/tests/on_prem_admin_ui.rs`, `rust/tests/on_prem_identity.rs`, `rust/tests/on_prem_budgets.rs`, `rust/tests/database_compatibility.rs` and `rust/tests/cli_contract.rs`; include no-default fallback, create-time admin race, alias/provider bypass, schema restore and no-dashboard CLI path. + +## 11. Verification commands + +```bash +cargo fmt --manifest-path rust/Cargo.toml --all -- --check +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings +cargo check --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- -D warnings +cargo test --manifest-path rust/Cargo.toml --workspace --all-targets -- --test-threads=1 +uv run pytest tests/tooling/ -q --tb=short --maxfail=1 +git diff --check +``` + +## 12. Documentation + +Update `docs/deployment.md`, `docs/model-routing.md`, `docs/model-limits.md`, `docs/cli-reference.md`, `architecture/deep-dive-security.md`, and embedded admin help. Preserve configuration clarity for a single admin with no profiles. + +## 13. Acceptance criteria + +One-admin solo mode remains unrestricted by default, and small-team admins can provide deterministic member defaults via CLI/browser. New member's catalog and admission agree, no surprise model grants occur, and instance/global budget authority is unchanged. + +## 14. Stop conditions + +Do not start before M020 closure. Stop for architectural review if default policy changes existing users, requires wildcard-based bypass or cannot be created atomically with the member. + +## 15. Closure evidence + +Create `plans/closure/on-prem-user-access/021-status.md` with policy and budget admission matrix, browser/CLI user journey, negative cross-user tests, migration evidence if any, serial/no-default checks, docs, severity-tagged residuals and unblock audit. + +## 16. Handoff + +P2 product ergonomics; ordinary small teams should not need a new policy DSL to establish a few reusable access classes. From 6a1de9ddc129e63ca4380382cecfcd460e6d7863 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 11:39:49 +0000 Subject: [PATCH 06/62] Plan M022 hard budget client compatibility --- .../022-hard-budget-agent-compatibility.md | 101 ++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md diff --git a/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md b/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md new file mode 100644 index 00000000..97026266 --- /dev/null +++ b/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md @@ -0,0 +1,101 @@ +# On-Prem User Access M022 — Hard-budget interoperability with coding agents + +Status: blocked + +Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). + +Source roadmap: `plans/subsystems/on-prem-user-access-roadmap.md#m022--hard-budget-agent-interoperability` + +Long-term requirements: `plans/000-long-term-specification.md#2-end-state-invariants-normative`, `plans/002-long-term-roadmap.md#phase-5--optional-on-prem-small-team-access-and-budgeting`. + +Applicable ADR: `plans/adrs/ADR-0002-on-prem-user-identity-and-policy-boundaries.md` (accepted). + +Primary class: polish + +## 1. Objective + +Measure and, only where provably safe, improve the usability of hard per-user and instance-wide request/token/USD budgets with Eggpool's supported coding-agent clients. Do not weaken hard reservation guarantees to make a client appear compatible. + +## 2. Readiness and dependencies + +Hard: M017 client credential/replay integration must close first. Interface: existing M008/M009 hard/estimated admission API, canonical wire fields and provider-qualified price bounds. Operational: use synthetic clients/request captures with no personal code/prompts or production keys. + +## 3. Current evidence + +`rust/src/user_budget.rs` and `rust/src/instance_budget.rs` reserve worst-case hard costs; `docs/model-limits.md` documents proof-or-reject semantics. Hard token rules require explicit caller output cap. Hard USD rules also require conservative operator tariff for all eligible providers. Some supported clients may omit compatible output-cap fields for one or more OpenAI Responses, Chat, Anthropic Messages or compact surfaces; those requests correctly fail before dispatch. M015 corrected under-reserved hard-dollar proof, so a relaxed estimated fallback would regress security. + +## 4. Invariants + +Proof-or-reject semantics remain strict: no output/price bound means deny, never assign zero USD or silently downgrade hard to estimated. Preserve model policy intersection, retry and selector overhead, stream/cancellation settlement, provider account scoring and backward-compatible client wire. Private local models may use request/token budgets without USD tariffs. + +## 5. Scope + +In: reproduce which clients/protocols omit bounds, clearer budget failure diagnostics, optional operator-clamped output cap only if upstream and wire semantics guarantee an enforceable bound. Out: estimating output as hard proof, pricing consumer subscriptions as metered invoices, changing user quota to upstream account routing, unlimited retries. + +## 6. Production changes + +Start by creating a protocol/client matrix with OpenCode, Codex and other supported clients using controlled fixture requests. Identify exact native output-limit fields. Investigate an explicit per-model/provider operator maximum output cap that the adapter can enforce in upstream request payload and prove across native/transcoded/streaming/compact paths. If impossible on a surface, preserve rejection and add clear guidance. Do not add a cross-client injected limit just to improve pass rates. + +## 7. Ordered work packages + +### A — Freeze compatibility corpus + +Collect bounded, sanitized synthetic request shapes for Chat, Responses, compact and Messages, including explicit/omitted/invalid output caps; distinguish user UI and agent API behavior. Confirm actual client versions and surface selection at test time. + +Acceptance: reproducible proof-or-reject table by client/surface, no real request bodies. + +### B — Bounded remediation where supported + +For a demonstrably enforceable native upstream cap, add explicit opt-in operator limit and adapt admission proof to the guaranteed output bound. Ensure that no upstream can exceed its advertised limit through alias, transcoding, retry or selector; otherwise retain 429. Do not require custom clients to send a nonstandard header. + +Acceptance: adversarial proof test for every affected surface, including malformed/ambiguous cap and fallback provider. + +### C — UX and failure mapping + +Return stable sanitized reason codes and meaningful corrective hints for missing output cap, unknown tariff, exhausted budget and concurrency. Preserve correct `Retry-After` behavior only for retryable conditions. Surface non-priced private-model token/request limits accurately. + +Acceptance: a member can identify whether to change the client request, budget or provider tariff without seeing another user's budgets or accounts. + +## 8. Failure, cancellation, restart and contention + +Atomic pre-dispatch holds and per-attempt retry bounds must survive stream termination, cancellation, restart and double finalization. A config reload changing model limits or provider tariff cannot let in-flight requests evade reserved proof. Do not hold the DB gate across I/O. + +## 9. Compatibility and migration + +Preserve existing default `estimated` rules and every currently supported hard behavior; opt-in operator bound is additional config with correct rehash/restart classification. No DB migration anticipated unless contract evidence proves necessary; if so use append-only/checksum plus restore tests. + +## 10. Tests + +Extend `rust/tests/on_prem_budgets.rs`, `rust/tests/on_prem_global_budgets.rs`, `rust/tests/on_prem_attribution.rs`, `rust/tests/coordinator_boundaries.rs`, `rust/tests/provider_transport.rs` and focused wire/transcoder suites. Add sanitized client contract fixtures and prove no unbounded upstream dispatch; include streaming, compact, virtual routes and server reload. + +## 11. Verification commands + +```bash +cargo fmt --manifest-path rust/Cargo.toml --all -- --check +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings +cargo check --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- -D warnings +cargo test --manifest-path rust/Cargo.toml --workspace --all-targets -- --test-threads=1 +uv run pytest tests/tooling/ -q --tb=short --maxfail=1 +git diff --check +``` + +## 12. Documentation + +`docs/model-limits.md`, `docs/agent-configuration.md`, `docs/api-reference.md`, `architecture/deep-dive-request-lifecycle.md`; include support matrix and preserved proof-or-reject boundary. + +## 13. Acceptance criteria + +Hard budgets never dispatch requests without conservative upper bounds; supported clients either work with provably bounded output or receive clear actionable errors; local-only models can use token/request quotas without fabricated USD cost. + +## 14. Stop conditions + +Do not start before M017. Stop if a client/provider lacks an enforceable output cap, if a solution dilutes M015 proof, or if verifying compatibility requires real user prompts. An unfixable surface is a documented limitation, not permission to relax admission. + +## 15. Closure evidence + +`plans/closure/on-prem-user-access/022-status.md` with client/wire matrix, worst-case bound proofs, synthetic negative traces, exact SHAs, serial/no-default CI and unresolved cases; unblock audit. + +## 16. Handoff + +This is a scoped interoperability and optional corrective milestone; never promise automatic output-cap injection without protocol evidence. From a0aedd8fd61207eed716a145089da682a925e104 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 11:40:44 +0000 Subject: [PATCH 07/62] Plan M023 small-team short-window admission fairness --- .../023-short-window-fairness-controls.md | 101 ++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md diff --git a/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md b/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md new file mode 100644 index 00000000..cfaf97fa --- /dev/null +++ b/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md @@ -0,0 +1,101 @@ +# On-Prem User Access M023 — Short-window team fairness and admission controls + +Status: blocked + +Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). + +Source roadmap: `plans/subsystems/on-prem-user-access-roadmap.md#m023--short-window-team-fairness-controls` + +Long-term requirements: `plans/000-long-term-specification.md#1-product-definition`, `#2-end-state-invariants-normative`; `plans/002-long-term-roadmap.md#phase-5--optional-on-prem-small-team-access-and-budgeting`. + +Applicable ADR: `plans/adrs/ADR-0002-on-prem-user-identity-and-policy-boundaries.md` (accepted); require new ADR if durable budget semantics or fairness ownership changes. + +Primary class: capability + +## 1. Objective + +Prevent one user's bursty coding agents from monopolizing shared provider capacity on a small on-prem team server while retaining minimal solo/SBC overhead. Complement existing daily/monthly request, token, cost and concurrency ceilings with optional bounded short-window admission fairness; do not replace provider-health/quota routing. + +## 2. Dependencies + +Hard: M022 client/budget interoperability closure. Interface: existing principal authentication, M008/M009 durable reservations and `coordinator/endpoints.rs` admission order. Operational: qualify on representative ordinary on-prem Linux server; Pi/microSD is a solo low-end reference, not the team load-release gate. + +## 3. Current evidence + +`rust/src/user_budget.rs` and `rust/src/instance_budget.rs` already handle daily/monthly and max-concurrent rules transactionally. `routing/` owns provider-account quota fairness, not downstream-user fairness. An authorized user's burst may exhaust provider throughput or dominate latency without violating day/month budgets. Existing single-gate SQLite performance, particularly Pi microSD write tails, argues against persisting per-request rolling-window rows by default. + +## 4. Invariants + +Do not let short-window counters substitute for durable daily/monthly budget holds, global ceilings, typed principals or model restrictions. No extra SQLite writer, no queue with unbounded request-body retention, no hidden priority routing or provider-health penalty on user refusal. Single-user instance has zero new operational setup and negligible idle footprint. + +## 5. Scope + +In: opt-in per-principal short-window requests/sec or requests/minute, optional bounded global admission burst, actionable 429/Retry-After and lightweight visibility. Out: token-throughput shaping based on unknown generated output, tiered billing, weighted-fair queuing, SSO, a durable distributed rate limiter, 50-simultaneous inference promise. + +## 6. Production changes + +Implement a bounded in-memory per-user token bucket or sliding-window admission guard in the single daemon. Limit number of retained principals and stale-window lifetime, prevent key spoofing and excessive hash-map growth, and reject before body/upstream reservation where principal is known. Consider typed admission result with exact distinction between short-window throttle, hard budget, and provider 429. Define restart semantics explicitly: ephemeral limiter resets on restart, but durable budgets do not. + +## 7. Ordered work packages + +### A — Define fairness contract and defaults + +Freeze absent-rule behavior (no additional limit), operator-specified policy and best-effort limiter semantics. Choose a monotonic clock and deterministic Retry-After calculation. Separate identity from token: several agent credentials for one human share the same principal bucket. + +Acceptance: documented no-rule solo fast path and bounded memory by unique principals. + +### B — Implement bounded admission guard + +Insert at an existing authenticated pre-dispatch boundary without blocking SQLite or current-thread execution; avoid background process and request-body buffering. Handle simultaneous last-slot requests and configured per-user/global rate intersection. Reject with stable 429 plus Retry-After and no upstream attempt. + +Acceptance: concurrent bursts enforce the declared bucket limits without impacting provider-account health or durable budget counters. + +### C — Verify ordinary server throughput and UX + +Use a synthetic provider and multiple principals on ordinary server/storage. Measure admitted/rejected rates and p50/p95/p99 local overhead against no-rule baseline; add sanitized member/admin diagnostic indication and documentation. Run an inexpensive Pi microSD smoke only as compatibility, not team sizing. + +Acceptance: no material no-rule regression and no unbounded memory under many bogus credentials or clients. + +## 8. Failure, cancellation, restart, contention + +Limiter state is ephemeral and resettable after process restart; durable user/global budget state remains authoritative. Cancellation before upstream dispatch should not corrupt long-lived counters; clearly define whether attempts consume an admission slot. No per-client unbounded wait queue. Under max cached users, fail or evict only with a documented safe rule, never silently grant excess throughput. + +## 9. Compatibility/migration + +Opt-in defaults to disabled; no SQLite schema needed unless an existing admin policy table is minimally extended, which requires checksums and restore evidence. Preserve existing 429 provider/budget errors and avoid breaking client retries. Additional operator config classified by existing reload policy; respect immutable generation transitions. + +## 10. Tests + +New focused fairness/admission test target plus `rust/tests/on_prem_budgets.rs`, `on_prem_global_budgets.rs`, `on_prem_attribution.rs`, `coordinator_boundaries.rs`. Cover 1/5/25/49 named users with explicitly declared concurrency, many credentials one principal, clock edges, collision/race, absent config, restart, fake provider 429 and maximum cache cardinality. + +## 11. Verification commands + +```bash +cargo fmt --manifest-path rust/Cargo.toml --all -- --check +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings +cargo check --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- -D warnings +cargo test --manifest-path rust/Cargo.toml --workspace --all-targets -- --test-threads=1 +uv run pytest tests/tooling/ -q --tb=short --maxfail=1 +git diff --check +``` + +## 12. Documentation + +`docs/model-limits.md`, `docs/deployment.md`, `docs/api-reference.md`, `architecture/deep-dive-request-lifecycle.md` and `architecture/deep-dive-security.md`; provide ordinary server team recommendations, no SBC SLA. + +## 13. Acceptance criteria + +With rate limits absent, same solo behavior; with limits present, one user's burst does not exhaust all shared admission slots, a 429 is actionable and distinct from provider failure, and durable user/global budgets remain exactly enforced. + +## 14. Stop conditions + +Do not start before M022; stop if needed design changes durable reservation or provider fairness without ADR, introduces unbounded queues, adds significant per-request SQLite writes, or has no measured on-prem server baseline. + +## 15. Closure evidence + +`plans/closure/on-prem-user-access/023-status.md` with exact commit, contention/rate matrix, ordinary Linux hardware/storage metadata and baseline deltas, serial/no-default checks, failure/restart cases, compatibility notes and unblock audit. + +## 16. Handoff + +P3 optional small-team capability. Ordinary server is the primary team target; SBC microSD remains a lightweight solo regression probe, never the release throughput requirement. From 41ff19fb73eb8bc97d4422abd96b2a96a45eb2e5 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 11:41:45 +0000 Subject: [PATCH 08/62] Plan on-prem concurrency M002 ordinary-server small-team baseline --- .../002-ordinary-server-team-baseline.md | 102 ++++++++++++++++++ 1 file changed, 102 insertions(+) create mode 100644 plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md diff --git a/plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md b/plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md new file mode 100644 index 00000000..40810431 --- /dev/null +++ b/plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md @@ -0,0 +1,102 @@ +# On-Prem Concurrency Performance M002 — Ordinary-server small-team capacity baseline + +Status: ready + +Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). + +Source roadmap: `plans/subsystems/on-prem-concurrency-performance-roadmap.md#m002--ordinary-server-team-baseline` + +Long-term requirements: `plans/000-long-term-specification.md#1-product-definition`, `#2-end-state-invariants-normative`, `plans/002-long-term-roadmap.md#phase-5--optional-on-prem-small-team-access-and-budgeting`. + +Applicable ADR: `plans/adrs/ADR-0002-on-prem-user-identity-and-policy-boundaries.md` (accepted). Persistence/storage ADRs unchanged. + +Primary class: polish + +## 1. Objective + +Measure credible on-prem small-team capacity on representative ordinary Linux server hardware with durable SSD/NVMe, explicitly separate named team membership from active simultaneous inference requests, and retain Pi 5 microSD as a low-end solo performance/durability regression signal. This is not a mandate to optimize Pi microSD tails or claim 50 concurrent generations. + +## 2. Readiness and dependencies + +Hard: none; on-prem PR #7 and Pi M001 attribution are integrated/conditionally closed. Operational: an explicitly provisioned ordinary Linux target with hardware/OS/storage details is mandatory for a server-capacity claim; otherwise close only a tooling/setup slice conditionally and record unmeasured cells. No external data/provider credentials. + +## 3. Current evidence + +`plans/closure/on-prem-user-access/012-status.md` measured 49 named Pi accounts, RSS around 20 MB and single-request p50 around 5 ms, with microSD contention and eight-way saturation. `plans/closure/on-prem-concurrency-performance/001-status.md` isolated recurrent microSD SQLite publication tails versus tmpfs; a separate exploratory NVMe run reported 1,000/1,000 completions at low latency. These are not representative ordinary-server capacity/SLA evidence. `scripts/qualification_sbc.py` already has principal, concurrency and optional budget load parameters and emits sanitized artifacts. + +## 4. Invariants + +Do not change production DB journal, worker/gate topology, admission, policy, provider transport, quota fairness or budget semantics from benchmarking alone. No real user secrets/prompts, no live external provider billing, no false 50-concurrent service claim, and no insecure performance shortcut (e.g. production tmpfs database). + +## 5. Scope + +In: repeatable ordinary-server synthetic qualification corpus, resource/latency metrics, safety and deployment guidance; optional small tooling adjustments. Out: runtime performance optimization, capacity promises without hardware, re-testing Pi M001 closure as a blocker, switching to external DB, distributed cluster, web load balancer. + +## 6. Required production changes + +None by default. Extend `scripts/qualification_sbc.py` or extract shared sanitized tooling if needed for ordinary-server fixtures; retain existing Pi fixture and predeclared metric meanings. Record build identity, physical hardware CPU/RAM/disk/fs/mount, container/VM status, server OS, actual storage persistence, WAL/NORMAL and database worker count. Freeze workload and gates prior to collecting results. + +## 7. Ordered work packages + +### A — Freeze workload and qualification design + +Define at least 1,5,10,25,49 named accounts (metadata) separately from 1,2,4,8,16 concurrent in-flight requests, bounded batch sizes and run duration; further levels optional until safe. Use a synthetic upstream with configurable latency distribution, finite/streaming/translated surfaces, realistic payload sizes, per-user/global budget controls, mixed policy denies and token revocation. Include solo baseline on same host. + +Acceptance: reproducible manifest, build hashes, no customer secrets, capacity terminology unambiguous. + +### B — Paired durable server runs + +On stable ordinary Linux SSD/NVMe test host, run three fresh-root repetitions per primary condition with WAL/NORMAL, one writer, no over-commit/benchmark artifacts. Record p50/p95/p99/max, throughput, requests/timeouts, stream TTFT, pre-upstream/dispatch/provider phases, CPU, RSS, SQLite gate, disk IO/pressure, WAL/checkpoints, and pending/reservation convergence. Include saturated load without causing host destabilization; report safety stop conditions. + +Acceptance: accepted fixture JSON, accurate per-condition percentiles (enough samples to support p99), no orphaned active reservations, no data leakage. + +### C — Interpret deployment envelope + +Produce an ordinary-server deployment guide with honest throughput/concurrency ranges for the measured host/workload and sizing factors (provider latency, number of backends, SSD/CPU, login and rate/budget policy). Compare Pi microSD solo reference without rewriting old results. If an Eggpool-owned hot-path issue emerges, register a separate evidence-gated implementation plan rather than silently retuning SQLite or WAL. + +Acceptance: explicit no-SLA language and reproducible raw-to-summary report guarded by tooling tests. + +## 8. Failure, cancellation, restart and contention + +Exercise shutdown/restart while users and budgets are active, retry and partial upstream errors, disk slowdowns and user disable. Count observed 429s separately for provider, identity and budget. Do not infer percentiles from a few samples; run on isolated target and abort pressure tests on sustained IO pressure/OOM. No batch success report can hide incomplete reservations. + +## 9. Compatibility/migration + +No storage migration or runtime API change expected. Keep Pi M001/MMC evidence immutable. Qualify the installed release candidate and its current migration state; do not claim capacity for unpublished binaries or different storage/media. + +## 10. Tests + +Extend `tests/tooling/test_qualification_sbc.py`, `tests/tooling/test_on_prem_mmc_tail_attribution.py` or add ordinary-server result validation tests; sanity-check concurrency/principal counts, p99 sample size, synthetic identities and secret absence. Re-run `rust/tests/on_prem_budgets.rs`, `rust/tests/on_prem_global_budgets.rs` and `rust/tests/on_prem_attribution.rs` if fixture logic changes. + +## 11. Verification commands + +```bash +cargo fmt --manifest-path rust/Cargo.toml --all -- --check +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings +cargo check --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features +cargo test --manifest-path rust/Cargo.toml --workspace --all-targets -- --test-threads=1 +uv run pytest tests/tooling/test_qualification_sbc.py tests/tooling/test_on_prem_mmc_tail_attribution.py -q +uv run ruff check scripts/ tests/tooling/ +uv run pyright scripts/ +git diff --check +``` + +## 12. Documentation + +`docs/deployment.md`, `docs/raspberry-pi.md`, `architecture/deep-dive-observability.md`; report isolated ordinary-server results without rewriting Pi closure data or promising that every small team has the same workload. + +## 13. Acceptance criteria + +At least one ordinary-server/SSD target with three repeated measured conditions, explicit 1/5/10/25/49 named-user and concurrent-request separation, durable settlement/cancel/recovery evidence, honest measured envelope and solo low-end regression qualification. + +## 14. Stop conditions + +Stop and report conditional/unmeasured results if physical target unavailable, unsafe IO pressure, unreproducible provider fixture, insufficient p99 sample size, or architecture would need persistence-policy ADR. Do not invent a capacity number. + +## 15. Closure evidence + +Create `plans/closure/on-prem-concurrency-performance/002-status.md` with exact target/build/source, experiment matrix and sanitized artifacts, gate outcomes (including failures), measured latency/throughput/resource/settlement observations, security and content audit, conditionals and unblock audit. + +## 16. Handoff + +Performance evidence only, independent of M017–M023 code. Ordinary server is the likely team host; Pi microSD remains the canonical low-budget solo platform and a meaningful lightweight performance bound. From 576d20def270ae3a4d357f19c83c7d966b134f45 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 11:42:26 +0000 Subject: [PATCH 09/62] Clarify solo SBC versus on-prem team-server target --- plans/000-long-term-specification.md | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/plans/000-long-term-specification.md b/plans/000-long-term-specification.md index 98346598..ed8e1c74 100644 --- a/plans/000-long-term-specification.md +++ b/plans/000-long-term-specification.md @@ -29,12 +29,17 @@ including `/v1/responses/compact`), and Anthropic Messages executable plus metadata/assets only. Repo-root `pyproject.toml` is tooling-only. -EggPool's canonical operating profile is a single on-prem instance on SBC or ordinary -local hardware, administered by one owner. An optional local-users mode serves -small teams (target: fewer than approximately 50 named users) sharing -operator-controlled routing to authorized provider APIs/subscriptions and -self-hosted models. This is not a SaaS, enterprise IAM, or multi-organization -hosting product; a 50-user capacity claim requires measured target evidence. +EggPool's canonical operating profile is a single on-prem instance +administered by a local owner. Single-user installations are especially suited +to SBCs (including ordinary microSD-backed devices), which remain a useful +lightweight performance/durability reference and should require no additional +service or remote IAM. Optional local-users mode serves small teams (target: +fewer than approximately 50 named users), typically on an ordinary on-prem +server with durable storage, sharing operator-controlled routing to authorized +provider APIs/subscriptions and self-hosted models. Named users are not +simultaneous generations; any team throughput/concurrency claim requires +representative ordinary-server hardware, workload and storage measurements. +This is not a SaaS, enterprise IAM, or multi-organization hosting product. Provider topology, upstream credentials, and routing configuration remain in operator-owned TOML/environment files; human identity, access profiles, sessions, individual credentials, user quotas, and usage statistics belong in From 6c4f0be49af956a67b14a85eef98ada702f0eaa0 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 11:42:29 +0000 Subject: [PATCH 10/62] Clarify phase 5 operational and hardware acceptance --- plans/002-long-term-roadmap.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/plans/002-long-term-roadmap.md b/plans/002-long-term-roadmap.md index 341c96a2..c5d0ea5d 100644 --- a/plans/002-long-term-roadmap.md +++ b/plans/002-long-term-roadmap.md @@ -81,9 +81,12 @@ changes. ## Phase 5 — Optional on-prem small-team access and budgeting Objective: add identity-bound use of a single locally administered EggPool -instance without regressing the single-owner SBC and LAN deployment. -Authorized clients may route to operator-configured hosted or self-hosted LLMs; -all provider secrets/topology remain under TOML/environment ownership. +instance without regressing the solo owner's SBC/microSD and LAN experience. +Small teams are expected primarily on ordinary on-prem servers with durable +storage; Pi/microSD tests are a lightweight solo regression baseline, not the +default team throughput acceptance target. Authorized clients may route to +operator-configured hosted or self-hosted LLMs; all provider secrets/topology +remain under TOML/environment ownership. - Authority: `plans/adrs/ADR-0002-on-prem-user-identity-and-policy-boundaries.md` and `plans/subsystems/on-prem-user-access-roadmap.md`. @@ -98,10 +101,12 @@ all provider secrets/topology remain under TOML/environment ownership. mandatory cloud/SSO, native public internet access, and unqualified claims about 50 concurrent generations. - Completion: legacy upgrade preserves data and access; fresh bootstrap is - fail closed without defaults; user policy cannot be bypassed via virtual - routing or alternate surfaces; per-user budgets recover across cancellation/ - restart; measured SBC and ordinary-hardware evidence scopes supported - team size and concurrent workload honestly. + fail closed without defaults; downstream credentials, agent configuration, + status CLI and browser access work after identity cutover; user policy + cannot be bypassed via virtual routing or alternate surfaces; per-user + budgets recover across cancellation/restart. Measured solo SBC/microSD + evidence demonstrates light footprint; representative ordinary on-prem + server evidence scopes team concurrency claims honestly. Phases 2–5 create NO automatic milestones. Subsystem roadmaps are written on demand when a slice is ready to be reasoned about, with typed dependencies From 98142c59ce94297c757e86cf593cd94abb246069 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 11:43:18 +0000 Subject: [PATCH 11/62] Reconcile on-prem post-merge roadmap and M017-M023 dependencies --- .../subsystems/on-prem-user-access-roadmap.md | 147 +++++++++++++----- 1 file changed, 108 insertions(+), 39 deletions(-) diff --git a/plans/subsystems/on-prem-user-access-roadmap.md b/plans/subsystems/on-prem-user-access-roadmap.md index b6551161..474f4582 100644 --- a/plans/subsystems/on-prem-user-access-roadmap.md +++ b/plans/subsystems/on-prem-user-access-roadmap.md @@ -73,44 +73,31 @@ as authority, custom upstream per-user provider secrets, or a published ## 4. Current state -M001–M007 were implemented on PR #7 from baseline -`c18dcc2051ae4987a2cb95a7cd046b8aff2095ff` and -each has a retained closure record. M002 and M004 are closed. -M001, M003, M005, M006 and M007 are **conditionally closed**, -not equivalent to release qualification. - -The current Rust implementation contains: -- Append-only SQLite identity/session/request-attribution/policy/budget/admin - migrations **0055–0060**; `rust/src/identity.rs`, `web_session.rs`, - `model_policy.rs`, `user_budget.rs`. -- `rust/src/server/middleware.rs` / `auth.rs`: personal agent tokens, - opaque browser sessions, CSRF and managed-route role fencing. -- `rust/src/coordinator/endpoints.rs`, `publication.rs`, - `finalization.rs`: per-request principal, concrete-model authorization - and downstream-user budget holds/usage settlement. -- `rust/src/server/dashboard/admin.rs` and `render/admin.rs`: - admin/member user, profile, budget, usage and audit views. -- `rust/tests/on_prem_*.rs` and successful local/hosted default CI; - the M007 synthetic provider artifact exercises seven loopback requests. - -Remaining verified gaps (see immutable closure records M001/M003/M005–M007): -- M005 token/USD hard-limit semantics are not proven from estimated holds; - unknown-price models can receive generic USD estimates; **global instance - ceilings are absent**. -- Budget-specific process-crash and actual schema-54 restored-backup - qualification, plus ordinary-host SQLite-gate overhead measurements, - were not performed. -- M006 new admin/member pages lack browser screenshot/interaction - qualification. The Python oracle applies to pre-existing pages only. -- Physical Pi/Le Potato Argon2/SQLite/WAL/TTFT and 1/5/10/25/49 **named** - user workloads are unmeasured. No concurrency/capacity claim is authorized. -- M003 records local `operations::update` failures; the latest hosted - branch CI is green, but reproducible platform classification is not yet - documented. - -Correctives M008–M012 were created by explicit user direction; -the predecessor closures remain immutable. This roadmap is **active** -until required corrective closure evidence exists. +PR #7 landed on `main` as merge commit `88f58bbf2e9095cf46437945edf83e580270bdb6` +on 2026-10-15. M001–M016 closure records remain immutable historical +evidence; M016's pre-merge review hold no longer describes the live branch. +The current runtime includes SQLite identity/session/policy/budget/audit +migrations 0055–0062, browser and personal-token auth, user/model/instance +budgets, CSRF-fenced administration and scoped discovery. Physical Pi 5 +qualification measured solo/lightweight footprint and microSD saturation, +not ordinary-server small-team capacity; see M012 closure and separate +on-prem concurrency M001. + +A post-merge integration audit found independent correctness gaps not covered +by the M016 pre-merge checks: `configsetup`/`configremote`/`getkey` +still rely on the retired shared server key; `status` and +`runtime-status` authenticate via that key and fail after bootstrap; +remote plaintext `GET /login` renders a password form that cannot safely +submit; first-member web transition and CLI `enable-managed` disagree +about config prerequisites; headless member CRUD and safe default provisioning +are incomplete. Hard-cost admission is already proof-or-reject (M008/M015), +but coding-agent compatibility needs bounded qualification. + +Product/hardware scope clarified 2026-10-15: **SBC/microSD is primarily +the solo-owner/lightweight reference**. Small teams are expected primarily +on ordinary on-prem Linux servers with durable SSD/NVMe and operator-managed +HTTPS for browsers. None of the new plans assumes 49 concurrent requests or +relaxes the single-process/SQLite/WAL architecture. ## 5. Target architecture @@ -309,6 +296,81 @@ was performed and no 49-concurrent capacity claim is made. Closure: or revise M012's closed physical evidence. Any new storage topology needs a separate accepted ADR. +### Post-merge M017–M023 corrective and team-utility milestones + +#### M017 — Managed client credential reconciliation + +Class: capability. Ready independently. Repair `onboard`, +`getkey/newkey`, `configsetup`, `configremote` and `eggpool-connect` +to use valid personal keys after bootstrap, preserve `EGGPOOL_API_KEY` +and secret-free `epc1`, and test real end-to-end agent requests. +Source: `plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md`. + +#### M018 — Local operator status corrective + +Class: capability. Ready independently. Restore `eggpool status` and +`runtime-status` for OS-authorized local operators without granting agent +tokens browser/admin rights. Prefer existing owner-protected local control +channel; new privilege decisions require ADR review. +Source: `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md`. + +#### M019 — Login transport and solo LAN operability + +Class: invariant. Ready independently. Prevent remote plaintext +`GET /login` rendering credential fields, verify proxy/CSRF invariants, +and document a clear SSH/loopback solo SBC browser path plus ordinary-server +TLS-proxy team path. No native TLS server or insecure LAN default. +Source: `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md`. + +#### M020 — Managed transition and headless administration + +Class: capability. Blocked on hard M017/M019 closure. Unify +`enable-managed` with member-create transition requirements and add local +CLI create/disable/enable/show member workflows; preserve audit, CSRF, +password-stdin, last-admin and atomic creation guarantees. +Source: `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md`. + +#### M021 — Default member policy and provisioning UX + +Class: capability. Blocked on hard M020 closure. Add explicit reusable +member default profile (and optional budget template) with safe fallback, +transactional assignment, admin/CLI consistency and understandable +catalog-empty/deny feedback. Never grant unaudited model access. +Source: `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md`. + +#### M022 — Hard-budget agent interoperability + +Class: polish. Blocked on hard M017 closure. Measure whether client/surface +requests carry sufficient output-token proof; improve diagnostics and +introduce output-cap insertion only where upstream and wire evidence prove a +hard bound. Do not regress M008/M015 hard-cost safety. +Source: `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md`. + +#### M023 — Short-window team fairness controls + +Class: capability. Blocked on hard M022 closure. Optional per-principal +short-window admission throttle and stable 429/Retry-After, bounded in-memory +state, no unbounded queues or new SQLite write pressure; qualify on ordinary +server hardware while preserving solo no-rule behavior. +Source: `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md`. + +### Post-merge dependency and hardware policy + +- Parallel ready: M017, M018 and M019. None depends on a Pi/Le Potato run. +- Hard: M017 + M019 -> M020 -> M021; M017 -> M022 -> M023. +- Interface: M018's privileged local transport must not broaden HTTP rights; + any new auth authority stops for ADR review. M022 hard-cost proof contract + is fixed by M008/M015. +- Operational: ordinary-server M002 performance qualification in + `plans/subsystems/on-prem-concurrency-performance-roadmap.md` is independent + and nonblocking to M017–M022; it supplies evidence for M023 and future + team-capacity claims. Pi microSD tail investigation is a solo benchmark, + not a team-server release gate. +- Deferred, not silently bundled: optional personal-token expiry/rotation, + secondary administrators (requires admin uniqueness migration and security + review), notifications and SSO; take them as separate bounded plans only + after core operability closure. + ## 8. Cross-cutting requirements - Storage: append immutable numbered migrations and checksums; preserve DB @@ -389,4 +451,11 @@ A ready plan is not an implemented capability. | M013 | closed | `plans/implementation/on-prem-user-access/013-transcode-eligibility-corrective.md` | `plans/closure/on-prem-user-access/013-status.md` | One-line transcode-protocol restoration plus admission bridge guard; Q007 green. | | M014 | closed | `plans/implementation/on-prem-user-access/014-rustls-advisory-dependency-corrective.md` | `plans/closure/on-prem-user-access/014-status.md` | Rustls 0.23.46; exact-SHA hosted cargo-deny, CI, and installer jobs pass. Main receives the fix when PR #7 merges. | | M015 | closed | `plans/implementation/on-prem-user-access/015-authz-threat-review-corrective.md` | `plans/closure/on-prem-user-access/015-status.md` | Exact candidate `d8fd9c2a`; findings fixed and independently re-reviewed; exact-SHA hosted gates pass. | -| M016 | closed | `plans/implementation/on-prem-user-access/016-final-pr7-merge-reconciliation.md` | `plans/closure/on-prem-user-access/016-status.md` | Exact candidate checks pass; merge held pending maintainer PR review. The roadmap remains active until the PR decision. | +| M016 | closed | `plans/implementation/on-prem-user-access/016-final-pr7-merge-reconciliation.md` | `plans/closure/on-prem-user-access/016-status.md` | Exact candidate checks pass; merge held pending maintainer PR review. PR #7 subsequently merged at `88f58bbf`; historical M016 closure remains unchanged. | +| M017 | ready | `plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md` | — | Independent; full onboarding/desktop client replay needed. | +| M018 | ready | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | — | Independent; inspect local control authority/ADR threshold. | +| M019 | ready | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | — | Independent; insecure remote GET password form. | +| M020 | blocked | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | — | Hard: M017 and M019 closure. | +| M021 | blocked | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | — | Hard: M020 closure. | +| M022 | blocked | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | — | Hard: M017 closure. | +| M023 | blocked | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Hard: M022 closure; ordinary-server evidence operational. | From 38e46b42b9c441a632f2bff564ea26c4d0f105d9 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 11:43:43 +0000 Subject: [PATCH 12/62] Separate solo SBC baseline from ordinary-server team qualification --- ...on-prem-concurrency-performance-roadmap.md | 50 ++++++++++++------- 1 file changed, 33 insertions(+), 17 deletions(-) diff --git a/plans/subsystems/on-prem-concurrency-performance-roadmap.md b/plans/subsystems/on-prem-concurrency-performance-roadmap.md index fe819e10..9a6eea84 100644 --- a/plans/subsystems/on-prem-concurrency-performance-roadmap.md +++ b/plans/subsystems/on-prem-concurrency-performance-roadmap.md @@ -16,13 +16,14 @@ Related ADRs: ## 1. Purpose and ownership boundary -Improve the *measured* concurrent latency envelope of one local/SBC -EggPool instance without regressing the fast single-user path or -introducing SaaS/cluster complexity. This is a **separate non-release- +Provide evidence for two distinct profiles: a solo owner using a lightweight +SBC (microSD included) and a small team using an ordinary on-prem server +with durable SSD/NVMe storage, without regressing single-user response latency +or introducing SaaS/cluster complexity. This is a **separate non-release- blocking diagnostic workstream**, opened because the original on-prem M012 Pi 5/MMC qualification exceeded predeclared tail-latency gates. It does **not** retroactively reopen the M012 closure or block -on-prem M016 final security/merge-readiness reconciliation. It may +post-merge user access corrective milestones M017–M022. It may find a new bounded optimization opportunity; no such implementation is authorized until evidence isolates cause and a plan is accepted. @@ -112,10 +113,10 @@ claiming a runtime optimization. target using microSD/MMC boot storage, controlled host activity and performance counters. SSD/external storage is optional exploratory evidence, not a completion dependency. -- Soft/interface: ongoing Persistence roadmap owns any - architectural storage-policy experiment; a future M002 - must not be registered until M001 identifies measurable - cause and independently accepted constraints. +- Interface: Persistence roadmap owns any architectural storage-policy + experiment; M002 is a separate **ordinary-server measurement** plan, not + the previously deferred production optimization. M001's storage + attribution supplies a baseline; no new WAL/SQLite change is authorized. - **No dependency edge** from this subsystem to on-prem M014/M015/M016 security or merge readiness. The current documented small-team envelope is limited but truthful. @@ -135,9 +136,22 @@ budget cases remain conditions for future optimization or broader operating claims. Source: `plans/implementation/on-prem-concurrency-performance/001-pi5-mmc-tail-latency-attribution.md`. -Future optimization milestone: **not registered**. Requires -separate scoped plan, architectural check and measurable -regression guard; not authorized by the diagnostic alone. +### M002 — Ordinary-server small-team baseline + +Class: polish. Ready independently of the user-access correctives. +Run a repeatable, redacted, synthetic qualification corpus on a documented +ordinary Linux server with durable SSD/NVMe and 1/5/10/25/49 named +principal records, separately from measured concurrent levels and workload. +Report CPU/RSS, throughput, local overhead, streaming TTFT, SQLite +gate/storage tails, budget recovery and admitted/denied requests. Three +paired runs and adequate sample size are required for percentile claims. +SBC microSD remains the solo low-end performance/durability reference; +it is **not** a gate for team capacity or server release. +Source: `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md`. + +Future **production** optimization milestone: not registered. Requires a +separate scope and measurable regression guard; no journal, checkpoint, +worker, or storage topology change is authorized by either M001 or M002. ## 8. Cross-cutting requirements @@ -185,14 +199,16 @@ a normal developer workstation. ## 11. Completion definition -M001 accepted closure with reproducible physical artifacts -and an explicit causal/no-causal conclusion; scope can close -even when an environment-only tail remains, provided -the restricted Pi/MMC envelope continues to be documented. -No optimization promise or new 8-way SLA is inferred. +M001 preserves its conditionally closed microSD attribution. +M002's eventual disposition must publish a separately measured, +realistically sized ordinary-server team envelope (or honestly mark +unavailable hardware unmeasured). It does not require eliminating +the Pi/MMC solo latency tail. No optimization promise or new +8-way/49-concurrent SLA is inferred. ## 12. Milestone status | Milestone | Status | Implementation plan | Closure record | Blockers | |---|---|---|---|---| -| M001 | conditionally closed | `plans/implementation/on-prem-concurrency-performance/001-pi5-mmc-tail-latency-attribution.md` | `plans/closure/on-prem-concurrency-performance/001-status.md` | Four-way microSD/MMC tail attributed to durable storage; controlled competing-I/O and broader account/budget matrix remain conditions. SSD/NVMe is optional and not a closure dependency. No dependency edge to on-prem M016. | +| M001 | conditionally closed | `plans/implementation/on-prem-concurrency-performance/001-pi5-mmc-tail-latency-attribution.md` | `plans/closure/on-prem-concurrency-performance/001-status.md` | Four-way microSD/MMC tail attributed to durable storage; controlled competing-I/O and broader account/budget matrix remain conditions. SSD/NVMe is optional and not a closure dependency. No dependency edge to post-merge on-prem operability plans. | +| M002 | ready | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | — | Ordinary Linux server with durable SSD/NVMe and repeated synthetic multi-principal workload is an operational evidence requirement, not a Pi/microSD gate. | From 24a57759d94f1a38869bcf1271598961f6e6b7a5 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 11:44:14 +0000 Subject: [PATCH 13/62] Register post-merge M017-M023 and ordinary-server M002 handoffs --- plans/registry.md | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/plans/registry.md b/plans/registry.md index 34d496af..32d67572 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -23,6 +23,14 @@ Legacy archive latest: Plan 250 241 remain historical per their own closure passes; the `146-*` duplicate pair is a known numbering accident. +Post-merge planning reconciliation (2026-10-15): PR #7 has merged to +`main` (`88f58bbf`). M016's closure accurately describes its historical +pre-merge hold, not the current branch. M017–M023 register independent +post-merge correctness, headless-administration and optional small-team +capability passes. Solo SBC/microSD remains the lightweight reference; ordinary +on-prem server hardware with durable storage is the expected team host. +No new runtime, schema or performance claim lands with these plans. + ## Status vocabulary - **proposed** — roadmap or plan exists but is not approved for execution. @@ -50,14 +58,18 @@ pair is a known numbering accident. | Persistence | active | `plans/subsystems/persistence-roadmap.md` | M011 ready — post-M010 roadmap/documentation reconciliation | M009/M010 closures accepted. Documentation/tooling-only corrective; production remains WAL/NORMAL. No technical persistence successor is registered. | | Deployment and packaging | closed | `plans/subsystems/deployment-packaging-roadmap.md` | M005 closed — active PATH integration detection corrective | M001–M005 closed; no successor registered. | | Dashboard | closed | `plans/subsystems/dashboard-roadmap.md` | M012 closed — lifecycle closure and documentation polish | M001–M012 closed; no registered successor; future dashboard work requires a new bounded plan. | -| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M016 closed — merge held pending maintainer review | M014/M015 and exact M016 candidate checks are closed/passed; PR #7 remains open pending maintainer review. M001–M013 history preserved; physical Pi 5 restricted envelope remains qualified only for solo/light concurrency. | -| On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M001 conditionally closed — Pi 5/microSD durable-storage tail attribution | Optional follow-up, **nonblocking** to PR #7 merge. Controlled competing-I/O and account/budget matrix evidence remain before broader claims or optimization. SSD/NVMe is optional, not a closure dependency. | +| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M017/M018/M019 ready — post-merge operability and browser transport | PR #7 merged as `88f58bbf` on 2026-10-15; M020–M023 have explicit hard dependencies. Keep M001–M016 closures immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | +| On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 ready — ordinary-server small-team synthetic capacity baseline | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 is independent measurement on ordinary on-prem Linux with durable SSD/NVMe (operational target required), not an authorization to change WAL/SQLite topology. | ## Dependency-ready implementation plans | Subsystem | Milestone | Status | Implementation plan | Dependencies / handoff note | |---|---|---|---|---| | Persistence | M011 — post-M010 roadmap and documentation reconciliation | ready | `plans/implementation/persistence/011-post-m010-roadmap-documentation-reconciliation.md` | Hard: M009/M010 closures accepted. Documentation/tooling only; no runtime or physical target dependency. | +| On-prem user access | M017 — managed client credential reconciliation | ready | `plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md` | Independent P1; repair post-bootstrap local/remote client credentials and end-to-end synthetic setup. | +| On-prem user access | M018 — local operator status corrective | ready | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | Independent P1; preserve privileged HTTP fences, inspect local OS control authority. | +| On-prem user access | M019 — browser transport and solo LAN corrective | ready | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | Independent P1; remote HTTP GET login form must not solicit credentials. | +| On-prem concurrency performance | M002 — ordinary-server team baseline | ready | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | Independent evidence work; ordinary Linux + durable SSD/NVMe required for team-capacity claims, no Pi/microSD release gate. | ## Active implementation plans @@ -69,6 +81,10 @@ pair is a known numbering accident. | Subsystem | Milestone | Blocker | |---|---|---| +| On-prem user access | M020 — managed transition and headless administration | Hard: M017 and M019 closure. | +| On-prem user access | M021 — default member profile and provisioning | Hard: M020 closure. | +| On-prem user access | M022 — hard-budget agent compatibility | Hard: M017 closure; never relax M008/M015 proof. | +| On-prem user access | M023 — short-window team fairness | Hard: M022 closure; ordinary-server evidence operational. | Unblock audit (on-prem user access reopening after M012 closure, baseline From cf64a86f57b6d180558bf64ef8608c20b263ba65 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 11:45:01 +0000 Subject: [PATCH 14/62] Clarify M016 as historical pre-merge hold after PR7 landing --- plans/subsystems/on-prem-user-access-roadmap.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/plans/subsystems/on-prem-user-access-roadmap.md b/plans/subsystems/on-prem-user-access-roadmap.md index 474f4582..6f15a965 100644 --- a/plans/subsystems/on-prem-user-access-roadmap.md +++ b/plans/subsystems/on-prem-user-access-roadmap.md @@ -275,13 +275,13 @@ remaining M015 finding. Closure evidence: ### M016 — Final integration and merge-readiness reconciliation -Class: polish. Closed as a pre-merge reconciliation on exact -head `3d68c715`. Local, browser, release-doc, Q007 and exact-head -hosted CI/installer/**cargo-deny** evidence pass. Recommendation: -**merge held pending maintainer review** because GitHub has no -submitted PR review. PR #7 remains open; the subsystem roadmap stays -active until a maintainer decides whether to merge. No automatic merge -was performed and no 49-concurrent capacity claim is made. Closure: +Class: polish. Closed as a **historical pre-merge reconciliation** +on exact head `3d68c715`. Local, browser, release-doc, Q007 and exact-head +hosted CI/installer/**cargo-deny** evidence passed. The M016 closure +recommended holding for maintainer review at that point; the PR was +subsequently merged on 2026-10-15 as `88f58bbf`. This current roadmap +tracks new M017–M023 post-merge work, not the old merge hold. No 49-concurrent +capacity claim was made. Closure: `plans/closure/on-prem-user-access/016-status.md`. Source: `plans/implementation/on-prem-user-access/016-final-pr7-merge-reconciliation.md`. From b837a6f4b5dd28b8e1d4b96a375f8a252cd9b09b Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 11:54:26 +0000 Subject: [PATCH 15/62] Activate on-prem access plan 017 --- .../017-managed-client-credential-reconciliation.md | 2 +- plans/registry.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md b/plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md index 097a554f..a4f89fc7 100644 --- a/plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md +++ b/plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md @@ -1,6 +1,6 @@ # On-Prem User Access M017 — Managed client credentials and integration reconciliation -Status: ready +Status: active Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, PR #7 merged). diff --git a/plans/registry.md b/plans/registry.md index 32d67572..a30acd5d 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -66,7 +66,6 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Dependencies / handoff note | |---|---|---|---|---| | Persistence | M011 — post-M010 roadmap and documentation reconciliation | ready | `plans/implementation/persistence/011-post-m010-roadmap-documentation-reconciliation.md` | Hard: M009/M010 closures accepted. Documentation/tooling only; no runtime or physical target dependency. | -| On-prem user access | M017 — managed client credential reconciliation | ready | `plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md` | Independent P1; repair post-bootstrap local/remote client credentials and end-to-end synthetic setup. | | On-prem user access | M018 — local operator status corrective | ready | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | Independent P1; preserve privileged HTTP fences, inspect local OS control authority. | | On-prem user access | M019 — browser transport and solo LAN corrective | ready | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | Independent P1; remote HTTP GET login form must not solicit credentials. | | On-prem concurrency performance | M002 — ordinary-server team baseline | ready | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | Independent evidence work; ordinary Linux + durable SSD/NVMe required for team-capacity claims, no Pi/microSD release gate. | @@ -75,6 +74,7 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| +| On-prem user access | M017 — managed client credential reconciliation | active | `plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md` | Repair post-bootstrap local/remote client credentials and end-to-end synthetic setup. | ## Blocked work From d726bcec5dc794e79290752cbffcef1d3115b7b0 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 11:57:49 +0000 Subject: [PATCH 16/62] Reconcile managed client credentials --- README.md | 16 ++--- architecture/deep-dive-integrations.md | 7 ++- rust/src/operations/integrations.rs | 45 ++++++++++---- rust/src/runtime.rs | 59 +++++++++++++++++-- rust/tests/operations_o005.rs | 29 ++++++++- .../test_on_prem_credential_guidance.py | 12 ++++ 6 files changed, 138 insertions(+), 30 deletions(-) create mode 100644 tests/tooling/test_on_prem_credential_guidance.py diff --git a/README.md b/README.md index cdcef1dc..3cc85e3e 100644 --- a/README.md +++ b/README.md @@ -73,7 +73,10 @@ eggpool update 0.8.0 # exact historical-version switch eggpool serve eggpool status -export EGGPOOL_API_KEY="$(eggpool getkey)" +# For a managed installation, create a personal credential for this user and +# set its one-time value in the client environment: +eggpool users token USER --show-secret +export EGGPOOL_API_KEY='' # Liveness / readiness (no key needed) curl http://127.0.0.1:11300/v1/healthz @@ -90,11 +93,10 @@ curl -H "Authorization: Bearer $EGGPOOL_API_KEY" \ http://127.0.0.1:11300/v1/chat/completions ``` -Inference (`/v1/*` except `/v1/healthz` and `/v1/readyz`), integration -(`/api/integrations/*`), and status endpoints (`/api/status`, -`/api/stats/runtime`, `/api/stats/update`) require the server key as -`Authorization: Bearer ` (OpenAI-style) or `x-api-key: ` -(Anthropic-style); without it they return `401`. To pin a provider, qualify +Managed inference and integration requests use the caller's personal token as +`Authorization: Bearer ` (OpenAI-style) or `x-api-key: ` +(Anthropic-style). Legacy, pre-bootstrap installs may continue using the +shared server key until migration; it does not grant managed-user access. To pin a provider, qualify the model as `model/provider` (e.g. `"gpt-4o/openai"`); `eggpool accounts explain --model ` shows per-account eligibility. Full endpoint list: [API reference](docs/api-reference.md). @@ -108,7 +110,7 @@ eggpool configsetup claude-code eggpool configremote codex # secret-free remote profile for other machines ``` -Generated configs reference `EGGPOOL_API_KEY` without embedding the key (`export EGGPOOL_API_KEY="$(eggpool getkey)"`). See [Agent Configuration](docs/agent-configuration.md) for all targets, the managed `--apply`/`--sync`/`--check`/`--remove` lifecycle, remote profiles, and the transactional `eggpool-connect` desktop helper. +Generated configs reference `EGGPOOL_API_KEY` without embedding the key. Set it to a personal token from `eggpool users token USER --show-secret`; `eggpool getkey` is only for legacy installs. See [Agent Configuration](docs/agent-configuration.md) for all targets, the managed `--apply`/`--sync`/`--check`/`--remove` lifecycle, remote profiles, and the transactional `eggpool-connect` desktop helper. ## Dashboard & LAN Access diff --git a/architecture/deep-dive-integrations.md b/architecture/deep-dive-integrations.md index 0cd8a826..18bbdf4d 100644 --- a/architecture/deep-dive-integrations.md +++ b/architecture/deep-dive-integrations.md @@ -61,9 +61,10 @@ OpenCode are the exceptions: Codex TOML contains only `env_key = "EGGPOOL_API_KEY"` and OpenCode JSON uses `{env:EGGPOOL_API_KEY}` interpolation, so both snippets are non-secret, print by default, and `--print-secret` does not alter them. The delivery hints tell -operators to set that environment variable and use `eggpool getkey` to -retrieve the current value. Configsetup never executes a shell or writes a -shell profile. +operators to set that environment variable to the user's personal token. +`eggpool getkey` remains a legacy-installation command and does not issue a +managed-user credential. Configsetup never executes a shell or writes a shell +profile. ## Provider-neutral agent projection diff --git a/rust/src/operations/integrations.rs b/rust/src/operations/integrations.rs index 91be2e41..ed18c492 100644 --- a/rust/src/operations/integrations.rs +++ b/rust/src/operations/integrations.rs @@ -240,14 +240,33 @@ pub struct Delivery { pub async fn build_integration_context( config_path: &Path, ) -> Result { - build_context(config_path, true, true).await + build_integration_context_with_credential(config_path, None).await +} + +/// Build a client context using an explicit personal credential when supplied. +pub async fn build_integration_context_with_credential( + config_path: &Path, + credential: Option<&str>, +) -> Result { + build_context(config_path, true, true, credential).await } /// Build only the endpoint/key context used by Claude Code. Its frozen /// command contract does not inspect catalog state or alter transcoder config. pub fn build_endpoint_context(config_path: &Path) -> Result { + build_endpoint_context_with_credential(config_path, None) +} + +/// Endpoint-only context with an explicit personal credential. +pub fn build_endpoint_context_with_credential( + config_path: &Path, + credential: Option<&str>, +) -> Result { let config = Config::from_toml(config_path)?; - let (api_key, config_mutated) = config_mutation::resolve_server_key(config_path)?; + let (api_key, config_mutated) = match credential { + Some(value) if !value.trim().is_empty() => (value.to_owned(), false), + _ => config_mutation::resolve_server_key(config_path)?, + }; let port = config.server.port; let host = detect_lan_ip(); Ok(IntegrationContext { @@ -268,9 +287,13 @@ async fn build_context( config_path: &Path, load_catalog: bool, enable_transcoder: bool, + credential: Option<&str>, ) -> Result { let config = Config::from_toml(config_path)?; - let (api_key, config_mutated) = config_mutation::resolve_server_key(config_path)?; + let (api_key, config_mutated) = match credential { + Some(value) if !value.trim().is_empty() => (value.to_owned(), false), + _ => config_mutation::resolve_server_key(config_path)?, + }; let port = config.server.port; let host = detect_lan_ip(); let mut models = if load_catalog { @@ -464,12 +487,10 @@ async fn build_remote_context_from_config( let base_url = resolve_advertised_base_url(config, base_url_override)?; let base_url_root = base_url.strip_suffix("/v1").unwrap_or(&base_url).to_owned(); let auth_env = REMOTE_AUTH_ENV.to_owned(); - let auth_configured = config.resolved_server_api_key().is_some(); - if !auth_configured { - return Err(IntegrationError::ProfileUnavailable { - detail: "server API key is not configured; run `eggpool newkey` or set [server].api_key before sharing a remote profile".to_owned(), - }); - } + // Remote profiles contain only an environment-variable reference. They + // remain useful in managed mode without a shared server key; the receiving + // operator supplies the member's personal token separately. + let auth_configured = true; let mut models = if read_catalog { match load_remote_catalog_models(config).await { Ok(models) => models, @@ -906,7 +927,7 @@ pub async fn deliver( Some(snippet.to_owned()) } else { messages.push( - "Secret not printed to stdout. Use --print-secret to include it, or use `eggpool getkey` to retrieve it." + "Secret not printed to stdout. Use --print-secret on a secure console, or issue a personal token with `eggpool users token USER --show-secret`." .to_owned(), ); None @@ -943,7 +964,7 @@ pub fn paste_hint(target: Target) -> Option<&'static str> { Target::Goose => Some("Export these variables before running Goose."), Target::Openhands => Some("Pass these environment variables to the OpenHands runtime."), Target::Codex => Some( - "Set EGGPOOL_API_KEY in the environment used to launch Codex; retrieve the current key with `eggpool getkey`.", + "Set EGGPOOL_API_KEY to the user's personal token in the environment used to launch Codex.", ), _ => None, } @@ -1621,7 +1642,7 @@ pub fn codex_lifecycle( .push("Codex config and catalog already current.".to_owned()); } report.messages.push( - "Set EGGPOOL_API_KEY in the environment used to launch Codex; retrieve the current key with `eggpool getkey`.".to_owned(), + "Set EGGPOOL_API_KEY to the user's personal token in the environment used to launch Codex.".to_owned(), ); Ok(report) } diff --git a/rust/src/runtime.rs b/rust/src/runtime.rs index 35ab27fe..30665aa2 100644 --- a/rust/src/runtime.rs +++ b/rust/src/runtime.rs @@ -1785,13 +1785,18 @@ async fn configsetup( ), }; + let personal_credential = integration_personal_credential(&pre_mutation_config).await?; + let mut context = if target == Target::ClaudeCode { - integrations::build_endpoint_context(path) + integrations::build_endpoint_context_with_credential(path, personal_credential.as_deref()) .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))? } else { - integrations::build_integration_context(path) - .await - .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))? + integrations::build_integration_context_with_credential( + path, + personal_credential.as_deref(), + ) + .await + .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))? }; if context.config_mutated { eprintln!("Generated new server API key."); @@ -1832,7 +1837,7 @@ async fn configsetup( .any(|message| message == "Copied config to clipboard."); if !copied { eprintln!( - "Could not copy to clipboard. Use `eggpool getkey` and pass --api-key to the Claude Code CLI." + "Could not copy to clipboard. Issue a personal token with `eggpool users token USER --show-secret` and pass it to the Claude Code CLI on a secure terminal." ); } for message in delivery.messages { @@ -1884,6 +1889,50 @@ async fn configsetup( Ok(()) } +/// Managed installs require an active personal token for local client setup. +/// Before identity bootstrap, preserve the legacy shared-key setup contract. +async fn integration_personal_credential( + config: &config::Config, +) -> Result, BootstrapError> { + let database_path = config::Config::runtime_path(&config.database.path); + if !database_path.is_file() { + return Ok(None); + } + let database = open_operational_database(config, true).await?; + let state = crate::identity::access_state(&database) + .await + .map_err(|error| command_error(EXIT_VALIDATION, error.to_string())); + let close_result = database.close().await; + let state = state?; + close_result.map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))?; + if !matches!(state.as_str(), "admin_initialized" | "managed") { + return Ok(None); + } + let token = env::var("EGGPOOL_API_KEY") + .ok() + .filter(|value| !value.trim().is_empty()); + let Some(token) = token else { + return Err(command_error( + EXIT_VALIDATION, + "managed client setup requires the member's personal token in EGGPOOL_API_KEY; do not use eggpool getkey or the retired shared server key", + )); + }; + let database = open_operational_database(config, true).await?; + let valid = crate::identity::verify_api_token(&database, &token) + .await + .map_err(|error| command_error(EXIT_VALIDATION, error.to_string())); + let close_result = database.close().await; + let valid = valid?; + close_result.map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))?; + if !valid { + return Err(command_error( + EXIT_VALIDATION, + "EGGPOOL_API_KEY is not an active personal credential for this installation", + )); + } + Ok(Some(token)) +} + async fn run_managed_configsetup( path: &Path, pre_mutation_config: &config::Config, diff --git a/rust/tests/operations_o005.rs b/rust/tests/operations_o005.rs index 69796f5d..9ca6137d 100644 --- a/rust/tests/operations_o005.rs +++ b/rust/tests/operations_o005.rs @@ -195,9 +195,9 @@ async fn default_delivery_hides_secrets_and_explicit_print_reveals_them() { Some("env_key = \"EGGPOOL_API_KEY\"") ); assert!( - codex.messages.iter().any( - |message| message.contains("EGGPOOL_API_KEY") && message.contains("eggpool getkey") - ) + codex.messages + .iter() + .any(|message| message.contains("EGGPOOL_API_KEY") && message.contains("personal token")) ); let codex_print_secret = deliver( @@ -615,6 +615,29 @@ async fn configremote_is_read_only_deterministic_and_secret_free() { assert_eq!(before, after); assert!(!db_path.exists()); + // A managed server has no shared key. The read-only remote profile is + // still exportable because its credential is supplied by the member. + let keyless_path = directory.path().join("managed-keyless.toml"); + fs::write( + &keyless_path, + format!( + "[server]\nhost = \"0.0.0.0\"\nport = 11300\n\n[database]\npath = \"{}\"\n\n[integrations]\nadvertise_base_url = \"https://pool.example.internal/v1\"\n", + directory.path().join("managed.sqlite3").display() + ), + ) + .expect("keyless managed config"); + let managed_remote = build_remote_context(&keyless_path, None) + .await + .expect("keyless remote context"); + assert!(managed_remote.auth_configured); + assert!( + remote_connection_token( + &managed_remote, + parse_remote_target("codex").expect("target") + ) + .is_ok() + ); + // Loopback-only without advertisement fails closed. let loopback_path = directory.path().join("loopback.toml"); fs::write( diff --git a/tests/tooling/test_on_prem_credential_guidance.py b/tests/tooling/test_on_prem_credential_guidance.py new file mode 100644 index 00000000..7de4e494 --- /dev/null +++ b/tests/tooling/test_on_prem_credential_guidance.py @@ -0,0 +1,12 @@ +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[2] + + +def test_first_request_uses_personal_token_after_managed_bootstrap() -> None: + readme = (ROOT / "README.md").read_text(encoding="utf-8") + assert "eggpool users token USER --show-secret" in readme + assert 'export EGGPOOL_API_KEY="$(eggpool getkey)"' not in readme + assert "it does not grant managed-user access" in readme + From 218df45f2b82858ee5a42ac8ada9fa33e6d62a56 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 12:06:22 +0000 Subject: [PATCH 17/62] Clarify personal credential delivery --- rust/src/operations/integrations.rs | 2 +- rust/tests/operations_o005.rs | 39 +++++++++++++++++++++++------ 2 files changed, 33 insertions(+), 8 deletions(-) diff --git a/rust/src/operations/integrations.rs b/rust/src/operations/integrations.rs index ed18c492..71b5895e 100644 --- a/rust/src/operations/integrations.rs +++ b/rust/src/operations/integrations.rs @@ -775,7 +775,7 @@ pub fn render_remote_human( ); lines.push(commands); lines.push(String::new()); - lines.push("Each desktop still needs its own authorized EggPool key".to_owned()); + lines.push("Each desktop still needs its user's personal EggPool token".to_owned()); lines.push( "in the environment that launches the client (EGGPOOL_API_KEY).".to_owned(), ); diff --git a/rust/tests/operations_o005.rs b/rust/tests/operations_o005.rs index 9ca6137d..429b770e 100644 --- a/rust/tests/operations_o005.rs +++ b/rust/tests/operations_o005.rs @@ -5,10 +5,10 @@ use std::{fs, os::unix::fs::PermissionsExt, path::PathBuf}; use eggpool::operations::integrations::{ IntegrationContext, IntegrationModel, LifecycleAction, LifecycleOptions, ModelLimits, SnippetOptions, Target, aggregate_projections, apply_overrides, build_codex_catalog_json, - build_integration_context, build_remote_context, codex_lifecycle, deliver, opencode_lifecycle, - parse_remote_target, paste_hint, project_context_models, project_model, - remote_connection_token, render_remote_json, render_target, resolve_advertised_base_url, - resolve_model, validate_codex_catalog_json, + build_integration_context, build_integration_context_with_credential, build_remote_context, + codex_lifecycle, deliver, opencode_lifecycle, parse_remote_target, paste_hint, + project_context_models, project_model, remote_connection_token, render_remote_json, + render_target, resolve_advertised_base_url, resolve_model, validate_codex_catalog_json, }; use serde_json::{Map, Value, json}; use tempfile::tempdir; @@ -195,9 +195,9 @@ async fn default_delivery_hides_secrets_and_explicit_print_reveals_them() { Some("env_key = \"EGGPOOL_API_KEY\"") ); assert!( - codex.messages - .iter() - .any(|message| message.contains("EGGPOOL_API_KEY") && message.contains("personal token")) + codex.messages.iter().any( + |message| message.contains("EGGPOOL_API_KEY") && message.contains("personal token") + ) ); let codex_print_secret = deliver( @@ -267,6 +267,31 @@ async fn context_uses_static_models_and_owns_transcoder_mutation() { assert!(config.contains("[transcoder]\nenabled = true")); } +#[tokio::test] +async fn explicit_personal_credential_does_not_resurrect_shared_server_key() { + let directory = tempdir().expect("temporary directory"); + let config_path = directory.path().join("managed.toml"); + fs::write( + &config_path, + format!( + "[server]\nhost = \"0.0.0.0\"\nport = 11300\n\n[database]\npath = \"{}\"\n", + directory.path().join("missing.sqlite3").display() + ), + ) + .expect("keyless config"); + + let context = build_integration_context_with_credential( + &config_path, + Some("epk1_0123456789abcdef_personal-secret"), + ) + .await + .expect("personal credential context"); + assert_eq!(context.api_key, "epk1_0123456789abcdef_personal-secret"); + assert!(!context.config_mutated); + let after = fs::read_to_string(&config_path).expect("config remains readable"); + assert!(!after.contains("api_key =")); +} + #[test] fn agent_projection_is_conservative_and_never_infers_from_names() { let tricky = IntegrationModel { From f83af45b9d3f8fe43297d1dc1294289951afb5ad Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 12:07:05 +0000 Subject: [PATCH 18/62] Start closing on-prem access plan 017 --- .../017-managed-client-credential-reconciliation.md | 2 +- plans/registry.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md b/plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md index a4f89fc7..25f89d2a 100644 --- a/plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md +++ b/plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md @@ -1,6 +1,6 @@ # On-Prem User Access M017 — Managed client credentials and integration reconciliation -Status: active +Status: closing Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, PR #7 merged). diff --git a/plans/registry.md b/plans/registry.md index a30acd5d..3670cbe8 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -74,7 +74,7 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| -| On-prem user access | M017 — managed client credential reconciliation | active | `plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md` | Repair post-bootstrap local/remote client credentials and end-to-end synthetic setup. | +| On-prem user access | M017 — managed client credential reconciliation | closing | `plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md` | Focused integration/auth/docs evidence gathered; closure record and dependency audit in progress. | ## Blocked work From cdfb6fa8d4aaa1d9d5b43af8c87eaac68f83c402 Mon Sep 17 00:00:00 2001 From: wr3n-ai Date: Thu, 15 Oct 2026 12:10:14 +0000 Subject: [PATCH 19/62] Add ordinary-server qualification corpus --- architecture/deep-dive-observability.md | 8 + docs/deployment.md | 30 ++ docs/raspberry-pi.md | 3 + .../002-ordinary-server-team-baseline.md | 2 +- plans/registry.md | 2 +- scripts/qualification_ordinary_server.py | 373 ++++++++++++++++++ scripts/qualification_sbc.py | 254 +++++++++++- .../test_qualification_ordinary_server.py | 124 ++++++ tests/tooling/test_qualification_sbc.py | 25 ++ 9 files changed, 807 insertions(+), 14 deletions(-) create mode 100644 scripts/qualification_ordinary_server.py create mode 100644 tests/tooling/test_qualification_ordinary_server.py diff --git a/architecture/deep-dive-observability.md b/architecture/deep-dive-observability.md index 00a42799..caf8d726 100644 --- a/architecture/deep-dive-observability.md +++ b/architecture/deep-dive-observability.md @@ -19,6 +19,14 @@ bodies, cache keys, token values, or raw upstream error prose. | Streaming-outcome counters | `rust/src/coordinator/streaming/diagnostics.rs` (`StreamDiagnostics`, `StreamDiagnosticEvent`) | | Operator rollups, explain plans, dashboard queries | `rust/src/operations/operator.rs` over `rust/src/db/` repositories | +The offline M002 host-capacity corpus is owned by +`scripts/qualification_ordinary_server.py` and its Q008 runner. It records +sanitized host/build identity, storage class, host I/O pressure, response +percentiles, and durable settlement for synthetic loopback workloads. These +artifacts are qualification evidence, not runtime telemetry or a production +capacity promise; the SQLite gate timing diagnostic remains opt-in and is not +inferred from host I/O counters. + See [Metrics](deep-dive-metrics.md) for the coalescer contract (including the M002 ownership paragraph, which is authoritative) and [Dashboard](deep-dive-dashboard.md) for the HTTP/CLI presentation layer. diff --git a/docs/deployment.md b/docs/deployment.md index 2ede58d4..1e4afd4c 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -469,6 +469,36 @@ descriptive observations rather than universal RSS or latency gates, and keep EggPool-local `local_pre_upstream`/`dispatch_overhead` separate from upstream connect, header, TTFT, and total response latency. +### Ordinary-server team sizing evidence + +The M002 synthetic reference uses an ordinary Linux host with durable +SSD/NVMe storage, WAL/NORMAL, and one database worker. It records CPU, RAM, +filesystem, device class/model, virtualization/container status, process RSS, +request throughput and percentiles, I/O pressure, and durable reservation +settlement. Named account rows (1/5/10/25/49) are measured separately from +simultaneous inference requests (1/2/4/8/16); neither count is a service +guarantee. The synthetic upstream cycles a fixed 0/10/50 ms service-time +distribution to show how provider latency affects observed response time. + +The driver writes the manifest, raw runs, and generated summary under +`artifacts/qualification/on-prem-concurrency-performance/ordinary-server/`. +Until a complete run is recorded, there is no M002 capacity result. Once +available, use it only for relative sizing; qualify the actual provider mix, +model payloads, login rate, and policy/budget profile before deployment. No +universal throughput, concurrency, or latency SLA follows from one host and +synthetic workload. The Pi microSD envelope remains a separate solo-device +reference in [`raspberry-pi.md`](raspberry-pi.md). + +To repeat the frozen corpus from a release binary, run +`uv run python scripts/qualification_ordinary_server.py --binary `. +The driver writes its manifest before collection, creates a fresh qualification +root for each run, requires three repetitions per condition, validates p99 +sample counts and settlement, and derives `summary.json` from the raw Q008 +reports. It refuses non-Linux hosts and non-durable or rotational qualification +storage, and stops before load when initial full I/O pressure exceeds 25% over +the 10-second window. Qualification does not send requests to a real provider +or charge a provider account. + ### Lean (default) Recommended for Raspberry Pi and low-power devices. Optimized for diff --git a/docs/raspberry-pi.md b/docs/raspberry-pi.md index 77be7dd1..3ebab53d 100644 --- a/docs/raspberry-pi.md +++ b/docs/raspberry-pi.md @@ -272,6 +272,9 @@ Le Potato and a separate ordinary-Linux reference host were not measured here deltas); 16-way concurrency was not attempted because 8-way already saturates. +The ordinary-server team baseline is maintained separately from this Pi +microSD solo envelope; do not use the Pi limits as the team-host sizing source. + ## Verify from LAN 1. Find Pi IP: `hostname -I` diff --git a/plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md b/plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md index 40810431..372b35fc 100644 --- a/plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md +++ b/plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md @@ -1,6 +1,6 @@ # On-Prem Concurrency Performance M002 — Ordinary-server small-team capacity baseline -Status: ready +Status: active Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index 32d67572..fcd3acad 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -69,12 +69,12 @@ No new runtime, schema or performance claim lands with these plans. | On-prem user access | M017 — managed client credential reconciliation | ready | `plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md` | Independent P1; repair post-bootstrap local/remote client credentials and end-to-end synthetic setup. | | On-prem user access | M018 — local operator status corrective | ready | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | Independent P1; preserve privileged HTTP fences, inspect local OS control authority. | | On-prem user access | M019 — browser transport and solo LAN corrective | ready | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | Independent P1; remote HTTP GET login form must not solicit credentials. | -| On-prem concurrency performance | M002 — ordinary-server team baseline | ready | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | Independent evidence work; ordinary Linux + durable SSD/NVMe required for team-capacity claims, no Pi/microSD release gate. | ## Active implementation plans | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| +| On-prem concurrency performance | M002 — ordinary-server team baseline | active | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | Host qualification and frozen synthetic matrix in progress; no runtime or SQLite-policy change. | ## Blocked work diff --git a/scripts/qualification_ordinary_server.py b/scripts/qualification_ordinary_server.py new file mode 100644 index 00000000..eefe009a --- /dev/null +++ b/scripts/qualification_ordinary_server.py @@ -0,0 +1,373 @@ +"""Run and summarize the frozen M002 ordinary-server qualification corpus. + +The matrix separates named-account metadata from simultaneous requests. Each +cell gets a fresh Q008 root and three independent runs; the tool writes the +manifest before running any cell and only emits aggregate scalar summaries. +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import statistics +import subprocess +import sys +from pathlib import Path +from typing import Any, cast + +from scripts.qualification_sbc import ( + ORDINARY_SERVER_MAX_INITIAL_IO_PRESSURE_PERCENT, + host_io_full_avg10, + host_io_snapshot, + ordinary_server_metadata, +) + +ROOT = Path(__file__).resolve().parents[1] +DEFAULT_OUTPUT = ( + ROOT / "artifacts/qualification/on-prem-concurrency-performance/ordinary-server" +) +SCHEMA_VERSION = "on-prem-ordinary-server-matrix.v1" +SUMMARY_VERSION = "on-prem-ordinary-server-summary.v1" +REPETITIONS = 3 +REQUESTS_PER_CELL = 1000 +SEQUENTIAL_SAMPLES = 30 +PRINCIPAL_CELLS = (1, 5, 10, 25, 49) +CONCURRENCY_CELLS = (1, 2, 4, 8, 16) +PROVIDER_LATENCY_PROFILE = "mixed-0-10-50ms" + + +def _sha256(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as stream: + for chunk in iter(lambda: stream.read(1024 * 1024), b""): + digest.update(chunk) + return digest.hexdigest() + + +def _commit() -> str: + result = subprocess.run( + ["git", "rev-parse", "HEAD"], + cwd=ROOT, + capture_output=True, + text=True, + timeout=5, + check=False, + ) + value = result.stdout.strip() + return value if result.returncode == 0 and len(value) == 40 else "unavailable" + + +def _conditions() -> list[dict[str, int]]: + pairs = {(users, 1) for users in PRINCIPAL_CELLS} + pairs.update((5, concurrency) for concurrency in CONCURRENCY_CELLS) + return [ + {"principals": principals, "concurrency": concurrency} + for principals, concurrency in sorted(pairs) + ] + + +def _io_pressure_safe(snapshot: dict[str, Any]) -> bool: + avg10 = host_io_full_avg10(snapshot) + return avg10 is not None and avg10 < ORDINARY_SERVER_MAX_INITIAL_IO_PRESSURE_PERCENT + + +def _manifest(binary: Path, output_dir: Path) -> dict[str, Any]: + target, reason = ordinary_server_metadata() + if reason or target is None: + raise RuntimeError(reason or "ordinary-server host facts unavailable") + if ( + target["root_filesystem"] in {"tmpfs", "ramfs", "overlay"} + or target["root_storage_medium_class"] != "non-rotational" + or target["root_storage_device_class"] not in {"nvme", "scsi-disk"} + ): + raise RuntimeError("ordinary-server host root must use persistent SSD/NVMe") + initial_io = host_io_snapshot() + io_safe = _io_pressure_safe(initial_io) + return { + "schema_version": SCHEMA_VERSION, + "plan": "on-prem-concurrency-performance M002", + "status": "ready" if io_safe else "blocked", + "preflight_io": initial_io, + "stop_reason": ( + None + if io_safe + else ( + "initial full I/O pressure avg10 is missing or exceeds " + "the 25% stop gate" + ) + ), + "candidate": { + "sha256": _sha256(binary), + "repository_commit_sha": _commit(), + "build_profile": "release", + "build_mode": "locked release candidate", + }, + "target": target, + "workload": { + "fresh_root_per_run": True, + "named_principal_counts": list(PRINCIPAL_CELLS), + "principal_separation_condition": "concurrency=1", + "concurrent_request_counts": list(CONCURRENCY_CELLS), + "concurrency_separation_condition": "principals=5", + "requests_per_condition": REQUESTS_PER_CELL, + "repetitions_per_condition": REPETITIONS, + "sequential_wire_samples_per_surface": SEQUENTIAL_SAMPLES, + "provider_latency_profile": PROVIDER_LATENCY_PROFILE, + "budget_mode": "user-global hard request limits", + "surfaces": [ + "native Responses finite", + "native Responses streaming", + "translated Responses-to-Messages streaming", + ], + }, + "gates": { + "all_requests_complete": True, + "zero_http_failures": True, + "zero_pending_requests_after_stabilization": True, + "zero_active_reservations_after_stabilization": True, + "p99_minimum_completed_observations": 1000, + "three_repetitions_required": True, + "maximum_initial_io_pressure_full_avg10_percent": ( + ORDINARY_SERVER_MAX_INITIAL_IO_PRESSURE_PERCENT + ), + "no_capacity_sla_claim": True, + }, + "conditions": [ + { + **condition, + "run_artifacts": [ + f"u{condition['principals']}-c{condition['concurrency']}-run-{run}.json" + for run in range(1, REPETITIONS + 1) + ], + } + for condition in _conditions() + ], + "artifacts_directory": output_dir.name, + } + + +def _load(path: Path) -> dict[str, Any]: + value = json.loads(path.read_text(encoding="utf-8")) + if not isinstance(value, dict): + raise ValueError(f"artifact is not an object: {path.name}") + return cast("dict[str, Any]", value) + + +def summarize(manifest_path: Path) -> dict[str, Any]: + """Validate every raw cell against the frozen gates and aggregate it.""" + manifest = _load(manifest_path) + if manifest.get("schema_version") != SCHEMA_VERSION: + raise ValueError("ordinary-server manifest schema is unsupported") + if manifest.get("status") != "complete": + raise ValueError("ordinary-server matrix is incomplete") + candidate = manifest.get("candidate") + if not isinstance(candidate, dict): + raise ValueError("candidate identity is missing") + candidate = cast("dict[str, Any]", candidate) + output_dir = manifest_path.parent + conditions = manifest.get("conditions") + if not isinstance(conditions, list): + raise ValueError("condition matrix is missing") + conditions = cast("list[Any]", conditions) + summaries: list[dict[str, Any]] = [] + for raw_condition in conditions: + if not isinstance(raw_condition, dict): + raise ValueError("condition entry is invalid") + condition = cast("dict[str, Any]", raw_condition) + principals = condition.get("principals") + concurrency = condition.get("concurrency") + raw_names = condition.get("run_artifacts") + if not isinstance(principals, int) or not isinstance(concurrency, int): + raise ValueError("condition counts are invalid") + if not isinstance(raw_names, list): + raise ValueError("each condition must name exactly three raw runs") + raw_names = cast("list[Any]", raw_names) + if len(raw_names) != REPETITIONS: + raise ValueError("each condition must name exactly three raw runs") + runs: list[dict[str, Any]] = [] + for name in raw_names: + if not isinstance(name, str) or Path(name).name != name: + raise ValueError("run artifact name is unsafe") + document = _load(output_dir / name) + if document.get("status") != "pass": + raise ValueError(f"qualification run did not pass: {name}") + if document.get("schema_version") != "runtime-q008.v2": + raise ValueError(f"unexpected Q008 schema: {name}") + run_candidate = document.get("candidate") + if not isinstance(run_candidate, dict): + raise ValueError(f"candidate identity missing: {name}") + run_candidate = cast("dict[str, Any]", run_candidate) + if run_candidate.get("sha256") != candidate.get("sha256"): + raise ValueError(f"candidate hash changed: {name}") + environment = document.get("environment", {}) + if environment.get("target_class") != "ordinary-server": + raise ValueError(f"ordinary-server attestation missing: {name}") + database_storage = environment.get("qualification_database_storage", {}) + if ( + database_storage.get("filesystem") in {"tmpfs", "ramfs", "overlay"} + or database_storage.get("storage_medium_class") != "non-rotational" + or database_storage.get("device_class") not in {"nvme", "scsi-disk"} + ): + raise ValueError(f"run database was not on durable SSD/NVMe: {name}") + benchmark = document.get("benchmark", {}) + if ( + benchmark.get("principal_count") != principals + or benchmark.get("budget_mode") != "user-global" + or benchmark.get("provider_latency_profile") != PROVIDER_LATENCY_PROFILE + ): + raise ValueError(f"workload does not match condition: {name}") + load = benchmark.get("runs", {}).get( + "concurrent_native_responses_finite", {} + ) + timing = load.get("timings", {}) + if ( + load.get("client_concurrency") != concurrency + or load.get("request_count") != REQUESTS_PER_CELL + or load.get("completed_count") != REQUESTS_PER_CELL + or load.get("failed_count") != 0 + or timing.get("sample_count") != REQUESTS_PER_CELL + or "p99_elapsed_ms" not in timing + ): + raise ValueError(f"request or p99 gate failed: {name}") + durable = document.get("durable", {}) + if ( + durable.get("pending_requests") != 0 + or durable.get("active_reservations") != 0 + ): + raise ValueError(f"durable reservations did not converge: {name}") + runs.append({"artifact": name, "document": document, "load": load}) + + def median( + field: str, *, cell_runs: list[dict[str, Any]] = runs + ) -> int | float | None: + values = [run["load"][field] for run in cell_runs] + if not all(isinstance(value, (int, float)) for value in values): + return None + return statistics.median(values) + + cpu_values = [ + run["load"]["cpu"].get("cpu_percent") + for run in runs + if isinstance(run["load"].get("cpu"), dict) + ] + + def timing_median( + field: str, *, cell_runs: list[dict[str, Any]] = runs + ) -> int | float | None: + values = [run["load"]["timings"].get(field) for run in cell_runs] + if not all(isinstance(value, (int, float)) for value in values): + return None + return statistics.median(values) + + summaries.append( + { + "principals": principals, + "concurrency": concurrency, + "repetitions": len(runs), + "requests_per_run": REQUESTS_PER_CELL, + "median_requests_per_second": median("requests_per_second"), + "median_cpu_percent": statistics.median(cpu_values) + if cpu_values + and all(isinstance(value, (int, float)) for value in cpu_values) + else None, + "median_p50_ms": timing_median("p50_elapsed_ms"), + "median_p95_ms": timing_median("p95_elapsed_ms"), + "median_p99_ms": timing_median("p99_elapsed_ms"), + "maximum_ms": max( + run["load"]["timings"]["maximum_elapsed_ms"] for run in runs + ), + "maximum_peak_rss_bytes": max( + run["document"]["benchmark"]["resource_summary"]["peak_rss_bytes"] + for run in runs + ), + "settlement": ( + "zero pending requests and active reservations in every run" + ), + "raw_runs": [run["artifact"] for run in runs], + } + ) + return { + "schema_version": SUMMARY_VERSION, + "plan": manifest.get("plan"), + "candidate": candidate, + "target": manifest.get("target"), + "workload": manifest.get("workload"), + "interpretation": ( + "Descriptive loopback capacity characterization for this host and " + "synthetic workload; not an SLA or provider-throughput promise." + ), + "conditions": summaries, + } + + +def run(binary: Path, output_dir: Path) -> int: + output_dir.mkdir(parents=True, exist_ok=True) + manifest_path = output_dir / "matrix.json" + summary_path = output_dir / "summary.json" + manifest = _manifest(binary, output_dir) + manifest_path.write_text(json.dumps(manifest, indent=2) + "\n", encoding="utf-8") + if manifest["status"] == "blocked": + return 77 + for condition in manifest["conditions"]: + for run_number, name in enumerate(condition["run_artifacts"], start=1): + output = output_dir / name + command = [ + sys.executable, + str(ROOT / "scripts/qualification_sbc.py"), + "--binary", + str(binary), + "--target-class", + "ordinary-server", + "--config-fixture", + str(ROOT / "tests/tooling/fixtures/qualification/sbc-benchmark.toml"), + "--benchmark-samples", + str(SEQUENTIAL_SAMPLES), + "--concurrency-workers", + str(condition["concurrency"]), + "--concurrency-requests", + str(REQUESTS_PER_CELL), + "--benchmark-principals", + str(condition["principals"]), + "--benchmark-budget-mode", + "user-global", + "--provider-latency-profile", + PROVIDER_LATENCY_PROFILE, + "--candidate-origin", + "on-device-release-build", + "--output", + str(output), + ] + result = subprocess.run(command, cwd=ROOT, check=False) + if result.returncode != 0: + manifest["status"] = "incomplete" + manifest["stopped_at"] = { + "artifact": name, + "run_number": run_number, + "returncode": result.returncode, + } + manifest_path.write_text( + json.dumps(manifest, indent=2) + "\n", encoding="utf-8" + ) + return result.returncode + manifest["status"] = "complete" + manifest_path.write_text(json.dumps(manifest, indent=2) + "\n", encoding="utf-8") + summary = summarize(manifest_path) + summary_path.write_text(json.dumps(summary, indent=2) + "\n", encoding="utf-8") + return 0 + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--binary", type=Path, required=True) + parser.add_argument("--output-dir", type=Path, default=DEFAULT_OUTPUT) + args = parser.parse_args(argv) + try: + return run(args.binary, args.output_dir) + except (OSError, RuntimeError, ValueError, subprocess.TimeoutExpired) as error: + parser.error(str(error)) + return 2 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/qualification_sbc.py b/scripts/qualification_sbc.py index aca84a96..ebe98951 100644 --- a/scripts/qualification_sbc.py +++ b/scripts/qualification_sbc.py @@ -87,11 +87,12 @@ BENCHMARK_WARMUPS = 5 CONCURRENCY_BATCH_SIZE = 32 CONCURRENCY_WORKERS = 4 -CONCURRENCY_WORKERS_MAX = 8 +CONCURRENCY_WORKERS_MAX = 16 CONCURRENCY_REQUESTS_MAX = 4000 P99_MIN_SAMPLES = 1000 COMPETING_IO_MAX_MIB_PER_SECOND = 4 COMPETING_IO_MAX_MIB = 128 +ORDINARY_SERVER_MAX_INITIAL_IO_PRESSURE_PERCENT = 25.0 SYNTHETIC_ADMIN_PASSWORD = "SBC synthetic bootstrap password 123" STABILIZATION_SECONDS = 3.0 DIAGNOSTIC_MIN_SAMPLES = 10 @@ -379,6 +380,70 @@ def _path_storage_metadata(path: Path) -> dict[str, str | None]: } +def host_io_snapshot() -> dict[str, Any]: + """Read aggregate root-device bytes and Linux I/O pressure, when available.""" + root_device = _root_block_device(_root_mount_source()) + device_stats = ( + _read_text(Path(f"/sys/class/block/{root_device}/stat")) + if root_device + else None + ) + read_bytes = write_bytes = None + if device_stats: + fields = device_stats.split() + if len(fields) >= 7: + with contextlib.suppress(ValueError): + read_bytes = int(fields[2]) * 512 + write_bytes = int(fields[6]) * 512 + pressure: dict[str, dict[str, float]] = {} + pressure_text = _read_text(Path("/proc/pressure/io")) or "" + for line in pressure_text.splitlines(): + fields = line.split() + if not fields or fields[0] not in {"some", "full"}: + continue + values: dict[str, float] = {} + for item in fields[1:]: + key, separator, raw = item.partition("=") + if separator and key in {"avg10", "avg60", "avg300"}: + with contextlib.suppress(ValueError): + values[key] = float(raw) + pressure[fields[0]] = values + return { + "root_device_read_bytes": read_bytes, + "root_device_write_bytes": write_bytes, + "io_pressure": pressure, + } + + +def host_io_full_avg10(snapshot: Mapping[str, Any]) -> float | None: + pressure = snapshot.get("io_pressure") + if not isinstance(pressure, Mapping): + return None + pressure = cast("Mapping[str, Any]", pressure) + full = pressure.get("full") + if not isinstance(full, Mapping): + return None + full = cast("Mapping[str, Any]", full) + value = full.get("avg10") + return float(value) if isinstance(value, (int, float)) else None + + +def _host_io_delta( + before: Mapping[str, Any], after: Mapping[str, Any] +) -> dict[str, Any]: + deltas: dict[str, Any] = {} + for key in ("root_device_read_bytes", "root_device_write_bytes"): + start, end = before.get(key), after.get(key) + deltas[key.removesuffix("_bytes") + "_delta_bytes"] = ( + end - start + if isinstance(start, int) and isinstance(end, int) and end >= start + else None + ) + deltas["io_pressure_before"] = before.get("io_pressure") + deltas["io_pressure_after"] = after.get("io_pressure") + return deltas + + def board_metadata() -> tuple[dict[str, Any] | None, str | None]: """Return board facts, or why physical SBC evidence is unavailable.""" if platform.system().lower() != "linux": @@ -477,6 +542,94 @@ def board_metadata() -> tuple[dict[str, Any] | None, str | None]: }, None +def ordinary_server_metadata() -> tuple[dict[str, Any] | None, str | None]: + """Describe an ordinary Linux host without claiming it is bare metal.""" + if platform.system().lower() != "linux": + return None, "ordinary-server qualification requires Linux" + + root_source = _root_mount_source() + root_device = _root_block_device(root_source) + storage = _path_storage_metadata(Path("/")) + mem_total = _read_text(Path("/proc/meminfo")) or "" + memory_kib_match = re.search(r"(?m)^MemTotal:\s+(\d+)\s+kB", mem_total) + cpu_model = None + cpu_info = _read_text(Path("/proc/cpuinfo")) or "" + cpu_match = re.search(r"(?m)^(?:model name|Hardware)\s*:\s*(.+)$", cpu_info) + if cpu_match: + cpu_model = cpu_match.group(1).strip()[:160] + dmi = { + name: _read_text(Path(f"/sys/devices/virtual/dmi/id/{name}")) + for name in ("sys_vendor", "product_name", "board_name") + } + disk_model = ( + _read_text(Path(f"/sys/class/block/{root_device}/device/model")) + if root_device + else None + ) + mounts = _read_text(Path("/proc/mounts")) or "" + root_mount_options = None + for line in mounts.splitlines(): + fields = line.split() + if len(fields) >= 4 and fields[1] == "/": + root_mount_options = fields[3][:128] + break + try: + virtualization = subprocess.run( + ["systemd-detect-virt"], + capture_output=True, + text=True, + timeout=2, + check=False, + ) + virtualization_type = ( + "none" + if virtualization.returncode == 1 + else virtualization.stdout.strip()[:64] + if virtualization.returncode == 0 + else "unavailable" + ) + except (OSError, subprocess.TimeoutExpired): + virtualization_type = "unavailable" + container_type = ( + _read_text(Path("/run/systemd/container")) + or ("docker-marker" if Path("/.dockerenv").exists() else None) + or ( + "container-cgroup" + if re.search( + r"(?i)docker|containerd|kubepods|lxc", + _read_text(Path("/proc/1/cgroup")) or "", + ) + else "none" + ) + ) + release = _os_release() + return { + "target_class": "ordinary-server", + "architecture": platform.machine().lower(), + "kernel": platform.release()[:128], + "os": release.get("PRETTY_NAME", "unknown"), + "os_version_id": release.get("VERSION_ID", "unknown"), + "cpu_model": cpu_model, + "logical_cpu_count": os.cpu_count(), + "ram_bytes": int(memory_kib_match.group(1)) * 1024 + if memory_kib_match + else None, + "system_vendor": dmi["sys_vendor"], + "product_name": dmi["product_name"], + "board_name": dmi["board_name"], + "root_filesystem": storage["filesystem"], + "root_storage_medium_class": storage["storage_medium_class"], + "root_storage_device_class": storage["device_class"], + "root_storage_device_model": disk_model[:128] if disk_model else None, + "root_mount_options": root_mount_options, + "virtualization_type": virtualization_type, + "container_type": container_type, + "attestation": ( + "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt" + ), + }, None + + class _LoopbackHandler(BaseHTTPRequestHandler): protocol_version = "HTTP/1.1" owner: ClassVar[LoopbackProvider] @@ -503,7 +656,9 @@ def do_POST(self) -> None: # noqa: N802 try: length = int(self.headers.get("content-length", "0")) body = self.rfile.read(min(length, MAX_HTTP_BODY_BYTES)) - self.owner.record(self.path) + delay_ms = self.owner.record(self.path) + if delay_ms: + time.sleep(delay_ms / 1000) streaming = b'"stream":true' in body.replace(b" ", b"") if self.path.endswith("/chat/completions"): response = ( @@ -601,7 +756,10 @@ def _respond( class LoopbackProvider: """Threaded loopback-only provider with fixed-bucket request counters.""" - def __init__(self) -> None: + def __init__(self, latency_profile: str = "instant") -> None: + if latency_profile not in {"instant", "mixed-0-10-50ms"}: + raise ValueError("unknown loopback provider latency profile") + self.latency_profile = latency_profile self.requests = 0 self._lock = threading.Lock() self._diagnostic_condition = threading.Condition(self._lock) @@ -618,14 +776,17 @@ def __init__(self) -> None: _LoopbackHandler.owner = self self.thread = threading.Thread(target=self.server.serve_forever, daemon=True) - def record(self, path: str) -> None: - """Count one loopback request in a fixed bucket; no URLs retained.""" + def record(self, path: str) -> int: + """Count one request and return its deterministic synthetic service delay.""" with self._lock: self.requests += 1 for bucket in self._path_counts: if path.endswith(bucket): self._path_counts[bucket] += 1 break + if self.latency_profile == "mixed-0-10-50ms": + return (0, 0, 0, 10, 10, 50)[(self.requests - 1) % 6] + return 0 def path_counts(self) -> dict[str, int]: with self._lock: @@ -2450,7 +2611,7 @@ def _concurrent_finite_benchmark( ) -> dict[str, Any]: """Run a bounded finite batch with optional same-filesystem I/O load.""" if not 1 <= client_concurrency <= CONCURRENCY_WORKERS_MAX: - raise ValueError("concurrent benchmark workers must be between 1 and 8") + raise ValueError("concurrent benchmark workers must be between 1 and 16") if not client_concurrency <= request_count <= CONCURRENCY_REQUESTS_MAX: raise ValueError( "concurrent benchmark requests must be between worker count and 4000" @@ -2509,6 +2670,7 @@ def write_competing_io() -> None: io_thread.join(timeout=5) raise QualificationError("competing I/O worker did not become ready") + host_io_before = host_io_snapshot() first_cpu = _proc_cpu(process.pid) started = time.monotonic() @@ -2536,6 +2698,7 @@ def request_once(item: int) -> tuple[int, bytes, int, int | None]: raise QualificationError("competing I/O worker failed") elapsed_seconds = max(time.monotonic() - started, 0.001) second_cpu = _proc_cpu(process.pid) + host_io_after = host_io_snapshot() elapsed_ms = [result[2] for result in results] ttft_ms = [result[3] for result in results if result[3] is not None] completed = sum(1 for status, body, _, _ in results if status == 200 and body) @@ -2553,6 +2716,7 @@ def request_once(item: int) -> tuple[int, bytes, int, int | None]: "batch_elapsed_ms": round(elapsed_seconds * 1000), "requests_per_second": round(request_count / elapsed_seconds, 3), "cpu": _cpu_batch_summary(first_cpu, second_cpu, elapsed_seconds), + "host_io": _host_io_delta(host_io_before, host_io_after), "timings": _timing_summary(elapsed_ms, ttft_ms, include_p99=True), "competing_io": { "enabled": bool(competing_io_max_mib), @@ -2990,6 +3154,8 @@ def run_qualification( competing_io_max_mib: int = 0, benchmark_principals: int = 1, benchmark_budget_mode: str = "none", + target_class: str = "sbc", + provider_latency_profile: str = "instant", ) -> dict[str, Any]: """Run SBC qualification and return a bounded machine-readable report.""" if benchmark_samples < 0 or benchmark_samples > BENCHMARK_MAX_SAMPLES: @@ -2997,7 +3163,7 @@ def run_qualification( f"benchmark sample count must be 0 or 1..{BENCHMARK_MAX_SAMPLES}" ) if not 1 <= concurrency_workers <= CONCURRENCY_WORKERS_MAX: - raise ValueError("concurrency workers must be between 1 and 8") + raise ValueError("concurrency workers must be between 1 and 16") if not concurrency_workers <= concurrency_requests <= CONCURRENCY_REQUESTS_MAX: raise ValueError("concurrency requests must be between workers and 4000") if (competing_io_mib_per_second == 0) != (competing_io_max_mib == 0): @@ -3010,6 +3176,10 @@ def run_qualification( raise ValueError("benchmark principals must be between 1 and 49") if benchmark_budget_mode not in {"none", "user", "global", "user-global"}: raise ValueError("invalid benchmark budget mode") + if target_class not in {"sbc", "ordinary-server"}: + raise ValueError("target class must be sbc or ordinary-server") + if provider_latency_profile not in {"instant", "mixed-0-10-50ms"}: + raise ValueError("invalid provider latency profile") if benchmark_samples == 0 and ( concurrency_workers != CONCURRENCY_WORKERS or concurrency_requests != CONCURRENCY_BATCH_SIZE @@ -3147,7 +3317,11 @@ def run_qualification( ) report: dict[str, Any] = { "schema_version": SCHEMA_V2 if benchmark_mode else SCHEMA_V1, - "plan": "SBC qualification", + "plan": ( + "SBC qualification" + if target_class == "sbc" + else "ordinary-server qualification" + ), "manifest": MANIFEST_VERSION, "status": "blocked", "functional": [], @@ -3155,7 +3329,10 @@ def run_qualification( "commands": [], "findings": [], } - board, block_reason = board_metadata() + if target_class == "sbc": + board, block_reason = board_metadata() + else: + board, block_reason = ordinary_server_metadata() if block_reason: report["reason"] = block_reason report["environment"] = { @@ -3165,6 +3342,20 @@ def run_qualification( return report environment = cast("dict[str, Any]", board) report["environment"] = environment + if target_class == "ordinary-server" and benchmark_mode: + preflight_io = host_io_snapshot() + environment["preflight_io"] = preflight_io + pressure = host_io_full_avg10(preflight_io) + if ( + pressure is None + or pressure >= ORDINARY_SERVER_MAX_INITIAL_IO_PRESSURE_PERCENT + ): + report["status"] = "blocked" + report["reason"] = ( + "ordinary-server benchmark stopped before load because initial " + "full I/O pressure is missing or at/above the 25% avg10 gate" + ) + return report if not binary.is_file(): report["status"] = "fail" report["reason"] = "candidate binary does not exist" @@ -3210,6 +3401,20 @@ def run_qualification( ): (root / name).mkdir() database = root / "data-home" / "usage.sqlite3" + if target_class == "ordinary-server" and benchmark_samples: + database_storage = _path_storage_metadata(database) + report["environment"]["qualification_database_storage"] = database_storage + if ( + database_storage["storage_medium_class"] != "non-rotational" + or database_storage["device_class"] not in {"nvme", "scsi-disk"} + or database_storage["filesystem"] in {"tmpfs", "ramfs", "overlay"} + ): + report["status"] = "blocked" + report["reason"] = ( + "ordinary-server benchmark database must use durable " + "SSD/NVMe storage, not an ephemeral filesystem" + ) + return report backup_dir = root / "backups" config = root / "config.toml" recovery_root = root / "recovery" @@ -3243,7 +3448,10 @@ def run_qualification( "second_workload_elapsed_ms": [], "second_workload_ttft_ms": [], } - with contextlib.ExitStack() as resources, LoopbackProvider() as provider: + with ( + contextlib.ExitStack() as resources, + LoopbackProvider(provider_latency_profile) as provider, + ): if ( diagnose_publication_storage is not None or diagnose_publication_phases ) and diagnostic_database_dir is not None: @@ -4270,6 +4478,7 @@ def run_qualification( "client_concurrency": concurrency_workers, "database_storage": _path_storage_metadata(database), "config_fixture": Path(config_fixture).name, + "provider_latency_profile": provider_latency_profile, "cadence": benchmark_cadence_facts(content), "runs": {}, } @@ -4647,7 +4856,11 @@ def benchmark_sample( report["commands"] = [item.as_dict(root, binary, database) for item in commands] report["isolated_temporary_root"] = True if benchmark_mode: - end_board, _ = board_metadata() + end_board, _ = ( + board_metadata() + if target_class == "sbc" + else ordinary_server_metadata() + ) if end_board: report["environment"]["power_thermal_mode_celsius_end"] = end_board.get( "power_thermal_mode_celsius" @@ -4700,6 +4913,12 @@ def _parser() -> argparse.ArgumentParser: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--binary", type=Path, required=True) parser.add_argument("--config-fixture", type=Path, default=DEFAULT_FIXTURE) + parser.add_argument( + "--target-class", + choices=("sbc", "ordinary-server"), + default="sbc", + help="Host evidence profile; SBC remains the default qualification target.", + ) parser.add_argument("--expected-sha256") parser.add_argument( "--build-elapsed-ms", @@ -4725,7 +4944,16 @@ def _parser() -> argparse.ArgumentParser: type=int, default=CONCURRENCY_WORKERS, metavar="N", - help="Concurrent finite-request workers, 1..8 (benchmark mode only).", + help="Concurrent finite-request workers, 1..16 (benchmark mode only).", + ) + parser.add_argument( + "--provider-latency-profile", + choices=("instant", "mixed-0-10-50ms"), + default="instant", + help=( + "Deterministic loopback upstream service distribution; mixed profile " + "cycles 0/0/0/10/10/50 ms." + ), ) parser.add_argument( "--concurrency-requests", @@ -4920,6 +5148,8 @@ def main(argv: Sequence[str] | None = None) -> int: competing_io_max_mib=args.competing_io_max_mib, benchmark_principals=args.benchmark_principals, benchmark_budget_mode=args.benchmark_budget_mode, + target_class=args.target_class, + provider_latency_profile=args.provider_latency_profile, ) except (OSError, QualificationError, ValueError, sqlite3.Error) as error: requested_benchmark = ( diff --git a/tests/tooling/test_qualification_ordinary_server.py b/tests/tooling/test_qualification_ordinary_server.py new file mode 100644 index 00000000..df796a77 --- /dev/null +++ b/tests/tooling/test_qualification_ordinary_server.py @@ -0,0 +1,124 @@ +"""Contract tests for the M002 ordinary-server corpus and summary guard.""" + +from __future__ import annotations + +import json +from typing import TYPE_CHECKING, Any + +import pytest + +if TYPE_CHECKING: + from pathlib import Path + +from scripts.qualification_ordinary_server import ( + PROVIDER_LATENCY_PROFILE, + REPETITIONS, + REQUESTS_PER_CELL, + SCHEMA_VERSION, + SUMMARY_VERSION, + _conditions, + _io_pressure_safe, + summarize, +) + + +def _run(principals: int, concurrency: int, p95: int) -> dict[str, Any]: + return { + "status": "pass", + "schema_version": "runtime-q008.v2", + "candidate": {"sha256": "a" * 64}, + "environment": { + "target_class": "ordinary-server", + "qualification_database_storage": { + "filesystem": "ext4", + "storage_medium_class": "non-rotational", + "device_class": "nvme", + }, + }, + "benchmark": { + "principal_count": principals, + "budget_mode": "user-global", + "provider_latency_profile": PROVIDER_LATENCY_PROFILE, + "resource_summary": {"peak_rss_bytes": 4096}, + "runs": { + "concurrent_native_responses_finite": { + "client_concurrency": concurrency, + "request_count": REQUESTS_PER_CELL, + "completed_count": REQUESTS_PER_CELL, + "failed_count": 0, + "requests_per_second": 100.0, + "cpu": {"cpu_percent": 50.0}, + "timings": { + "sample_count": REQUESTS_PER_CELL, + "p50_elapsed_ms": 10, + "p95_elapsed_ms": p95, + "p99_elapsed_ms": p95 + 1, + "maximum_elapsed_ms": p95 + 2, + }, + } + }, + }, + "durable": {"pending_requests": 0, "active_reservations": 0}, + } + + +def test_m002_matrix_separates_named_account_and_concurrency_levels() -> None: + conditions = _conditions() + assert {item["principals"] for item in conditions if item["concurrency"] == 1} == { + 1, + 5, + 10, + 25, + 49, + } + assert {item["concurrency"] for item in conditions if item["principals"] == 5} == { + 1, + 2, + 4, + 8, + 16, + } + + +def test_m002_refuses_to_start_under_high_or_unavailable_io_pressure() -> None: + assert _io_pressure_safe({"io_pressure": {"full": {"avg10": 5.0}}}) + assert not _io_pressure_safe({"io_pressure": {"full": {"avg10": 25.0}}}) + assert not _io_pressure_safe({"io_pressure": {"some": {"avg10": 1.0}}}) + + +def test_m002_raw_to_summary_requires_three_durable_p99_runs(tmp_path: Path) -> None: + names = [f"run-{index}.json" for index in range(1, REPETITIONS + 1)] + for index, name in enumerate(names, start=1): + (tmp_path / name).write_text( + json.dumps(_run(5, 4, 20 + index)), encoding="utf-8" + ) + manifest = { + "schema_version": SCHEMA_VERSION, + "status": "complete", + "plan": "on-prem-concurrency-performance M002", + "candidate": {"sha256": "a" * 64}, + "target": {"architecture": "x86_64"}, + "workload": {"requests_per_condition": REQUESTS_PER_CELL}, + "conditions": [{"principals": 5, "concurrency": 4, "run_artifacts": names}], + } + manifest_path = tmp_path / "matrix.json" + manifest_path.write_text(json.dumps(manifest), encoding="utf-8") + + summary = summarize(manifest_path) + + assert summary["schema_version"] == SUMMARY_VERSION + cell = summary["conditions"][0] + assert cell["repetitions"] == 3 + assert cell["median_p95_ms"] == 22 + assert cell["median_p99_ms"] == 23 + assert cell["settlement"].startswith("zero pending") + + +def test_m002_raw_to_summary_rejects_incomplete_matrix(tmp_path: Path) -> None: + path = tmp_path / "matrix.json" + path.write_text( + json.dumps({"schema_version": SCHEMA_VERSION, "status": "incomplete"}), + encoding="utf-8", + ) + with pytest.raises(ValueError, match="incomplete"): + summarize(path) diff --git a/tests/tooling/test_qualification_sbc.py b/tests/tooling/test_qualification_sbc.py index 6b058e66..9af6c3fa 100644 --- a/tests/tooling/test_qualification_sbc.py +++ b/tests/tooling/test_qualification_sbc.py @@ -14,6 +14,7 @@ from scripts.qualification_sbc import ( BENCHMARK_FIXTURE, BENCHMARK_MAX_SAMPLES, + CONCURRENCY_WORKERS_MAX, DEFAULT_FIXTURE, DIAGNOSTIC_FIXTURE_PATH_SUFFIX, DIAGNOSTIC_MAX_SAMPLES, @@ -69,6 +70,7 @@ benchmark_cadence_facts, bounded, main, + ordinary_server_metadata, resource_sample, run_qualification, ) @@ -126,6 +128,29 @@ def test_q008_storage_metadata_uses_root_device_without_identity() -> None: assert _storage_device_class("nvme0n1") == "nvme" +def test_ordinary_server_profile_records_host_and_durable_storage_facts() -> None: + report, reason = ordinary_server_metadata() + if platform.system().lower() != "linux": + assert report is None + assert reason + return + assert reason is None + assert report is not None + assert report["target_class"] == "ordinary-server" + assert "virtualization_type" in report + assert "container_type" in report + assert "root_filesystem" in report + assert "root_storage_device_class" in report + + +def test_ordinary_benchmark_supports_sixteen_clients_and_fixed_latency_profile() -> ( + None +): + assert CONCURRENCY_WORKERS_MAX == 16 + provider = LoopbackProvider("mixed-0-10-50ms") + assert [provider.record("/responses") for _ in range(6)] == [0, 0, 0, 10, 10, 50] + + def test_q008_missing_candidate_is_blocked_before_mutation_on_linux_sbc( tmp_path: Path, ) -> None: From af735f1bf4646d16e4a9d079cf1d75ef27ca2e67 Mon Sep 17 00:00:00 2001 From: wr3n-ai Date: Thu, 15 Oct 2026 12:12:18 +0000 Subject: [PATCH 20/62] Close M002 qualification tooling conditionally --- docs/deployment.md | 4 +- docs/raspberry-pi.md | 2 + .../002-status.md | 120 ++++++++++++++++++ .../002-ordinary-server-team-baseline.md | 2 +- plans/registry.md | 15 ++- ...on-prem-concurrency-performance-roadmap.md | 21 +-- 6 files changed, 150 insertions(+), 14 deletions(-) create mode 100644 plans/closure/on-prem-concurrency-performance/002-status.md diff --git a/docs/deployment.md b/docs/deployment.md index 1e4afd4c..7c589eb1 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -487,7 +487,9 @@ available, use it only for relative sizing; qualify the actual provider mix, model payloads, login rate, and policy/budget profile before deployment. No universal throughput, concurrency, or latency SLA follows from one host and synthetic workload. The Pi microSD envelope remains a separate solo-device -reference in [`raspberry-pi.md`](raspberry-pi.md). +reference in [`raspberry-pi.md`](raspberry-pi.md). See the +[M002 closure record](../plans/closure/on-prem-concurrency-performance/002-status.md) +for its measured/unmeasured status and required follow-up evidence. To repeat the frozen corpus from a release binary, run `uv run python scripts/qualification_ordinary_server.py --binary `. diff --git a/docs/raspberry-pi.md b/docs/raspberry-pi.md index 3ebab53d..5b6bb525 100644 --- a/docs/raspberry-pi.md +++ b/docs/raspberry-pi.md @@ -274,6 +274,8 @@ saturates. The ordinary-server team baseline is maintained separately from this Pi microSD solo envelope; do not use the Pi limits as the team-host sizing source. +The current M002 result is conditional and contains no server capacity claim; +see its [closure record](../plans/closure/on-prem-concurrency-performance/002-status.md). ## Verify from LAN diff --git a/plans/closure/on-prem-concurrency-performance/002-status.md b/plans/closure/on-prem-concurrency-performance/002-status.md new file mode 100644 index 00000000..ef0eff5a --- /dev/null +++ b/plans/closure/on-prem-concurrency-performance/002-status.md @@ -0,0 +1,120 @@ +# On-Prem Concurrency Performance M002 — Closure Status + +Status: conditionally closed — ordinary-server tooling and workload design are implemented; the isolated host run and capacity envelope remain unmeasured + +Source implementation plan: + +- `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` + +Source subsystem roadmap: + +- `plans/subsystems/on-prem-concurrency-performance-roadmap.md#milestone-002--ordinary-server-small-team-baseline` + +Repository baseline reviewed: `cdfb6fa8d4aaa1d9d5b43af8c87eaac68f83c402` + +Implementation commits or pull requests: + +- `cdfb6fa8d4aaa1d9d5b43af8c87eaac68f83c402` — ordinary Linux host attestation, safe SSD/NVMe gates, deterministic upstream latency profile, bounded host I/O observations, 16-client support, frozen 27-run driver, summary validation, and deployment/observability guidance. No runtime change. + +## 1. Executive finding + +The tooling/setup slice is complete. `scripts/qualification_ordinary_server.py` freezes the manifest before collection, schedules nine principal/concurrency conditions with three fresh-root runs each, and derives an aggregate report only after all raw runs pass the same build, storage, p99-sample, and settlement gates. `scripts/qualification_sbc.py` retains the SBC default and adds ordinary-host metadata, persistent database-storage checks, the 16-client tier, a deterministic 0/10/50 ms upstream profile, and host I/O observations. + +No capacity run was started and no release candidate was built. The available host is a bare-metal Ubuntu 24.04.5 x86_64 workstation with durable ext4/NVMe, but it is shared with long-running unrelated CPU work and is not an explicitly isolated server qualification target. During the required Rust suite build, Linux I/O pressure exceeded the predeclared ordinary-runner stop threshold of 25% full `avg10` by a wide margin. Both suite attempts were stopped. No timing or throughput result is inferred from the host or from M001's separate exploratory NVMe cell. The team capacity envelope, SQLite gate tails, active-load cancellation/restart behavior, and policy-denial rates remain unmeasured. + +Disposition is conditional under the plan's stop condition. Do not claim an ordinary-server throughput, concurrency, or latency envelope from this closure. + +## 2. Requirement-to-evidence matrix + +| Requirement | Evidence | Result | Notes | +|---|---|---|---| +| Freeze named-account and concurrent-request levels separately | `scripts/qualification_ordinary_server.py`, `tests/tooling/test_qualification_ordinary_server.py` | pass | Five account levels (1/5/10/25/49 at concurrency 1) and five concurrency levels (1/2/4/8/16 at five principals) form nine unique conditions and 27 planned runs. | +| Freeze workload, candidate identity, and gates before collection | M002 driver and summary contract tests | pass | The driver writes its matrix before invoking any condition; each condition is 1,000 finite requests, three repetitions, hard user/global budgets, and the fixed synthetic 0/10/50 ms upstream profile. | +| Record host/build/storage facts and require durable SSD/NVMe | `ordinary_server_metadata`, Q008 report schema, durable-path guard | partial | Host facts are implemented. The host was attested as Ubuntu 24.04.5, Linux 6.8.0-147-generic, Intel Core i9-9900K, 16 logical CPUs, 15 GiB RAM, Gigabyte Z390 AORUS MASTER, ext4 on Samsung SSD 980 1TB NVMe, bare metal (`systemd-detect-virt: none`, container: none). No release candidate was built and no raw run was produced. | +| Run three fresh-root durable-server repetitions per condition and collect latency/resource metrics | Frozen driver, full artifacts absent | not run | No ordinary-server experiment ran. CPU, RSS, throughput, TTFT, I/O, WAL, gate, and per-condition latency values are not reported. | +| Require adequate p99 observations and converged durable state | Q008 runner plus summary guard tests | pass — tooling contract only | The driver requires 1,000 completed observations, a p99 value, zero request failures, zero pending requests, and zero active reservations in all three raw runs before summarizing. No observed run passed or failed these gates because no run started. | +| Exercise failure, cancellation, active-user restart, mixed policy denies, and token revocation | Existing Q008 functional lifecycle coverage; no M002 raw run | not run | The new matrix measures successful budgeted inference. It does not yet drive cancellation/restart during active load or count provider, identity, and budget 429s separately. No behavior is claimed. | +| Publish an honest ordinary-server deployment envelope and compare with Pi solo evidence | `docs/deployment.md`, `docs/raspberry-pi.md`, this record | partial | The docs separate named users from simultaneous requests and state that there is no M002 capacity result. The immutable Pi M001/M012 evidence is unchanged. No sizing range is published. | +| Keep runtime, persistence, and user-policy invariants unchanged | Changed-file review | pass | No Rust, schema, admission, quota, auth, provider transport, SQLite journal, worker, or checkpoint policy changes. | + +## 3. Production implementation evidence + +No production behavior changed. M002 tooling adds ordinary-host inventory without exposing mount source paths or process command lines; root/database storage class and filesystem are bounded facts. The benchmark runner rejects tmpfs/ramfs/overlay and rotational storage, and standalone ordinary-server benchmark invocations enforce the same preflight I/O-pressure gate as the matrix driver. The deterministic latency profile cycles through 0, 0, 0, 10, 10, and 50 ms per synthetic upstream request. All provider traffic remains loopback-only. + +The frozen matrix contains 27 runs across nine unique cells. It uses a 1,000-request concurrent batch, three repetitions, 30 sequential samples per surface, native finite, native streaming, and translated streaming cases, synthetic managed principals, and hard user/global request budgets. No raw M002 report, summary, or capacity number exists because preflight did not proceed to collection. + +## 4. Verification executed + +### Commands run + +```bash +rtk cargo fmt --manifest-path rust/Cargo.toml --all -- --check +rtk cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings +rtk cargo check --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features +rtk cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- -D warnings +rtk cargo test --manifest-path rust/Cargo.toml --workspace --all-targets -- --test-threads=1 +rtk proxy bash -c 'CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --workspace --all-targets -- --test-threads=1' +rtk uv sync --frozen +rtk uv run ruff format --check scripts/ tests/tooling/ +rtk uv run ruff check scripts/ tests/tooling/ +rtk uv run pyright scripts/ +rtk uv run pytest tests/tooling/test_qualification_sbc.py tests/tooling/test_on_prem_mmc_tail_attribution.py tests/tooling/test_qualification_ordinary_server.py -q +rtk uv run python scripts/validate_release_docs.py +rtk git diff --check +``` + +### Results + +| Command | Result | +|---|---| +| Rust fmt, default Clippy, no-default check, and no-default Clippy | pass | +| Workspace serial Rust tests, default parallel build | interrupted before test results; observed I/O PSI reached `some avg10=94.05%`, `full avg10=81.43%`, `full avg60=46.12%` during compile/link | +| Workspace serial Rust tests with one Cargo build job | interrupted before test results; observed I/O PSI reached `full avg10=43.31%`, `full avg60=42.09%` during compile/link | +| `uv sync --frozen` | pass; generated a local `requires-python` lockfile normalization, which was reverted because this task changes no Python dependencies | +| Ruff format/check and Pyright | pass; 65 files formatted, no Ruff findings, Pyright 0 errors | +| Focused Q008, M001 artifact, and M002 tooling tests | pass; 51 tests | +| Release-doc validator | pass; seven docs checked | +| Release build and ordinary-server matrix | not run; shared-host I/O pressure and lack of an isolated, provisioned server target triggered the plan stop condition | +| Hosted CI | not run from this local closure pass | + +## 5. Invariant review + +- The runner uses a disposable root, loopback-only provider, and synthetic identities. No customer data, real provider key, prompt, or production database is used. +- Named principal count is reported independently from in-flight request concurrency. The plan makes no 49- or 50-concurrent service claim. +- The qualification database stays on durable non-rotational SSD/NVMe for ordinary-server measurements; tmpfs is rejected. +- No database topology, WAL/NORMAL, one-worker, routing, quota, admission, or budget semantics changed. +- Existing M001 and M012 physical artifacts and Pi guidance remain intact. + +## 6. Failure and recovery review + +The Q008 runner still checks isolated backup/recovery and graceful stop paths, and the matrix summary requires all requests to finish with no pending rows or active reservations. M002 did not exercise server shutdown or cancellation during active inference, retries/partial upstream failure, identity disable/revocation during load, or the separate observed 429 categories. These remain unverified and must be included before using the matrix as team-operation evidence. No active reservation or failure count is fabricated. + +The environment stop was safe: the Rust test build processes were interrupted, no qualification process or synthetic request batch was started, and no performance artifact was emitted. + +## 7. Migration and compatibility review + +No schema, migration, configuration, endpoint, CLI, or runtime compatibility change was made. The SBC runner's default target and schema remain unchanged; ordinary-server qualification is an explicit target profile. + +## 8. Security review + +The tools retain bounded aggregate host, timing, and storage-class facts. They do not serialize request bodies, hostnames, addresses, credentials, tokens, usernames, or environment values. Existing redaction guards and the new raw-to-summary tests pass. There are no M002 raw artifacts to audit. + +## 9. Documentation and operations + +`docs/deployment.md` documents the frozen workload, repeat command, safe storage requirements, preflight pressure stop, and no-SLA boundary. `docs/raspberry-pi.md` keeps the SBC solo envelope separate. `architecture/deep-dive-observability.md` distinguishes qualification artifacts from runtime telemetry. The release-documentation validator passes. + +## 10. Unresolved findings + +| Severity | Finding | Impact | Required action | +|---|---|---|---| +| medium | No isolated ordinary-server run, release candidate identity, or capacity observations were collected | There is no ordinary-server team sizing envelope; downstream work cannot cite M002 as measured capacity | On an explicitly provisioned, isolated Linux host, build the locked release candidate, retain its exact SHA and host/storage attestation, pass the 25% full-I/O-PSI preflight, and complete all 27 fresh-root runs. Preserve all failures and calculate the report from raw reports. | +| medium | Active-load cancellation/restart, retries/partial provider failures, token revocation, policy denies, and classified 429s are absent from the matrix execution | Budget recovery and failure behavior under simultaneous team load remain unknown | Extend the bounded synthetic scenarios and their sanitized guard before using M002 as operational readiness evidence; exercise and report provider, identity, and budget responses separately. | +| low | The full Rust workspace suite did not complete because both compile attempts crossed the I/O stop gate | This tooling-only change has focused Python coverage and compile/lint gates, but no serial workspace test result | Re-run the exact serial workspace suite on a stable, low-pressure host. | + +## 11. Roadmap disposition + +M002 is **conditionally closed for its tooling/setup slice only**. The ordinary-server measurements and deployment envelope are explicitly unmeasured. A future isolated hardware pass must provide the raw corpus and failure/recovery evidence above before any capacity recommendation is made. The Pi microSD solo benchmark remains a distinct reference and does not satisfy or block M002. + +## 12. Registry updates + +The implementation plan moves to `implemented` with this conditional closure; the roadmap remains `active` because no future production optimization is registered and M002's operational evidence is outstanding. The dependency-ready M002 row is removed and the conditional closure is recorded. Unblock audit: searched the on-prem implementation plans, both relevant roadmap dependency sections, and registry ready/active/blocked tables. No future plan declares M002 as a hard, interface, or soft dependency. On-prem user-access M023 names ordinary-server evidence as an operational requirement but remains blocked on its hard M022 dependency; M002 has not supplied that operational evidence, so M023 is not promoted and no other plan is unblocked. diff --git a/plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md b/plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md index 372b35fc..8bebd8b3 100644 --- a/plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md +++ b/plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md @@ -1,6 +1,6 @@ # On-Prem Concurrency Performance M002 — Ordinary-server small-team capacity baseline -Status: active +Status: implemented Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index fcd3acad..30c23e4f 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -59,7 +59,7 @@ No new runtime, schema or performance claim lands with these plans. | Deployment and packaging | closed | `plans/subsystems/deployment-packaging-roadmap.md` | M005 closed — active PATH integration detection corrective | M001–M005 closed; no successor registered. | | Dashboard | closed | `plans/subsystems/dashboard-roadmap.md` | M012 closed — lifecycle closure and documentation polish | M001–M012 closed; no registered successor; future dashboard work requires a new bounded plan. | | On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M017/M018/M019 ready — post-merge operability and browser transport | PR #7 merged as `88f58bbf` on 2026-10-15; M020–M023 have explicit hard dependencies. Keep M001–M016 closures immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | -| On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 ready — ordinary-server small-team synthetic capacity baseline | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 is independent measurement on ordinary on-prem Linux with durable SSD/NVMe (operational target required), not an authorization to change WAL/SQLite topology. | +| On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 conditionally closed — qualification tooling implemented; isolated ordinary-server baseline still unmeasured | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 authorizes no WAL/SQLite topology change. | ## Dependency-ready implementation plans @@ -74,7 +74,6 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| -| On-prem concurrency performance | M002 — ordinary-server team baseline | active | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | Host qualification and frozen synthetic matrix in progress; no runtime or SQLite-policy change. | ## Blocked work @@ -120,6 +119,7 @@ Historical M007 blocker assessment at baseline `7e241ad` (`plans/closure/persist | On-prem user access M015 — independent authentication/authorization threat review | closed — independent review findings on hard-cost proof bounds and browser-session access corrected and re-reviewed; default 1,024 / no-default 1,025 tests pass (one ignored each); exact-SHA CI, installer and dependency audit pass on `d8fd9c2a`; no M015 security findings remain; M016 promoted to ready | `plans/closure/on-prem-user-access/015-status.md`, implementation `a526980c`, `deb1622f`, `d8fd9c2a`, security scan `52bd79cc`, CI `38720507683`, installer `38720509034`, audit `38720507720` | | On-prem user access M014 — Rustls dependency security-advisory corrective | closed — lockfile-only Rustls 0.23.46 patch, unchanged transport feature graph; local default/no-default and 1,009/1,010-test workspace suites, provider TLS (trusted root, wrong-root and hostname rejection), budget/coordinator suites, release build and tooling pass; hosted cargo-deny, CI and both installers pass on `7462263ec482c5fda4bd1a70a9a55b0efc313cd3`; main remains pending PR merge | `plans/closure/on-prem-user-access/014-status.md`, implementation `7462263e`, audit `38671345587`, CI `38671345679`, installers `38671345597` | | On-prem concurrency performance M001 — Pi 5/microSD concurrent tail attribution | conditionally closed — three paired physical microSD/tmpfs runs identify durable publication/SQLite/storage as the dominant tested four-way tail contributor; no production change or capacity claim; SSD/NVMe is not required | `plans/closure/on-prem-concurrency-performance/001-status.md`, six Q008 reports + manifest under `artifacts/qualification/on-prem-concurrency-performance/`, artifact guard `tests/tooling/test_on_prem_mmc_tail_attribution.py` | +| On-prem concurrency performance M002 — ordinary-server team baseline | conditionally closed — ordinary-host attestation, safe frozen 27-run driver, raw-to-summary guard, and deployment guidance implemented; no release build or measurement because the shared host crossed the I/O-pressure stop gate; no capacity or SLA claim | `plans/closure/on-prem-concurrency-performance/002-status.md`, implementation `cdfb6fa8`, tests `tests/tooling/test_qualification_ordinary_server.py` | | On-prem user access M012 — physical SBC small-team qualification and final disposition | closed — Pi 5 managed envelope measured (functional 25 cells + 3×30 benchmark batches all pass, 145/145 each, 0 pending/active; Argon2id 51 ms 1-way / 71–73 ms 2-way; 49 named users at 51 ms p50 with spot logins passing; concurrency 1/2/4/8 completes with 8-way saturating at p50 4.1 s); runner managed-auth corrective + 2 contract tests; solo practical, 49 users inexpensive metadata, no 50-concurrent claim; one medium (MMC tails exceed Sep-21 gates, envelope restricted) + six low findings, no high | `plans/closure/on-prem-user-access/012-status.md`, artifacts `artifacts/qualification/on-prem-m012-pi5-functional.json` + `on-prem-m012-pi5-benchmark-run-{1,2,3}.json`, docs `docs/raspberry-pi.md` envelope | | On-prem user access M011 — admin/member dashboard browser qualification | closed — frozen Python oracle rebuilt and run against head with 54 parity gaps, identical to the pre-M011 baseline (0 new, 0 resolved); on-prem admin/member pages qualified against a Rust-owned approved reference (70 observations, 22 captures) covering both roles, the negative matrix, no-JS sign-in, HiDPI/mobile/themed viewports, and a DOM credential-leak guard; browser sign-in, top-nav overflow at 1280px, unstyled usage pages, missing viewport declarations, unbounded tables, and budget labelling all corrected; 967 serial Rust tests and 175 tooling tests pass at the M011 baseline (re-verified post-integration — see integration note) | `plans/closure/on-prem-user-access/011-status.md` | | On-prem user access M010 — budget crash recovery and schema-54 restore | closed @@ -527,3 +527,14 @@ unblocked. On-prem user-access M016 is ready after M014/M015 closure and still requires final-SHA gates; this optional diagnostic workstream has no edge to that sequence. The concurrency-performance roadmap remains active for any future separately scoped controlled-I/O/media/budget work; none is registered. + +Unblock audit (on-prem concurrency performance M002 conditionally closed, +`plans/closure/on-prem-concurrency-performance/002-status.md`): searched +`plans/implementation/`, the on-prem concurrency and user-access roadmap +dependency sections, and the dependency-ready, active, and blocked registry +tables. No future plan declares M002 as a hard, interface, or soft dependency. +On-prem user-access M023 names ordinary-server evidence as an operational +requirement, but the evidence remains unmeasured and M023 remains blocked on +its hard M022 dependency. No plan was promoted or unblocked. A future isolated +server run remains required before M023 can use M002 as its operational +capacity evidence. diff --git a/plans/subsystems/on-prem-concurrency-performance-roadmap.md b/plans/subsystems/on-prem-concurrency-performance-roadmap.md index 9a6eea84..a88b1a62 100644 --- a/plans/subsystems/on-prem-concurrency-performance-roadmap.md +++ b/plans/subsystems/on-prem-concurrency-performance-roadmap.md @@ -138,16 +138,17 @@ claims. Source: ### M002 — Ordinary-server small-team baseline -Class: polish. Ready independently of the user-access correctives. -Run a repeatable, redacted, synthetic qualification corpus on a documented -ordinary Linux server with durable SSD/NVMe and 1/5/10/25/49 named -principal records, separately from measured concurrent levels and workload. -Report CPU/RSS, throughput, local overhead, streaming TTFT, SQLite -gate/storage tails, budget recovery and admitted/denied requests. Three -paired runs and adequate sample size are required for percentile claims. -SBC microSD remains the solo low-end performance/durability reference; -it is **not** a gate for team capacity or server release. +Class: polish. Conditionally closed for the tooling/setup slice. The physical +measurement and operating envelope remain unmeasured because the available +host was shared and crossed the frozen I/O-pressure stop gate during required +build verification. A future isolated run must use durable SSD/NVMe and cover +1/5/10/25/49 named principal records separately from 1/2/4/8/16 concurrent +requests, with three fresh-root repetitions per condition, adequate p99 sample +counts, active-load recovery evidence, and sanitized summary output. SBC +microSD remains the solo low-end performance/durability reference; it is +**not** a gate for team capacity or server release. Source: `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md`. +Closure: `plans/closure/on-prem-concurrency-performance/002-status.md`. Future **production** optimization milestone: not registered. Requires a separate scope and measurable regression guard; no journal, checkpoint, @@ -211,4 +212,4 @@ the Pi/MMC solo latency tail. No optimization promise or new | Milestone | Status | Implementation plan | Closure record | Blockers | |---|---|---|---|---| | M001 | conditionally closed | `plans/implementation/on-prem-concurrency-performance/001-pi5-mmc-tail-latency-attribution.md` | `plans/closure/on-prem-concurrency-performance/001-status.md` | Four-way microSD/MMC tail attributed to durable storage; controlled competing-I/O and broader account/budget matrix remain conditions. SSD/NVMe is optional and not a closure dependency. No dependency edge to post-merge on-prem operability plans. | -| M002 | ready | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | — | Ordinary Linux server with durable SSD/NVMe and repeated synthetic multi-principal workload is an operational evidence requirement, not a Pi/microSD gate. | +| M002 | conditionally closed | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | `plans/closure/on-prem-concurrency-performance/002-status.md` | Tooling/setup complete; ordinary-server performance, active-load recovery, and capacity envelope remain unmeasured pending an isolated target below the frozen I/O-pressure stop gate. | From c5167aa64ae5a3f49ffda5f86010eeacb3a63929 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 12:23:08 +0000 Subject: [PATCH 21/62] Retire shared key CLI after bootstrap --- rust/src/runtime.rs | 33 +++++++++++++++++++++++++++++++-- 1 file changed, 31 insertions(+), 2 deletions(-) diff --git a/rust/src/runtime.rs b/rust/src/runtime.rs index 30665aa2..cc976782 100644 --- a/rust/src/runtime.rs +++ b/rust/src/runtime.rs @@ -71,7 +71,7 @@ pub async fn run(cli: Cli) -> Result<(), BootstrapError> { Some(Command::Connect(args)) => connect(&config_path, args).await?, Some(Command::Logout { target }) => logout(&config_path, target.as_deref()).await?, Some(Command::Edit) => edit(&config_path)?, - Some(Command::Getkey) => getkey(&config_path)?, + Some(Command::Getkey) => getkey(&config_path).await?, Some(Command::Newkey(args)) => { newkey(&config_path, args.show_old, args.show_secrets).await? } @@ -1933,6 +1933,21 @@ async fn integration_personal_credential( Ok(Some(token)) } +async fn identity_is_initialized(config: &config::Config) -> Result { + let database_path = config::Config::runtime_path(&config.database.path); + if !database_path.is_file() { + return Ok(false); + } + let database = open_operational_database(config, true).await?; + let state = crate::identity::access_state(&database) + .await + .map_err(|error| command_error(EXIT_VALIDATION, error.to_string())); + let close_result = database.close().await; + let state = state?; + close_result.map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))?; + Ok(matches!(state.as_str(), "admin_initialized" | "managed")) +} + async fn run_managed_configsetup( path: &Path, pre_mutation_config: &config::Config, @@ -2337,7 +2352,14 @@ fn which(name: &str) -> Option { .find(|candidate| candidate.is_file()) } -fn getkey(path: &Path) -> Result<(), BootstrapError> { +async fn getkey(path: &Path) -> Result<(), BootstrapError> { + let config = config::Config::from_toml(path)?; + if identity_is_initialized(&config).await? { + return Err(command_error( + EXIT_VALIDATION, + "the shared server key is retired after administrator initialization; issue a personal credential with `eggpool users token USER`", + )); + } let key = config_mutation::read_server_key(path) .map_err(|error| mutation_error(error, EXIT_VALIDATION))?; let Some(key) = key else { @@ -2354,6 +2376,13 @@ fn getkey(path: &Path) -> Result<(), BootstrapError> { } async fn newkey(path: &Path, show_old: bool, show_secrets: bool) -> Result<(), BootstrapError> { + let config = config::Config::from_toml(path)?; + if identity_is_initialized(&config).await? { + return Err(command_error( + EXIT_VALIDATION, + "the shared server key is retired after administrator initialization; use `eggpool users token USER` for managed access", + )); + } let old = config_mutation::read_server_key(path) .map_err(|error| mutation_error(error, EXIT_VALIDATION))?; let key = From 5ab758698da31b35d68e6bbf2c1d98a871c1a0d7 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 12:24:30 +0000 Subject: [PATCH 22/62] Close on-prem access plan 017 --- .../closure/on-prem-user-access/017-status.md | 145 ++++++++++++++++++ ...anaged-client-credential-reconciliation.md | 2 +- .../022-hard-budget-agent-compatibility.md | 2 +- plans/registry.md | 8 +- .../subsystems/on-prem-user-access-roadmap.md | 11 +- 5 files changed, 158 insertions(+), 10 deletions(-) create mode 100644 plans/closure/on-prem-user-access/017-status.md diff --git a/plans/closure/on-prem-user-access/017-status.md b/plans/closure/on-prem-user-access/017-status.md new file mode 100644 index 00000000..9d93a6a7 --- /dev/null +++ b/plans/closure/on-prem-user-access/017-status.md @@ -0,0 +1,145 @@ +# On-Prem User Access M017 — Closure Status + +Status: closed + +Source implementation plan: + +- `plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md` + +Source subsystem roadmap: + +- `plans/subsystems/on-prem-user-access-roadmap.md#m017--managed-client-credential-reconciliation` + +Repository baseline reviewed: `88f58bbf2e9095cf46437945edf83e580270bdb6` + +Implementation commits: + +- `d726bce` — managed local integration setup requires an active personal credential; keyless remote export and obsolete credential guidance reconciled. +- `218df45` — explicit-credential and keyless-export regression coverage; remote instructions name personal tokens. +- `c5167aa` — `getkey`/`newkey` reject retired shared-key use after administrator initialization. + +## 1. Executive finding + +Managed local client setup no longer resolves or creates the retired shared +server key. It requires `EGGPOOL_API_KEY` to contain an active personal token +from the local credential store. `getkey` and `newkey` clearly refuse after +identity initialization. Remote configuration remains a read-only, +credential-free profile export and works without a configured server key. + +The cross-component behavior is covered with a synthetic provider and managed +personal credentials. No real provider keys, user prompts, or third-party agent +runtime were used. + +## 2. Requirement-to-evidence matrix + +| Requirement | Evidence | Result | Notes | +|---|---|---|---| +| Legacy, fresh, initialized, and managed credential behavior stays separated | `rust/src/runtime.rs::integration_personal_credential`, `identity_is_initialized`, `getkey`, `newkey`; `rust/tests/on_prem_identity.rs`; `rust/tests/operations_o005.rs` | pass | Initialized installs verify the supplied token before rendering; legacy behavior remains before initialization. | +| Managed local setup does not resurrect a shared key | `explicit_personal_credential_does_not_resurrect_shared_server_key`; runtime managed credential resolution | pass | Config remains keyless and unchanged when an explicit personal credential is supplied. | +| Remote setup does not require or create a global key | `remote_context_is_read_only_for_keyless_managed_config`; `configremote_is_read_only_deterministic_and_secret_free` | pass | Config remains byte-identical; absent catalog DB is not created. | +| Profile/token output contains no credential | `operations_o005` profile assertions and `eggpool-connect` contract suite | pass | `epc1` carries endpoint/profile metadata and `EGGPOOL_API_KEY` remains a receiving-machine environment reference. | +| Inference, discovery, policy filtering, and revocation use personal identity | `rust/tests/on_prem_attribution.rs`, `rust/tests/on_prem_sessions.rs`, `rust/tests/on_prem_model_policy.rs` | pass | Synthetic local provider; unauthorized and revoked credentials are denied. | +| Documentation no longer directs managed users to `getkey` | `tests/tooling/test_on_prem_credential_guidance.py`; release-doc validator | pass | `getkey` is documented as legacy-only. | + +## 3. Production implementation evidence + +- `rust/src/operations/integrations.rs` accepts an explicit credential for + local rendering and omits the global-key prerequisite from remote export. +- `rust/src/runtime.rs` detects initialized identity state, verifies the + personal token before local setup, and refuses legacy key retrieval/rotation + after bootstrap. +- README and integration guidance distinguish member tokens from the retired + shared key. +- No database migration, credential plaintext persistence, or profile schema + change was introduced. + +## 4. Verification executed + +### Commands run + +```bash +cargo fmt --manifest-path rust/Cargo.toml --all -- --check +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings +cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- -D warnings +cargo test --manifest-path rust/Cargo.toml --workspace --all-targets -- --test-threads=1 +cargo test --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- --test-threads=1 +cargo test --manifest-path rust/Cargo.toml -p eggpool-connect -- --test-threads=1 +cargo test --manifest-path rust/crates/eggpool-client-config/Cargo.toml +cargo test --manifest-path rust/Cargo.toml --test operations_o005 -- --test-threads=1 +cargo test --manifest-path rust/Cargo.toml --test on_prem_identity -- --test-threads=1 +cargo test --manifest-path rust/Cargo.toml --test on_prem_sessions -- --test-threads=1 +cargo test --manifest-path rust/Cargo.toml --test on_prem_model_policy -- --test-threads=1 +cargo test --manifest-path rust/Cargo.toml --test cli_contract -- --test-threads=1 +uv run pytest tests/tooling/test_on_prem_credential_guidance.py -q --tb=short +uv run python scripts/validate_release_docs.py +git diff --check +``` + +### Results + +- Default local workspace: 1,028 passed, 1 ignored across 79 suites. +- No-default local workspace: 1,029 passed, 1 ignored across 79 suites. +- `eggpool-connect`: 59 passed; portable client-config: 49 passed. +- Focused post-implementation CLI, operations, identity, session, and model + policy suites passed (6, 15, 3, 10, and 4 tests respectively); runtime unit + target passed (4 tests). +- Strict Clippy passed with default and no-default features. Formatting, + tooling regression guard, release-doc validation, and diff checks passed. +- These are local results. Hosted CI and release installers were not run in + this closure pass. + +## 5. Invariant review + +- Provider secrets remain in provider configuration/environment; member + credentials remain one-way digests in SQLite. +- Managed authentication uses the typed personal principal and does not fall + back to a stale shared server key. +- Remote profiles remain secret-free and contain no user token. +- Existing legacy authentication remains available only before identity + initialization; no new shared-key access path was introduced. + +## 6. Failure and recovery review + +Missing, empty, expired, revoked, or non-personal credentials fail before +client configuration is rendered. Remote export is read-only and tolerates an +absent catalog file. Existing personal-token revocation tests confirm that +subsequent authenticated requests fail. No new asynchronous worker or durable +state was added. + +## 7. Migration and compatibility review + +No migration or profile protocol change. `EGGPOOL_API_KEY` remains the client +environment variable. Existing pre-bootstrap `getkey`/`newkey` behavior is +preserved; initialized installations receive an actionable personal-token +message instead of a dead shared key. + +## 8. Security review + +Personal credentials are verified against the selected installation before +local setup. No token is logged or placed in the remote profile. The generated +Codex/OpenCode profile remains environment-variable based; secret-bearing +targets retain their existing protected delivery behavior. + +## 9. Documentation and operations + +Updated `README.md` and `architecture/deep-dive-integrations.md`; added a +tooling guard against the obsolete post-bootstrap `getkey` example. The +release-document validator passed. + +## 10. Unresolved findings + +| Severity | Finding | Impact | Required action | +|---|---|---|---| +| none | No unresolved M017 implementation finding | — | — | + +## 11. Roadmap disposition + +Milestone closed. Hard-budget compatibility M022 may proceed. M020 remains +blocked on M019; M021 remains blocked on M020; M023 remains blocked on M022 +and ordinary-server operational evidence. + +## 12. Registry updates + +`plans/registry.md` records M017 closed, promotes M022 to ready, and includes +the dependency audit. The roadmap marks M017 closed and M022 ready. Historical +M001–M016 closure records were not changed. diff --git a/plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md b/plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md index 25f89d2a..aabb601d 100644 --- a/plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md +++ b/plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md @@ -1,6 +1,6 @@ # On-Prem User Access M017 — Managed client credentials and integration reconciliation -Status: closing +Status: implemented Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, PR #7 merged). diff --git a/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md b/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md index 97026266..a4ec58c7 100644 --- a/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md +++ b/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md @@ -1,6 +1,6 @@ # On-Prem User Access M022 — Hard-budget interoperability with coding agents -Status: blocked +Status: ready Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index 3670cbe8..b4c742dc 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -58,7 +58,7 @@ No new runtime, schema or performance claim lands with these plans. | Persistence | active | `plans/subsystems/persistence-roadmap.md` | M011 ready — post-M010 roadmap/documentation reconciliation | M009/M010 closures accepted. Documentation/tooling-only corrective; production remains WAL/NORMAL. No technical persistence successor is registered. | | Deployment and packaging | closed | `plans/subsystems/deployment-packaging-roadmap.md` | M005 closed — active PATH integration detection corrective | M001–M005 closed; no successor registered. | | Dashboard | closed | `plans/subsystems/dashboard-roadmap.md` | M012 closed — lifecycle closure and documentation polish | M001–M012 closed; no registered successor; future dashboard work requires a new bounded plan. | -| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M017/M018/M019 ready — post-merge operability and browser transport | PR #7 merged as `88f58bbf` on 2026-10-15; M020–M023 have explicit hard dependencies. Keep M001–M016 closures immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | +| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M018/M019/M022 ready — local operator status, login transport, and hard-budget compatibility | M017 closed; M020/M021/M023 retain their hard dependencies. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | | On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 ready — ordinary-server small-team synthetic capacity baseline | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 is independent measurement on ordinary on-prem Linux with durable SSD/NVMe (operational target required), not an authorization to change WAL/SQLite topology. | ## Dependency-ready implementation plans @@ -66,6 +66,7 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Dependencies / handoff note | |---|---|---|---|---| | Persistence | M011 — post-M010 roadmap and documentation reconciliation | ready | `plans/implementation/persistence/011-post-m010-roadmap-documentation-reconciliation.md` | Hard: M009/M010 closures accepted. Documentation/tooling only; no runtime or physical target dependency. | +| On-prem user access | M022 — hard-budget agent interoperability | ready | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | Hard M017 is closed; preserve M008/M015 proof-or-reject guarantees. | | On-prem user access | M018 — local operator status corrective | ready | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | Independent P1; preserve privileged HTTP fences, inspect local OS control authority. | | On-prem user access | M019 — browser transport and solo LAN corrective | ready | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | Independent P1; remote HTTP GET login form must not solicit credentials. | | On-prem concurrency performance | M002 — ordinary-server team baseline | ready | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | Independent evidence work; ordinary Linux + durable SSD/NVMe required for team-capacity claims, no Pi/microSD release gate. | @@ -74,7 +75,6 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| -| On-prem user access | M017 — managed client credential reconciliation | closing | `plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md` | Focused integration/auth/docs evidence gathered; closure record and dependency audit in progress. | ## Blocked work @@ -83,7 +83,6 @@ No new runtime, schema or performance claim lands with these plans. |---|---|---| | On-prem user access | M020 — managed transition and headless administration | Hard: M017 and M019 closure. | | On-prem user access | M021 — default member profile and provisioning | Hard: M020 closure. | -| On-prem user access | M022 — hard-budget agent compatibility | Hard: M017 closure; never relax M008/M015 proof. | | On-prem user access | M023 — short-window team fairness | Hard: M022 closure; ordinary-server evidence operational. | @@ -116,6 +115,7 @@ Historical M007 blocker assessment at baseline `7e241ad` (`plans/closure/persist | Subsystem / plan | Disposition | Evidence | |---|---|---| +| On-prem user access M017 — managed client credential reconciliation | closed — managed local config uses an explicitly supplied active personal token; keyless remote profiles remain read-only; client/helper suites and default/no-default serial workspace suites pass; M022 promoted to ready | `plans/closure/on-prem-user-access/017-status.md`, implementation `d726bce`, `218df45` | | On-prem user access M016 — final PR #7 integration and merge-readiness reconciliation | closed — local Rust/tooling, Q007, release-doc, browser and installed release-binary evidence pass; exact PR head `3d68c715` hosted CI, Linux/macOS installer and cargo-deny pass; merge held because no maintainer PR review has been submitted; no future plan depends on M016 | `plans/closure/on-prem-user-access/016-status.md`, PR #7, base `9a0a903c`, head `3d68c715`, CI `38721601336`, installer `38721601305`, audit `38721601318` | | On-prem user access M015 — independent authentication/authorization threat review | closed — independent review findings on hard-cost proof bounds and browser-session access corrected and re-reviewed; default 1,024 / no-default 1,025 tests pass (one ignored each); exact-SHA CI, installer and dependency audit pass on `d8fd9c2a`; no M015 security findings remain; M016 promoted to ready | `plans/closure/on-prem-user-access/015-status.md`, implementation `a526980c`, `deb1622f`, `d8fd9c2a`, security scan `52bd79cc`, CI `38720507683`, installer `38720509034`, audit `38720507720` | | On-prem user access M014 — Rustls dependency security-advisory corrective | closed — lockfile-only Rustls 0.23.46 patch, unchanged transport feature graph; local default/no-default and 1,009/1,010-test workspace suites, provider TLS (trusted root, wrong-root and hostname rejection), budget/coordinator suites, release build and tooling pass; hosted cargo-deny, CI and both installers pass on `7462263ec482c5fda4bd1a70a9a55b0efc313cd3`; main remains pending PR merge | `plans/closure/on-prem-user-access/014-status.md`, implementation `7462263e`, audit `38671345587`, CI `38671345679`, installers `38671345597` | @@ -527,3 +527,5 @@ unblocked. On-prem user-access M016 is ready after M014/M015 closure and still requires final-SHA gates; this optional diagnostic workstream has no edge to that sequence. The concurrency-performance roadmap remains active for any future separately scoped controlled-I/O/media/budget work; none is registered. + +Unblock audit (on-prem user access M017 closed, `plans/closure/on-prem-user-access/017-status.md`): searched `plans/implementation/on-prem-user-access/`, the roadmap dependency graph, and the dependency-ready, active, and blocked registry tables. M022's sole hard dependency, M017, is closed and its shared client/wire and hard-budget proof interfaces were already stable, so M022 moves `blocked` → `ready`. M020 remains blocked on M019; M021 remains blocked on M020; M023 remains blocked on M022 and its ordinary-server operational evidence. No other registered plan became eligible. M018 and M019 were already independent ready plans. No 49-user, team-capacity, or generated-output cap claim is introduced by this audit. diff --git a/plans/subsystems/on-prem-user-access-roadmap.md b/plans/subsystems/on-prem-user-access-roadmap.md index 6f15a965..752e1a4d 100644 --- a/plans/subsystems/on-prem-user-access-roadmap.md +++ b/plans/subsystems/on-prem-user-access-roadmap.md @@ -298,7 +298,7 @@ capacity claim was made. Closure: ### Post-merge M017–M023 corrective and team-utility milestones -#### M017 — Managed client credential reconciliation +#### M017 — Managed client credential reconciliation (closed) Class: capability. Ready independently. Repair `onboard`, `getkey/newkey`, `configsetup`, `configremote` and `eggpool-connect` @@ -340,7 +340,7 @@ Source: `plans/implementation/on-prem-user-access/021-default-member-policy-prov #### M022 — Hard-budget agent interoperability -Class: polish. Blocked on hard M017 closure. Measure whether client/surface +Class: polish. Ready after hard M017 closure. Measure whether client/surface requests carry sufficient output-token proof; improve diagnostics and introduce output-cap insertion only where upstream and wire evidence prove a hard bound. Do not regress M008/M015 hard-cost safety. @@ -356,7 +356,8 @@ Source: `plans/implementation/on-prem-user-access/023-short-window-fairness-cont ### Post-merge dependency and hardware policy -- Parallel ready: M017, M018 and M019. None depends on a Pi/Le Potato run. +- M017 is closed. M018, M019 and M022 are ready; none depends on a Pi/Le + Potato run. M022 became ready after the M017 closure. - Hard: M017 + M019 -> M020 -> M021; M017 -> M022 -> M023. - Interface: M018's privileged local transport must not broaden HTTP rights; any new auth authority stops for ADR review. M022 hard-cost proof contract @@ -452,10 +453,10 @@ A ready plan is not an implemented capability. | M014 | closed | `plans/implementation/on-prem-user-access/014-rustls-advisory-dependency-corrective.md` | `plans/closure/on-prem-user-access/014-status.md` | Rustls 0.23.46; exact-SHA hosted cargo-deny, CI, and installer jobs pass. Main receives the fix when PR #7 merges. | | M015 | closed | `plans/implementation/on-prem-user-access/015-authz-threat-review-corrective.md` | `plans/closure/on-prem-user-access/015-status.md` | Exact candidate `d8fd9c2a`; findings fixed and independently re-reviewed; exact-SHA hosted gates pass. | | M016 | closed | `plans/implementation/on-prem-user-access/016-final-pr7-merge-reconciliation.md` | `plans/closure/on-prem-user-access/016-status.md` | Exact candidate checks pass; merge held pending maintainer PR review. PR #7 subsequently merged at `88f58bbf`; historical M016 closure remains unchanged. | -| M017 | ready | `plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md` | — | Independent; full onboarding/desktop client replay needed. | +| M017 | closed | `plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md` | `plans/closure/on-prem-user-access/017-status.md` | Managed credential selection and keyless remote export; serial/default/no-default suites pass. | | M018 | ready | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | — | Independent; inspect local control authority/ADR threshold. | | M019 | ready | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | — | Independent; insecure remote GET password form. | | M020 | blocked | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | — | Hard: M017 and M019 closure. | | M021 | blocked | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | — | Hard: M020 closure. | -| M022 | blocked | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | — | Hard: M017 closure. | +| M022 | ready | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | — | Hard M017 closure is satisfied; preserve hard budget proof. | | M023 | blocked | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Hard: M022 closure; ordinary-server evidence operational. | From adc7278db49145f82a99783ff98a763c307eb752 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 12:24:46 +0000 Subject: [PATCH 23/62] Activate on-prem access plan 018 --- .../on-prem-user-access/018-local-operator-status-corrective.md | 2 +- plans/registry.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md b/plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md index 031350c8..f0e115bf 100644 --- a/plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md +++ b/plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md @@ -1,6 +1,6 @@ # On-Prem User Access M018 — Local operator status after identity cutover -Status: ready +Status: active Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index b4c742dc..c22d5cd9 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -67,7 +67,6 @@ No new runtime, schema or performance claim lands with these plans. |---|---|---|---|---| | Persistence | M011 — post-M010 roadmap and documentation reconciliation | ready | `plans/implementation/persistence/011-post-m010-roadmap-documentation-reconciliation.md` | Hard: M009/M010 closures accepted. Documentation/tooling only; no runtime or physical target dependency. | | On-prem user access | M022 — hard-budget agent interoperability | ready | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | Hard M017 is closed; preserve M008/M015 proof-or-reject guarantees. | -| On-prem user access | M018 — local operator status corrective | ready | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | Independent P1; preserve privileged HTTP fences, inspect local OS control authority. | | On-prem user access | M019 — browser transport and solo LAN corrective | ready | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | Independent P1; remote HTTP GET login form must not solicit credentials. | | On-prem concurrency performance | M002 — ordinary-server team baseline | ready | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | Independent evidence work; ordinary Linux + durable SSD/NVMe required for team-capacity claims, no Pi/microSD release gate. | @@ -75,6 +74,7 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| +| On-prem user access | M018 — local operator status corrective | active | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | Restore local OS-authorized status readback through bounded private control protocol. | ## Blocked work From 0a2a7e7c5450eb292ad84c21b01ffee09bd2432b Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 12:25:02 +0000 Subject: [PATCH 24/62] Correct keyless remote export regression --- rust/src/operations/integrations.rs | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/rust/src/operations/integrations.rs b/rust/src/operations/integrations.rs index 71b5895e..ff04ca22 100644 --- a/rust/src/operations/integrations.rs +++ b/rust/src/operations/integrations.rs @@ -3345,12 +3345,12 @@ mod tests { } #[tokio::test] - async fn remote_context_is_read_only_and_never_creates_keys() { + async fn remote_context_is_read_only_for_keyless_managed_config() { let directory = tempfile::tempdir().expect("tempdir"); let config_path = directory.path().join("config.toml"); let db_path = directory.path().join("usage.sqlite3"); - // No [server].api_key and no env: read-only export must fail rather - // than manufacturing a key file mutation. + // No [server].api_key and no env: managed export remains useful and + // must not manufacture a shared key file mutation. std::fs::write( &config_path, format!( @@ -3361,7 +3361,7 @@ mod tests { .expect("config"); let before = std::fs::read_to_string(&config_path).expect("before"); let result = build_remote_context(&config_path, None).await; - assert!(result.is_err()); + assert!(result.is_ok()); let after = std::fs::read_to_string(&config_path).expect("after"); assert_eq!(before, after); assert!(!after.contains("api_key")); From 235501240a8b961c63d936ddd35a2e7ca237e672 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 12:25:08 +0000 Subject: [PATCH 25/62] Record keyless export test correction --- plans/closure/on-prem-user-access/017-status.md | 1 + 1 file changed, 1 insertion(+) diff --git a/plans/closure/on-prem-user-access/017-status.md b/plans/closure/on-prem-user-access/017-status.md index 9d93a6a7..d42480dd 100644 --- a/plans/closure/on-prem-user-access/017-status.md +++ b/plans/closure/on-prem-user-access/017-status.md @@ -17,6 +17,7 @@ Implementation commits: - `d726bce` — managed local integration setup requires an active personal credential; keyless remote export and obsolete credential guidance reconciled. - `218df45` — explicit-credential and keyless-export regression coverage; remote instructions name personal tokens. - `c5167aa` — `getkey`/`newkey` reject retired shared-key use after administrator initialization. +- `0a2a7e7` — align the in-module keyless-export regression with the managed remote-export contract. ## 1. Executive finding From 8e2b66a732a853a7e489bd6d01df755110ff9de4 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 12:25:22 +0000 Subject: [PATCH 26/62] Add local operator status readback --- architecture/deep-dive-control.md | 16 +++-- docs/cli-reference.md | 4 +- docs/deployment.md | 13 +++- rust/src/operations/control.rs | 57 +++++++++++++++- rust/src/runtime.rs | 108 +++++++++++------------------- rust/src/server/health.rs | 10 ++- rust/src/server/mod.rs | 54 +++++++++++++++ rust/tests/operations_o002.rs | 33 +++++++++ 8 files changed, 214 insertions(+), 81 deletions(-) diff --git a/architecture/deep-dive-control.md b/architecture/deep-dive-control.md index 15ae7321..a1b9ee28 100644 --- a/architecture/deep-dive-control.md +++ b/architecture/deep-dive-control.md @@ -17,14 +17,20 @@ Back to [Architecture](README.md). See also the review index in [overview.md §1 ## Control socket -`rust/src/operations/control.rs` owns the Unix-domain JSON protocol, one request per connection, `PROTOCOL_VERSION = 1`, `MAX_REQUEST_BYTES = 65_536`, `CONTROL_TIMEOUT = 30s` (`rust/src/operations/control.rs:31-34`): +`rust/src/operations/control.rs` owns the Unix-domain JSON protocol, one request per connection, `PROTOCOL_VERSION = 1`, `MAX_REQUEST_BYTES = 65_536`, `CONTROL_TIMEOUT = 30s`: -- `ControlRequest` (`reload`, `parse_frame`): the only accepted command is `reload_config` (`control.rs:173-177`), with an optional 64-char lowercase hex `validated_digest`. Depth pre-checks bound parsing before DOM allocation. -- `ControlResponse` (`error`, `from_reload`): bounded metadata-only outcomes (stage, generation, changed sections, restart-required paths, retirement flag); no secrets, bodies, or provider error text. -- `ControlClient` (`new`, `with_timeout`, `reload`, `send`): typed one-shot client used by `rehash`. +- `ControlRequest` (`reload`, `readback`, `parse_frame`): accepts `reload_config`, `read_status`, and `read_runtime_status`. Reload alone accepts an optional 64-char lowercase hex `validated_digest`; readback requests reject parameters. Depth pre-checks bound parsing before DOM allocation. +- `ControlResponse` (`error`, `from_reload`, `with_payload`): bounded reload metadata or sanitized status projections; no secrets, bodies, or provider error text. Readback is capped below the 65,536-byte frame limit. +- `ControlClient` (`new`, `with_timeout`, `reload`, `read_status`, `send`): typed one-shot client used by `rehash`, `status`, and `runtime-status`. - `ControlServerHandle` (`path`, `close`) via `start`: sole process-local listener, socket mode `0o600`, runtime dir `0o700`, stale-socket identity checks; `close` unlinks only its own socket while a retained reload may still finish. -The control plane carries reload only. `runtime-status` is not a control command: `runtime.rs::fetch_runtime_status` (`rust/src/runtime.rs:2759-2761`) reads the authenticated `GET /api/stats/runtime` projection (`server/health.rs::runtime_status` at `rust/src/server/health.rs:129`), which renders `RuntimeDiagnosticsSnapshot`. +The same-user local operator reads status through this socket after identity +initialization; HTTP `/api/status` and `/api/stats/runtime` keep their existing +browser-session authorization. The socket resides in a private runtime +directory and is mode `0o600`, so possession of the OS-owned local identity is +the authority. Missing/stale sockets are reported as unavailable; malformed +or oversized protocol responses fail closed. Member tokens never gain status +or runtime-diagnostics access. ## Config mutation diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 17d32bc7..c94a9e42 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -39,8 +39,8 @@ Command list verified against `eggpool help` (0.8.2). | Command | Description | |---------|-------------| -| `eggpool status` | Concise proxy/provider health summary (one row per provider; `--json` for structured output) | -| `eggpool runtime-status` | Detailed process/runtime diagnostics (`--json` for the full snapshot) | +| `eggpool status` | Concise proxy/provider health summary from the private local control socket (one row per provider; `--json` for structured output) | +| `eggpool runtime-status` | Detailed local operator diagnostics from the private control socket (`--json` for the full snapshot) | | `eggpool stats transcoding` | Show protocol transcoding statistics | | `eggpool stats repair-costs` | Dry-run/apply repair for suspicious historical request costs | | `eggpool stats recompute-costs` | Recompute `cost_microdollars` on historical requests | diff --git a/docs/deployment.md b/docs/deployment.md index 2ede58d4..1292ad5f 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -769,7 +769,8 @@ eggpool runtime-status # detailed process/runtime diagnostics eggpool runtime-status --json # machine-readable JSON ``` -`eggpool status` calls the authenticated local `/api/status` endpoint and +`eggpool status` reads the sanitized snapshot over the same-user private local +control socket and prints a compact operator summary: one header line for proxy health, exactly one row per configured provider (`ready`/`degraded`/`unavailable`/`disabled`/`unknown` with a bounded reason code), and one runtime footer line. It aggregates @@ -782,10 +783,18 @@ as `unknown`/`disabled` from static config. `eggpool status --json` prints the typed server snapshot (deterministic provider ordering, numeric durations in named units, no credentials or raw upstream error text). -`eggpool runtime-status` calls the local `/api/stats/runtime` endpoint and +`eggpool runtime-status` reads the detailed projection over that local control +socket and prints a one-page overview of the running process. It is intended for operators debugging systemd, cron, or daemon deployments. +The control runtime directory is private and its Unix socket is mode `0600`. +This grants readback only to the operating-system account that owns the daemon +and socket; it does not change HTTP authorization. `/api/status` and +`/api/stats/runtime` remain browser-session protected after identity bootstrap, +and personal API tokens cannot access them. Missing or stale sockets are +reported as unavailable; malformed or oversized responses fail closed. + When the server is not running the command exits non-zero with a clear message. It does not start the server. diff --git a/rust/src/operations/control.rs b/rust/src/operations/control.rs index 580d0006..ee205f3b 100644 --- a/rust/src/operations/control.rs +++ b/rust/src/operations/control.rs @@ -126,6 +126,16 @@ impl ControlRequest { } } + pub fn readback(request_id: impl Into, command: &str) -> Self { + Self { + protocol_version: PROTOCOL_VERSION, + request_id: request_id.into(), + command: command.to_owned(), + validated_digest: None, + params: None, + } + } + pub fn parse_frame(frame: &[u8]) -> Result { if frame.is_empty() || frame == b"\n" || frame == b"\r\n" { return Err(ProtocolError::Empty); @@ -173,7 +183,12 @@ impl ControlRequest { let command = object .get("command") .and_then(Value::as_str) - .filter(|value| *value == "reload_config") + .filter(|value| { + matches!( + *value, + "reload_config" | "read_status" | "read_runtime_status" + ) + }) .ok_or(ProtocolError::UnknownCommand)?; let validated_digest = match object.get("validated_digest") { None | Some(Value::Null) => None, @@ -192,6 +207,9 @@ impl ControlRequest { Some(Value::Object(value)) => Some(value.clone().into_iter().collect()), _ => return Err(ProtocolError::InvalidParams), }; + if command != "reload_config" && (validated_digest.is_some() || params.is_some()) { + return Err(ProtocolError::InvalidParams); + } Ok(Self { protocol_version, request_id: request_id.to_owned(), @@ -214,6 +232,8 @@ pub struct ControlResponse { pub restart_required: Vec, pub retirement_pending: bool, pub message: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub payload: Option, } impl ControlResponse { @@ -229,6 +249,7 @@ impl ControlResponse { restart_required: Vec::new(), retirement_pending: false, message: message.to_owned(), + payload: None, } } @@ -259,6 +280,23 @@ impl ControlResponse { restart_required: result.restart_required_paths, retirement_pending: result.retirement_pending, message: message.to_owned(), + payload: None, + } + } + + pub fn with_payload(request_id: impl Into, payload: Value) -> Self { + Self { + protocol_version: PROTOCOL_VERSION, + request_id: request_id.into(), + ok: true, + stage: "readback".to_owned(), + generation: None, + changed_sections: Vec::new(), + warnings: Vec::new(), + restart_required: Vec::new(), + retirement_pending: false, + message: "local status snapshot".to_owned(), + payload: Some(payload), } } } @@ -325,6 +363,23 @@ impl ControlClient { .await } + pub async fn read_status(&self, runtime: bool) -> Result { + let request_id = format!( + "rust-{}-{}", + std::process::id(), + NEXT_REQUEST_ID.fetch_add(1, Ordering::Relaxed) + ); + self.send(ControlRequest::readback( + request_id, + if runtime { + "read_runtime_status" + } else { + "read_status" + }, + )) + .await + } + pub async fn send( &self, request: ControlRequest, diff --git a/rust/src/runtime.rs b/rust/src/runtime.rs index cc976782..e9b1ee5d 100644 --- a/rust/src/runtime.rs +++ b/rust/src/runtime.rs @@ -8,8 +8,6 @@ use std::{ use serde_json::{Value, json}; use tokio::{ - io::{AsyncReadExt, AsyncWriteExt}, - net::TcpStream, process::Command as TokioCommand, time::{Instant, sleep, timeout}, }; @@ -40,8 +38,6 @@ const EXIT_CONTROL_UNAVAILABLE: u8 = 3; const EXIT_RELOAD_BUSY: u8 = 4; const EXIT_PREPARATION_FAILED: u8 = 5; const EXIT_DIGEST_MISMATCH: u8 = 6; -const MAX_STATUS_BODY_BYTES: usize = 1_048_576; -const STATUS_TIMEOUT: Duration = Duration::from_secs(5); /// Initialize process-local diagnostics and dispatch the operational CLI. pub async fn run(cli: Cli) -> Result<(), BootstrapError> { @@ -3220,77 +3216,50 @@ async fn runtime_status(path: &Path, json_output: bool) -> Result<(), BootstrapE } async fn fetch_runtime_status(config: &config::Config) -> Result { - fetch_local_json(config, "/api/stats/runtime", "runtime-status").await + let _ = config; + fetch_local_control_status(true).await } async fn fetch_status_snapshot(config: &config::Config) -> Result { - fetch_local_json(config, "/api/status", "status").await + let _ = config; + fetch_local_control_status(false).await } -/// Bounded authenticated local-control fetch shared by operational status -/// commands. No outbound provider requests; one TCP connection, one bounded -/// body, strict JSON parsing. -async fn fetch_local_json( - config: &config::Config, - path: &str, - label: &str, -) -> Result { - let host = match config.server.host.as_str() { - "0.0.0.0" => "127.0.0.1", - "::" => "::1", - value => value, - }; - let mut stream = timeout( - STATUS_TIMEOUT, - TcpStream::connect((host, config.server.port)), - ) - .await - .map_err(|error| format!("{label} connection timed out: {error}"))? - .map_err(|error| format!("{label} server is not running: {error}"))?; - let mut request = format!("GET {path} HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n"); - if let Some(key) = config.resolved_server_api_key() { - request.push_str(&format!("Authorization: Bearer {key}\r\n")); - } - request.push_str("\r\n"); - timeout(STATUS_TIMEOUT, stream.write_all(request.as_bytes())) +async fn fetch_local_control_status(runtime: bool) -> Result { + let paths = RuntimePaths::resolve(); + let response = crate::operations::control::ControlClient::new(&paths.control_socket) + .read_status(runtime) .await - .map_err(|error| format!("{label} request timed out: {error}"))? - .map_err(|error| format!("{label} request failed: {error}"))?; - let mut bytes = Vec::with_capacity(4096); - let mut chunk = [0_u8; 8192]; - loop { - let count = timeout(STATUS_TIMEOUT, stream.read(&mut chunk)) - .await - .map_err(|error| format!("{label} response timed out: {error}"))? - .map_err(|error| format!("{label} response failed: {error}"))?; - if count == 0 { - break; - } - if bytes.len().saturating_add(count) > MAX_STATUS_BODY_BYTES { - return Err(format!("{label} response body is too large")); - } - bytes.extend_from_slice(&chunk[..count]); - } - let Some(header_end) = bytes.windows(4).position(|window| window == b"\r\n\r\n") else { - return Err(format!("{label} response was malformed")); - }; - let headers = &bytes[..header_end]; - let status = headers - .split(|byte| *byte == b'\n') - .next() - .and_then(|line| line.split(|byte| *byte == b' ').nth(1)) - .and_then(|value| std::str::from_utf8(value).ok()) - .and_then(|value| value.trim().parse::().ok()) - .ok_or_else(|| format!("{label} response status was malformed"))?; - if status != 200 { - return Err(match status { - 401 | 403 => format!("{label} authentication failed (HTTP {status})"), - 404 => format!("{label} endpoint is unavailable"), - _ => format!("{label} server returned HTTP {status}"), - }); + .map_err(|error| match error { + crate::operations::control::ControlClientError::Connection( + std::io::ErrorKind::PermissionDenied, + ) => "local operator does not own the control socket".to_owned(), + crate::operations::control::ControlClientError::Connection(_) => { + match process::read_pid(&paths.pid_file).ok().flatten() { + Some(pid) if !process::process_exists(pid) => { + format!("control socket is absent and the PID file is stale (PID {pid})") + } + Some(_) => "daemon process exists but its control socket is unavailable".into(), + None => "daemon control socket is absent; the server is not running".into(), + } + } + crate::operations::control::ControlClientError::Timeout => { + "local control socket timed out".to_owned() + } + crate::operations::control::ControlClientError::Protocol(_) => { + "local control response was malformed or oversized".to_owned() + } + other => other.to_string(), + })?; + if !response.ok { + return Err(format!( + "local status readback failed: {}", + response.message + )); } - serde_json::from_slice(&bytes[header_end + 4..]) - .map_err(|error| format!("{label} response body was malformed JSON: {error}")) + response + .payload + .ok_or_else(|| "local status response did not include a payload".to_owned()) } async fn status(path: &Path, json_output: bool) -> Result<(), BootstrapError> { @@ -3344,6 +3313,9 @@ fn is_auth_or_malformed_status_error(detail: &str) -> bool { detail.contains("authentication failed") || detail.contains("malformed") || detail.contains("too large") + || detail.contains("invalid control response") + || detail.contains("control response was malformed") + || detail.contains("does not own the control socket") } fn sanitize_status_value(value: &Value) -> Value { diff --git a/rust/src/server/health.rs b/rust/src/server/health.rs index 07ce95f9..e32e9341 100644 --- a/rust/src/server/health.rs +++ b/rust/src/server/health.rs @@ -196,6 +196,10 @@ pub(super) async fn integration_profile( } pub(super) async fn runtime_status(State(state): State) -> Response { + json_response(StatusCode::OK, runtime_status_value(&state).await) +} + +pub(super) async fn runtime_status_value(state: &AppState) -> Value { let diagnostics = state .process .as_ref() @@ -304,11 +308,11 @@ pub(super) async fn runtime_status(State(state): State) -> Response { { object.insert("database_qualification".to_owned(), value); } - json_response(StatusCode::OK, body) + body } #[cfg(not(feature = "qualification-db-diagnostics"))] { - json_response(StatusCode::OK, body) + body } } @@ -471,7 +475,7 @@ pub(super) async fn status_api(State(state): State) -> Response { } } -async fn build_status_snapshot(state: &AppState) -> status_service::ProxyStatusSnapshot { +pub(super) async fn build_status_snapshot(state: &AppState) -> status_service::ProxyStatusSnapshot { use std::collections::{BTreeMap, BTreeSet}; use std::time::{SystemTime, UNIX_EPOCH}; diff --git a/rust/src/server/mod.rs b/rust/src/server/mod.rs index 39c4356b..d46a3728 100644 --- a/rust/src/server/mod.rs +++ b/rust/src/server/mod.rs @@ -896,10 +896,64 @@ pub async fn run_with_digest( let reload_service = process.reload_service((*manager).clone()); let config_path_for_control = process.config_path().map(Path::to_path_buf); let control_path = crate::operations::paths::RuntimePaths::resolve().control_socket; + let status_state = AppState { + server: ServerState::from_config(&config), + database: database.clone(), + runtime: Arc::clone(&manager), + process: Some(process.clone()), + body_tasks: BodyTaskTracker::new(), + }; let control_server = match crate::operations::control::start(control_path, move |request| { let reload_service = reload_service.clone(); let config_path = config_path_for_control.clone(); + let status_state = status_state.clone(); async move { + match request.command.as_str() { + "read_status" => { + let snapshot = health::build_status_snapshot(&status_state).await; + return match serde_json::to_value(snapshot) { + Ok(payload) + if serde_json::to_vec(&payload).is_ok_and(|bytes| { + bytes.len() < crate::operations::control::MAX_REQUEST_BYTES - 2048 + }) => + { + crate::operations::control::ControlResponse::with_payload( + request.request_id, + payload, + ) + } + Err(_) => crate::operations::control::ControlResponse::error( + request.request_id, + "readback", + "local status projection failed", + ), + _ => crate::operations::control::ControlResponse::error( + request.request_id, + "readback", + "status projection exceeds the control-response bound", + ), + }; + } + "read_runtime_status" => { + let payload = health::runtime_status_value(&status_state).await; + let within_bound = serde_json::to_vec(&payload).is_ok_and(|bytes| { + bytes.len() < crate::operations::control::MAX_REQUEST_BYTES + }); + return if within_bound { + crate::operations::control::ControlResponse::with_payload( + request.request_id, + payload, + ) + } else { + crate::operations::control::ControlResponse::error( + request.request_id, + "readback", + "runtime status projection exceeds the control-response bound", + ) + }; + } + _ => {} + } let Some(config_path) = config_path else { return crate::operations::control::ControlResponse::error( request.request_id, diff --git a/rust/tests/operations_o002.rs b/rust/tests/operations_o002.rs index e140393c..fd006a1c 100644 --- a/rust/tests/operations_o002.rs +++ b/rust/tests/operations_o002.rs @@ -367,6 +367,39 @@ extra ControlRequest::parse_frame(wrong), Err(ProtocolError::WrongVersion) ); + + let readback = + serde_json::to_vec(&ControlRequest::readback("status", "read_status")).expect("serializes"); + assert_eq!( + ControlRequest::parse_frame(&[readback, vec![b'\n']].concat()) + .expect("readback is accepted") + .command, + "read_status" + ); +} + +#[tokio::test] +async fn local_control_returns_bounded_status_payload() { + let root = tempfile::tempdir().expect("temp root"); + private_directory(root.path()); + let socket = root.path().join("runtime/status.sock"); + let server = start(&socket, |request| async move { + assert_eq!(request.command, "read_status"); + ControlResponse::with_payload(request.request_id, serde_json::json!({"status":"ready"})) + }) + .await + .expect("listener starts"); + + let response = ControlClient::new(socket) + .read_status(false) + .await + .expect("status readback succeeds"); + assert!(response.ok); + assert_eq!( + response.payload, + Some(serde_json::json!({"status":"ready"})) + ); + server.close().await.expect("listener closes"); } #[tokio::test] From dee217558badbc6a48cb6d0d5d329e4852455788 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 12:25:46 +0000 Subject: [PATCH 27/62] Start closing on-prem access plan 018 --- .../on-prem-user-access/018-local-operator-status-corrective.md | 2 +- plans/registry.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md b/plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md index f0e115bf..610362c1 100644 --- a/plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md +++ b/plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md @@ -1,6 +1,6 @@ # On-Prem User Access M018 — Local operator status after identity cutover -Status: active +Status: closing Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index c22d5cd9..8dd46575 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -74,7 +74,7 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| -| On-prem user access | M018 — local operator status corrective | active | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | Restore local OS-authorized status readback through bounded private control protocol. | +| On-prem user access | M018 — local operator status corrective | closing | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | Bounded control protocol and status projection checks passed; closure audit in progress. | ## Blocked work From 3be5d83feec88da7ee3a742ae37dcb882f7f3c39 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 12:27:42 +0000 Subject: [PATCH 28/62] Prove keyless local status CLI readback --- rust/tests/status_command.rs | 65 +++++++++++++++++++++++++++++++++++- 1 file changed, 64 insertions(+), 1 deletion(-) diff --git a/rust/tests/status_command.rs b/rust/tests/status_command.rs index 2f8c4a40..aa5fe80d 100644 --- a/rust/tests/status_command.rs +++ b/rust/tests/status_command.rs @@ -14,11 +14,16 @@ use eggpool::{ Cli, Command as EggpoolCommand, Config, db::{AccountRepository, Database, DatabaseConfig, MigrationRunner, PingRepository}, health::{AccountHealthSnapshot, BackoffReason, CircuitState, CircuitStats}, - operations::status as status_service, + operations::{ + control::{ControlResponse, start}, + status as status_service, + }, runtime_lifecycle::{ProcessRuntime, RuntimeGenerationFactory, RuntimeManager}, server::{AppState, build_router}, }; use http_body_util::BodyExt; +use tempfile::tempdir; +use tokio::process::Command as TokioCommand; use tower::ServiceExt; fn account_input( @@ -76,6 +81,64 @@ fn status_cli_parses_with_and_without_json() { )); } +#[tokio::test] +async fn status_cli_uses_private_control_readback_without_a_server_key() { + let directory = tempdir().expect("temporary directory"); + let runtime_dir = directory.path().join("runtime"); + std::fs::create_dir(&runtime_dir).expect("runtime directory"); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(&runtime_dir, std::fs::Permissions::from_mode(0o700)) + .expect("private runtime directory"); + } + let config_path = directory.path().join("config.toml"); + std::fs::write( + &config_path, + "[server]\nhost = \"0.0.0.0\"\nport = 11300\n\n[database]\npath = \":memory:\"\n", + ) + .expect("keyless config"); + let socket = runtime_dir.join("eggpool.sock"); + let control = start(&socket, |request| async move { + let payload = if request.command == "read_status" { + serde_json::json!({ + "schema_version": 1, + "proxy": {"status":"ready", "ready":true, "base_url":"http://127.0.0.1:11300", "version":"test", "model_count":0, "routable_accounts":0, "enabled_accounts":0}, + "providers": [], + "runtime": {"generation":1} + }) + } else { + serde_json::json!({"status":"ok", "server":{"pid":1234,"uptime_seconds":1.0}}) + }; + ControlResponse::with_payload(request.request_id, payload) + }) + .await + .expect("private control listener"); + + for (command, expected) in [ + ("status", "\"status\": \"ready\""), + ("runtime-status", "\"status\": \"ok\""), + ] { + let output = TokioCommand::new(env!("CARGO_BIN_EXE_eggpool")) + .args([command, "--json"]) + .env("EGGPOOL_CONFIG", &config_path) + .env("EGGPOOL_RUNTIME_DIR", &runtime_dir) + .output() + .await + .expect("status CLI executes"); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + let stdout = String::from_utf8_lossy(&output.stdout); + assert!(stdout.contains(expected), "{stdout}"); + assert!(!stdout.contains("api_key"), "{stdout}"); + } + + control.close().await.expect("control socket closes"); +} + #[test] fn provider_rows_are_one_per_provider_and_deterministically_ordered() { let mut providers = vec![ From 49d2c4efbf3c6c24d1d5a944a7d3b1f8bbc1d1eb Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 12:28:26 +0000 Subject: [PATCH 29/62] Close on-prem local status corrective plan 018 --- .../closure/on-prem-user-access/018-status.md | 87 +++++++++++++++++++ .../018-local-operator-status-corrective.md | 2 +- plans/registry.md | 6 +- .../subsystems/on-prem-user-access-roadmap.md | 4 +- 4 files changed, 94 insertions(+), 5 deletions(-) create mode 100644 plans/closure/on-prem-user-access/018-status.md diff --git a/plans/closure/on-prem-user-access/018-status.md b/plans/closure/on-prem-user-access/018-status.md new file mode 100644 index 00000000..61d6d4f8 --- /dev/null +++ b/plans/closure/on-prem-user-access/018-status.md @@ -0,0 +1,87 @@ +# On-Prem User Access M018 — Closure Status + +Status: closed + +Source implementation plan: + +- `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` + +Source subsystem roadmap: + +- `plans/subsystems/on-prem-user-access-roadmap.md#m018--local-operator-status-corrective` + +Repository baseline reviewed: post-merge `88f58bbf`; implementation candidate +`8e2b66a` with CLI regression fixture `3be5d83`. + +## 1. Executive finding + +The local operator can read status and runtime status through the existing +owner-protected Unix control socket after identity cutover. The CLI no longer +depends on the retired shared API key for these read-only projections. The +HTTP status routes retain their managed-mode authorization; personal inference +credentials do not gain browser or operator status rights. The control channel +uses bounded, versioned requests and bounded JSON payloads, and the local +listener remains restricted to a private runtime directory. + +## 2. Requirement-to-evidence matrix + +| Requirement | Evidence | Result | Notes | +|---|---|---|---| +| Existing OS-local authority is used without network privilege | `rust/src/operations/control.rs`; private runtime directory checks; `deep-dive-control.md` | pass | No TCP listener, new process, or HTTP auth exception. | +| Local status and runtime status work without a configured server key | `rust/src/runtime.rs`; `rust/tests/status_command.rs::status_cli_uses_private_control_readback_without_a_server_key` | pass | Invokes both actual CLI commands against a private local control listener. | +| Response is bounded and protocol failures remain errors | `control.rs` readback request/response cap and timeout; `operations_o002` | pass | Existing control protocol limits are retained for these operations. | +| Managed HTTP privilege remains isolated | `rust/src/server/middleware.rs`; `on_prem_sessions` and server tests | pass | No personal-token scope or managed loopback bypass was added. | +| Operator documentation explains authority and failure modes | `docs/cli-reference.md`, `docs/deployment.md`, `architecture/deep-dive-control.md` | pass | No provider probe is performed by status readback. | + +## 3. Implementation evidence + +- `rust/src/operations/control.rs` adds read-only status and runtime-status + commands on the private control protocol and a typed client readback method. +- `rust/src/server/mod.rs` serves snapshots from the local control handler; + status aggregation remains owned by the status/health projection. +- `rust/src/runtime.rs` routes both CLI commands through local control and + reports absent, stale, malformed, timeout, and permission errors without an + HTTP/API-key fallback. +- `rust/tests/operations_o002.rs` covers the bounded control readback contract; + `rust/tests/status_command.rs` runs both real CLI commands with a keyless + config and private socket. +- No migration, new database writer, provider call, or public HTTP route was + added. + +## 4. Verification executed + +- `cargo test --manifest-path rust/Cargo.toml --test operations_o002 -- --test-threads=1` — 9 passed. +- `cargo test --manifest-path rust/Cargo.toml --test status_command -- --test-threads=1` — 14 passed, including the keyless CLI readback fixture. +- `cargo test --manifest-path rust/Cargo.toml --test on_prem_sessions -- --test-threads=1` — 10 passed. +- Default workspace serial suite — 1,028 passed, 1 ignored across 79 suites (before the additive CLI fixture). +- No-default workspace serial suite — 1,029 passed, 1 ignored across 79 suites (before the additive CLI fixture). +- Strict default and no-default Clippy, formatting, release-document validation, and `git diff --check` passed with the implementation present. The additive fixture passed its focused target. +- These are local results; hosted CI and deployed systemd integration were not run in this closure pass. + +## 5. Invariant and recovery review + +The privileged surface remains a local filesystem socket in an OS-private +runtime directory. No personal token is accepted by the status command, and +managed HTTP authorization remains unchanged. The CLI fails closed when the +socket is unavailable or its response is invalid; it does not infer readiness +from process existence or an offline snapshot. Readback is bounded and does +not hold the SQLite writer gate while waiting on provider or network work. + +## 6. Unresolved findings + +| Severity | Finding | Impact | Required action | +|---|---|---|---| +| none | No unresolved M018 implementation finding | — | — | + +## 7. Roadmap and unblock disposition + +M018 is closed. The dependency audit searched the on-prem implementation +plans, roadmap graph, and ready/active/blocked registry tables. No registered +plan has a hard dependency on M018, so no blocked work is promoted. M019 and +M022 remain ready; M020 remains blocked on M019 (M017 is already closed), +M021 remains blocked on M020, and M023 remains blocked on M022 plus ordinary- +server operational evidence. No 49-user, concurrent-capacity, or generated +output-cap claim is introduced. + +The registry and roadmap record M018 closed. Historical M001–M017 closure +records were not modified. diff --git a/plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md b/plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md index 610362c1..bfc9304e 100644 --- a/plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md +++ b/plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md @@ -1,6 +1,6 @@ # On-Prem User Access M018 — Local operator status after identity cutover -Status: closing +Status: implemented Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index 8dd46575..2f5680a1 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -58,7 +58,7 @@ No new runtime, schema or performance claim lands with these plans. | Persistence | active | `plans/subsystems/persistence-roadmap.md` | M011 ready — post-M010 roadmap/documentation reconciliation | M009/M010 closures accepted. Documentation/tooling-only corrective; production remains WAL/NORMAL. No technical persistence successor is registered. | | Deployment and packaging | closed | `plans/subsystems/deployment-packaging-roadmap.md` | M005 closed — active PATH integration detection corrective | M001–M005 closed; no successor registered. | | Dashboard | closed | `plans/subsystems/dashboard-roadmap.md` | M012 closed — lifecycle closure and documentation polish | M001–M012 closed; no registered successor; future dashboard work requires a new bounded plan. | -| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M018/M019/M022 ready — local operator status, login transport, and hard-budget compatibility | M017 closed; M020/M021/M023 retain their hard dependencies. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | +| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M019/M022 ready — login transport and hard-budget compatibility | M017/M018 closed; M020/M021/M023 retain their hard dependencies. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | | On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 ready — ordinary-server small-team synthetic capacity baseline | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 is independent measurement on ordinary on-prem Linux with durable SSD/NVMe (operational target required), not an authorization to change WAL/SQLite topology. | ## Dependency-ready implementation plans @@ -74,7 +74,6 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| -| On-prem user access | M018 — local operator status corrective | closing | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | Bounded control protocol and status projection checks passed; closure audit in progress. | ## Blocked work @@ -115,6 +114,7 @@ Historical M007 blocker assessment at baseline `7e241ad` (`plans/closure/persist | Subsystem / plan | Disposition | Evidence | |---|---|---| +| On-prem user access M018 — local operator status corrective | closed — status and runtime status use bounded private control-socket readback without a server API key; managed HTTP authorization is unchanged; no successor unblocked | `plans/closure/on-prem-user-access/018-status.md`, implementation `8e2b66a`, CLI fixture `3be5d83` | | On-prem user access M017 — managed client credential reconciliation | closed — managed local config uses an explicitly supplied active personal token; keyless remote profiles remain read-only; client/helper suites and default/no-default serial workspace suites pass; M022 promoted to ready | `plans/closure/on-prem-user-access/017-status.md`, implementation `d726bce`, `218df45` | | On-prem user access M016 — final PR #7 integration and merge-readiness reconciliation | closed — local Rust/tooling, Q007, release-doc, browser and installed release-binary evidence pass; exact PR head `3d68c715` hosted CI, Linux/macOS installer and cargo-deny pass; merge held because no maintainer PR review has been submitted; no future plan depends on M016 | `plans/closure/on-prem-user-access/016-status.md`, PR #7, base `9a0a903c`, head `3d68c715`, CI `38721601336`, installer `38721601305`, audit `38721601318` | | On-prem user access M015 — independent authentication/authorization threat review | closed — independent review findings on hard-cost proof bounds and browser-session access corrected and re-reviewed; default 1,024 / no-default 1,025 tests pass (one ignored each); exact-SHA CI, installer and dependency audit pass on `d8fd9c2a`; no M015 security findings remain; M016 promoted to ready | `plans/closure/on-prem-user-access/015-status.md`, implementation `a526980c`, `deb1622f`, `d8fd9c2a`, security scan `52bd79cc`, CI `38720507683`, installer `38720509034`, audit `38720507720` | @@ -529,3 +529,5 @@ to that sequence. The concurrency-performance roadmap remains active for any future separately scoped controlled-I/O/media/budget work; none is registered. Unblock audit (on-prem user access M017 closed, `plans/closure/on-prem-user-access/017-status.md`): searched `plans/implementation/on-prem-user-access/`, the roadmap dependency graph, and the dependency-ready, active, and blocked registry tables. M022's sole hard dependency, M017, is closed and its shared client/wire and hard-budget proof interfaces were already stable, so M022 moves `blocked` → `ready`. M020 remains blocked on M019; M021 remains blocked on M020; M023 remains blocked on M022 and its ordinary-server operational evidence. No other registered plan became eligible. M018 and M019 were already independent ready plans. No 49-user, team-capacity, or generated-output cap claim is introduced by this audit. + +Unblock audit (on-prem user access M018 closed, `plans/closure/on-prem-user-access/018-status.md`): searched the on-prem implementation plans, roadmap dependency graph, and dependency-ready, active, and blocked registry tables. No registered plan has a hard M018 dependency, so no blocked row is promoted. M019 and M022 remain ready; M020 remains blocked on M019, M021 on M020, and M023 on M022 plus ordinary-server operational evidence. No capacity or generated-output cap claim is introduced. diff --git a/plans/subsystems/on-prem-user-access-roadmap.md b/plans/subsystems/on-prem-user-access-roadmap.md index 752e1a4d..4de49586 100644 --- a/plans/subsystems/on-prem-user-access-roadmap.md +++ b/plans/subsystems/on-prem-user-access-roadmap.md @@ -356,7 +356,7 @@ Source: `plans/implementation/on-prem-user-access/023-short-window-fairness-cont ### Post-merge dependency and hardware policy -- M017 is closed. M018, M019 and M022 are ready; none depends on a Pi/Le +- M017 and M018 are closed. M019 and M022 are ready; neither depends on a Pi/Le Potato run. M022 became ready after the M017 closure. - Hard: M017 + M019 -> M020 -> M021; M017 -> M022 -> M023. - Interface: M018's privileged local transport must not broaden HTTP rights; @@ -454,7 +454,7 @@ A ready plan is not an implemented capability. | M015 | closed | `plans/implementation/on-prem-user-access/015-authz-threat-review-corrective.md` | `plans/closure/on-prem-user-access/015-status.md` | Exact candidate `d8fd9c2a`; findings fixed and independently re-reviewed; exact-SHA hosted gates pass. | | M016 | closed | `plans/implementation/on-prem-user-access/016-final-pr7-merge-reconciliation.md` | `plans/closure/on-prem-user-access/016-status.md` | Exact candidate checks pass; merge held pending maintainer PR review. PR #7 subsequently merged at `88f58bbf`; historical M016 closure remains unchanged. | | M017 | closed | `plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md` | `plans/closure/on-prem-user-access/017-status.md` | Managed credential selection and keyless remote export; serial/default/no-default suites pass. | -| M018 | ready | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | — | Independent; inspect local control authority/ADR threshold. | +| M018 | closed | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | `plans/closure/on-prem-user-access/018-status.md` | Private local control readback restores status after identity cutover; no HTTP privilege change; no successor unblocked. | | M019 | ready | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | — | Independent; insecure remote GET password form. | | M020 | blocked | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | — | Hard: M017 and M019 closure. | | M021 | blocked | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | — | Hard: M020 closure. | From b437524e2fd1bb05eb9f987602a0276cf1915cc1 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 12:28:42 +0000 Subject: [PATCH 30/62] Activate on-prem login transport plan 019 --- .../019-login-transport-solo-lan-corrective.md | 2 +- plans/registry.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md b/plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md index 26cbd1a5..9bad53e4 100644 --- a/plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md +++ b/plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md @@ -1,6 +1,6 @@ # On-Prem User Access M019 — Login transport and solo LAN operability corrective -Status: ready +Status: active Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index 2f5680a1..756a6885 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -67,13 +67,13 @@ No new runtime, schema or performance claim lands with these plans. |---|---|---|---|---| | Persistence | M011 — post-M010 roadmap and documentation reconciliation | ready | `plans/implementation/persistence/011-post-m010-roadmap-documentation-reconciliation.md` | Hard: M009/M010 closures accepted. Documentation/tooling only; no runtime or physical target dependency. | | On-prem user access | M022 — hard-budget agent interoperability | ready | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | Hard M017 is closed; preserve M008/M015 proof-or-reject guarantees. | -| On-prem user access | M019 — browser transport and solo LAN corrective | ready | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | Independent P1; remote HTTP GET login form must not solicit credentials. | | On-prem concurrency performance | M002 — ordinary-server team baseline | ready | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | Independent evidence work; ordinary Linux + durable SSD/NVMe required for team-capacity claims, no Pi/microSD release gate. | ## Active implementation plans | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| +| On-prem user access | M019 — browser transport and solo LAN corrective | active | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | GET login form now shares POST transport validation; solo SSH tunnel and team HTTPS guidance in progress. | ## Blocked work From b18a561cc29b8b17dfdd2a1e2bb0961af5748fe9 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 12:40:14 +0000 Subject: [PATCH 31/62] Gate browser login on trusted transport --- README.md | 6 +++ docs/deployment.md | 7 +++ docs/raspberry-pi.md | 7 +++ rust/src/server/auth.rs | 117 +++++++++++++++++++++++++++++++++++++++- 4 files changed, 135 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 3cc85e3e..1366c9ea 100644 --- a/README.md +++ b/README.md @@ -116,6 +116,12 @@ Generated configs reference `EGGPOOL_API_KEY` without embedding the key. Set it By default EggPool binds `0.0.0.0:11300` and serves a public read-only dashboard at `http://:11300/`. To share with desktops on your LAN, set `[integrations].advertise_base_url = "http://:11300/v1"`, run `eggpool rehash` (advertised URL is live-reloadable profile output; it never changes the listen socket), then export a profile with `eggpool configremote codex`. Lock it down with `[server].host = "127.0.0.1"` (local only) or `eggpool dashboard public --off` (key required on the dashboard too). See [Firewall](docs/firewall.md). +Browser password sign-in is shown only over loopback or trusted HTTPS proxy +transport. A remote plaintext HTTP visit to `/login` receives an explanation +without a username or password form. For a solo headless device, use SSH port +forwarding to loopback; teams should use an operator-managed HTTPS proxy or +VPN. LAN inference tokens remain a separate API transport choice. + ## CLI Essentials | Command | Description | diff --git a/docs/deployment.md b/docs/deployment.md index 1292ad5f..41f55637 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -454,6 +454,13 @@ Host header and the firewall must prevent direct access around the proxy. The optional `[security].allow_insecure_web_auth = true` switch is for isolated development only and emits a startup warning. +The `GET /login` handler applies the same transport decision before rendering +the form. Remote direct HTTP receives a static help page with no username or +password controls and performs no database access. Solo headless operators can +forward the browser locally with `ssh -L 11300:127.0.0.1:11300 operator@host` +and open `http://127.0.0.1:11300/login`; remote teams should use the trusted +HTTPS proxy path above. + Optional background features that remain explicit: set `[backup].enabled = true` for in-process archives or `[update_checker].enabled = true` for the periodic PyPI status probe. Set `[model_info].enabled = false` to disable the diff --git a/docs/raspberry-pi.md b/docs/raspberry-pi.md index 77be7dd1..21d33d7f 100644 --- a/docs/raspberry-pi.md +++ b/docs/raspberry-pi.md @@ -279,6 +279,13 @@ saturates. 3. Dashboard: `http://:11300/` 4. Point OpenCode at `http://:11300` +Browser password sign-in is not offered over remote plaintext HTTP. From a +solo operator workstation, create a loopback tunnel with +`ssh -L 11300:127.0.0.1:11300 operator@`, then open +`http://127.0.0.1:11300/login`. This adds no service to the Pi. Team browser +administration should use an operator-managed HTTPS proxy or VPN; the LAN +inference endpoint remains separate from browser password transport. + ## Headless Pi + Desktop Clients On the Pi, advertise one stable URL once (live-reloadable; never the bind diff --git a/rust/src/server/auth.rs b/rust/src/server/auth.rs index 38e8c480..68e8f727 100644 --- a/rust/src/server/auth.rs +++ b/rust/src/server/auth.rs @@ -27,8 +27,33 @@ pub(super) struct PasswordChangeForm { csrf: String, } -pub(super) async fn login_page() -> Html { - Html(login_html(None)) +pub(super) async fn login_page( + State(state): State, + headers: HeaderMap, + peer: Option>, +) -> Response { + let connection = peer.as_ref().map(|extension| &extension.0.0); + let (status, html) = login_page_for_transport(connection, &headers, &state.server); + (status, Html(html)).into_response() +} + +fn login_page_for_transport( + connection: Option<&eggserve_primitives::connection_info::ConnectionInfo>, + headers: &HeaderMap, + server: &ServerState, +) -> (StatusCode, String) { + if web_auth_transport_allowed(connection, headers, server) { + (StatusCode::OK, login_html(None)) + } else { + ( + StatusCode::FORBIDDEN, + unsafe_login_transport_html().to_owned(), + ) + } +} + +fn unsafe_login_transport_html() -> &'static str { + "Secure sign-in required

Secure sign-in required

This connection is not local and is not protected by a trusted HTTPS proxy. EggPool did not request a username or password.

For a solo headless device, use an SSH port forward such as ssh -L 11300:127.0.0.1:11300 operator@device and open http://127.0.0.1:11300/login.

For a team server, use an operator-managed HTTPS reverse proxy or VPN.

" } pub(super) async fn login_submit( @@ -424,6 +449,94 @@ fn unavailable() -> Response { mod tests { use super::*; + fn test_server() -> ServerState { + ServerState { + api_key: None, + dashboard_enabled: true, + dashboard_public: false, + dashboard_theme: "default".to_owned(), + dashboard_refresh_interval_s: 60, + configured_server_threads: 1, + database_path: ":memory:".to_owned(), + trusted_proxy_ips: vec!["192.0.2.10".to_owned()], + allow_insecure_web_auth: false, + started_at: Instant::now(), + } + } + + #[test] + fn login_get_never_solicits_credentials_on_untrusted_remote_http() { + let server = test_server(); + let connection = eggserve_primitives::connection_info::ConnectionInfo::new( + None, + Some("198.51.100.20:443".parse().expect("peer address")), + eggserve_primitives::connection_info::Scheme::Http, + None, + ); + let mut headers = HeaderMap::new(); + headers.insert(header::HOST, HeaderValue::from_static("trusted.example")); + headers.insert("x-forwarded-proto", HeaderValue::from_static("https")); + let (status, html) = login_page_for_transport(Some(&connection), &headers, &server); + assert_eq!(status, StatusCode::FORBIDDEN); + assert!(!html.contains("type=\"password\"")); + assert!(!html.contains("name=\"username\"")); + assert!(html.contains("ssh -L 11300:127.0.0.1:11300")); + } + + #[test] + fn login_get_fails_closed_for_missing_peer_and_ambiguous_proxy_scheme() { + let server = test_server(); + let (status, html) = login_page_for_transport(None, &HeaderMap::new(), &server); + assert_eq!(status, StatusCode::FORBIDDEN); + assert!(!html.contains("type=\"password\"")); + + let proxy = eggserve_primitives::connection_info::ConnectionInfo::new( + None, + Some("192.0.2.10:443".parse().expect("proxy peer")), + eggserve_primitives::connection_info::Scheme::Http, + None, + ); + let mut headers = HeaderMap::new(); + headers.append("x-forwarded-proto", HeaderValue::from_static("https")); + headers.append("x-forwarded-proto", HeaderValue::from_static("https")); + let (status, html) = login_page_for_transport(Some(&proxy), &headers, &server); + assert_eq!(status, StatusCode::FORBIDDEN); + assert!(!html.contains("type=\"password\"")); + + headers.clear(); + headers.insert( + "x-forwarded-proto", + HeaderValue::from_bytes(b"https\xff").expect("opaque header value"), + ); + let (status, html) = login_page_for_transport(Some(&proxy), &headers, &server); + assert_eq!(status, StatusCode::FORBIDDEN); + assert!(!html.contains("type=\"password\"")); + } + + #[test] + fn login_get_shows_credentials_only_on_loopback_or_trusted_https_proxy() { + let server = test_server(); + let make_connection = |peer: &str| { + eggserve_primitives::connection_info::ConnectionInfo::new( + None, + Some(peer.parse().expect("peer address")), + eggserve_primitives::connection_info::Scheme::Http, + None, + ) + }; + let loopback = make_connection("127.0.0.1:3000"); + let (status, html) = login_page_for_transport(Some(&loopback), &HeaderMap::new(), &server); + assert_eq!(status, StatusCode::OK); + assert!(html.contains("type=\"password\"")); + + let proxy = make_connection("192.0.2.10:443"); + let mut headers = HeaderMap::new(); + headers.insert("x-forwarded-proto", HeaderValue::from_static("https")); + let (status, html) = login_page_for_transport(Some(&proxy), &headers, &server); + assert_eq!(status, StatusCode::OK); + assert!(html.contains("type=\"password\"")); + } + #[test] fn same_origin_requires_matching_authority_and_scheme() { let mut headers = HeaderMap::new(); From 0bf9c1130130910c71650ea3a5545d61038afc55 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 12:40:20 +0000 Subject: [PATCH 32/62] Start closing on-prem login plan 019 --- .../019-login-transport-solo-lan-corrective.md | 2 +- plans/registry.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md b/plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md index 9bad53e4..9f960b5c 100644 --- a/plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md +++ b/plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md @@ -1,6 +1,6 @@ # On-Prem User Access M019 — Login transport and solo LAN operability corrective -Status: active +Status: closing Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index 756a6885..8cf357a5 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -73,7 +73,7 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| -| On-prem user access | M019 — browser transport and solo LAN corrective | active | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | GET login form now shares POST transport validation; solo SSH tunnel and team HTTPS guidance in progress. | +| On-prem user access | M019 — browser transport and solo LAN corrective | closing | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | Transport matrix, solo SSH tunnel guidance, team HTTPS path, and full serial suites passed; closure audit in progress. | ## Blocked work From cdfc4f58bcceaefdde02913fd6becfb6403671ae Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 12:41:51 +0000 Subject: [PATCH 33/62] Close login transport plan and unblock headless admin --- .../closure/on-prem-user-access/019-status.md | 87 +++++++++++++++++++ ...019-login-transport-solo-lan-corrective.md | 2 +- .../020-managed-transition-headless-admin.md | 2 +- plans/registry.md | 8 +- .../subsystems/on-prem-user-access-roadmap.md | 10 +-- 5 files changed, 99 insertions(+), 10 deletions(-) create mode 100644 plans/closure/on-prem-user-access/019-status.md diff --git a/plans/closure/on-prem-user-access/019-status.md b/plans/closure/on-prem-user-access/019-status.md new file mode 100644 index 00000000..9c8f0ced --- /dev/null +++ b/plans/closure/on-prem-user-access/019-status.md @@ -0,0 +1,87 @@ +# On-Prem User Access M019 — Closure Status + +Status: closed + +Source implementation plan: + +- `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` + +Source subsystem roadmap: + +- `plans/subsystems/on-prem-user-access-roadmap.md#m019--login-transport-and-solo-lan-operability` + +Repository baseline reviewed: post-merge `88f58bbf`; implementation candidate +`b18a561` after M018 closure and M019 activation. + +## 1. Executive finding + +The login page now uses the same transport policy as login submission. Remote +plaintext HTTP receives a static 403 explanation with no username/password +controls and no database access. Loopback HTTP and HTTPS connections arriving +from a specifically configured proxy can show the form. Forwarded scheme +headers from other peers, duplicate/chained values, and malformed values do +not grant transport trust. Solo operators have an SSH loopback-tunnel path; +team browser administration uses an operator-managed HTTPS proxy or VPN. + +## 2. Requirement-to-evidence matrix + +| Requirement | Evidence | Result | Notes | +|---|---|---|---| +| Unsafe remote GET never solicits credentials | `login_get_never_solicits_credentials_on_untrusted_remote_http`; `login_get_fails_closed_for_missing_peer_and_ambiguous_proxy_scheme` | pass | Asserts forbidden response and absence of password/username fields. | +| Trust derives from connection and configured proxy, not client headers | `forwarded_https_is_accepted_only_from_the_configured_proxy_socket`; login GET transport tests | pass | Spoofed headers at an untrusted peer, duplicate scheme, malformed scheme, or absent peer are denied. | +| Loopback and trusted HTTPS proxy can reach the login form | `login_get_shows_credentials_only_on_loopback_or_trusted_https_proxy`; `managed_browser_login_account_csrf_and_logout_work_over_loopback` | pass | Existing session test exercises successful login and cookie behavior. | +| Existing browser session and CSRF behavior remains intact | `on_prem_sessions`; `on_prem_admin_ui`; dashboard parity tooling | pass | Session cookies, CSRF, account/admin flows, and role projections pass. | +| Solo and team operator paths are documented | `README.md`, `docs/raspberry-pi.md`, `docs/deployment.md`, `docs/firewall.md` | pass | Pi SSH forwarding, proxy trust/firewall constraints, and LAN inference separation are explicit. | + +## 3. Implementation evidence + +- `rust/src/server/auth.rs` gates `GET /login` with + `web_auth_transport_allowed`, returning a static help page when transport + is unsafe. The rejected branch only reads request/connection facts and + server configuration; it does not touch the database or set cookies. +- Login POST continues to use the same transport predicate and same-origin + validation. Existing secure-cookie and CSRF behavior is unchanged. +- The trusted proxy check binds forwarded HTTPS to the socket peer IP and + requires exactly one `X-Forwarded-Proto: https` value. +- Docs explain the explicit local SSH tunnel and operator-managed team TLS + proxy paths. No TLS listener, CA management, migration, or inference-path + change was introduced. + +## 4. Verification executed + +- `cargo test --manifest-path rust/Cargo.toml --lib server::auth::tests -- --test-threads=1` — 10 passed. +- `cargo test --manifest-path rust/Cargo.toml --test on_prem_sessions -- --test-threads=1` — 10 passed. +- `cargo test --manifest-path rust/Cargo.toml --test on_prem_admin_ui -- --test-threads=1` — 3 passed. +- `uv run pytest tests/tooling/test_dashboard_parity_on_prem.py -q` — 10 passed. +- Default serial workspace: 1,030 passed, 1 ignored across 79 suites. +- No-default serial workspace: 1,031 passed, 1 ignored across 79 suites. +- Formatting, strict default/no-default Clippy, no-default workspace check, release-doc validation, and `git diff --check` passed. +- These are local results. Hosted CI and a separately deployed reverse-proxy/browser session were not run in this closure pass. + +## 5. Invariant and recovery review + +An absent or unrecognized peer, a remote direct HTTP peer, and untrusted or +ambiguous forwarded metadata all fail closed. The loopback exception relies +on the actual socket peer address; `Host`, `Forwarded`, and +`X-Forwarded-Proto` cannot turn a remote direct connection into loopback or +trusted proxy transport. The existing explicit insecure development setting +retains its warning and is not the default. No schema or session state changes +were made. + +## 6. Unresolved findings + +| Severity | Finding | Impact | Required action | +|---|---|---|---| +| none | No unresolved M019 implementation finding | — | — | + +## 7. Roadmap and unblock disposition + +M019 is closed. The dependency audit searched the on-prem implementation +plans, roadmap graph, and ready/active/blocked registry tables. M020's hard +dependencies, M017 and M019, are both closed; M020 is promoted from blocked +to ready. M021 remains blocked on M020. M023 remains blocked on M022 and +ordinary-server operational evidence. M022 remains ready. No other plan is +promoted and no user-count, concurrency, or capacity claim is introduced. + +The registry and roadmap record M019 closed and M020 ready. Historical +M001–M018 closure records were not modified. diff --git a/plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md b/plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md index 9f960b5c..e7468c16 100644 --- a/plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md +++ b/plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md @@ -1,6 +1,6 @@ # On-Prem User Access M019 — Login transport and solo LAN operability corrective -Status: closing +Status: implemented Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md b/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md index 83a91d25..4e74227e 100644 --- a/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md +++ b/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md @@ -1,6 +1,6 @@ # On-Prem User Access M020 — Managed transition consistency and headless administration -Status: blocked +Status: ready Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index 8cf357a5..946cfb17 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -58,7 +58,7 @@ No new runtime, schema or performance claim lands with these plans. | Persistence | active | `plans/subsystems/persistence-roadmap.md` | M011 ready — post-M010 roadmap/documentation reconciliation | M009/M010 closures accepted. Documentation/tooling-only corrective; production remains WAL/NORMAL. No technical persistence successor is registered. | | Deployment and packaging | closed | `plans/subsystems/deployment-packaging-roadmap.md` | M005 closed — active PATH integration detection corrective | M001–M005 closed; no successor registered. | | Dashboard | closed | `plans/subsystems/dashboard-roadmap.md` | M012 closed — lifecycle closure and documentation polish | M001–M012 closed; no registered successor; future dashboard work requires a new bounded plan. | -| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M019/M022 ready — login transport and hard-budget compatibility | M017/M018 closed; M020/M021/M023 retain their hard dependencies. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | +| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M020/M022 ready — headless administration and hard-budget compatibility | M017–M019 closed; M021/M023 retain their hard dependencies. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | | On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 ready — ordinary-server small-team synthetic capacity baseline | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 is independent measurement on ordinary on-prem Linux with durable SSD/NVMe (operational target required), not an authorization to change WAL/SQLite topology. | ## Dependency-ready implementation plans @@ -67,20 +67,19 @@ No new runtime, schema or performance claim lands with these plans. |---|---|---|---|---| | Persistence | M011 — post-M010 roadmap and documentation reconciliation | ready | `plans/implementation/persistence/011-post-m010-roadmap-documentation-reconciliation.md` | Hard: M009/M010 closures accepted. Documentation/tooling only; no runtime or physical target dependency. | | On-prem user access | M022 — hard-budget agent interoperability | ready | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | Hard M017 is closed; preserve M008/M015 proof-or-reject guarantees. | +| On-prem user access | M020 — managed transition and headless administration | ready | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | Hard M017 and M019 closures accepted; add local member lifecycle commands. | | On-prem concurrency performance | M002 — ordinary-server team baseline | ready | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | Independent evidence work; ordinary Linux + durable SSD/NVMe required for team-capacity claims, no Pi/microSD release gate. | ## Active implementation plans | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| -| On-prem user access | M019 — browser transport and solo LAN corrective | closing | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | Transport matrix, solo SSH tunnel guidance, team HTTPS path, and full serial suites passed; closure audit in progress. | ## Blocked work | Subsystem | Milestone | Blocker | |---|---|---| -| On-prem user access | M020 — managed transition and headless administration | Hard: M017 and M019 closure. | | On-prem user access | M021 — default member profile and provisioning | Hard: M020 closure. | | On-prem user access | M023 — short-window team fairness | Hard: M022 closure; ordinary-server evidence operational. | @@ -114,6 +113,7 @@ Historical M007 blocker assessment at baseline `7e241ad` (`plans/closure/persist | Subsystem / plan | Disposition | Evidence | |---|---|---| +| On-prem user access M019 — login transport and solo LAN operability corrective | closed — remote plaintext login GET returns no credential form; loopback and configured HTTPS-proxy paths pass; solo SSH tunnel and team proxy guidance documented; M020 promoted to ready | `plans/closure/on-prem-user-access/019-status.md`, implementation `b18a561` | | On-prem user access M018 — local operator status corrective | closed — status and runtime status use bounded private control-socket readback without a server API key; managed HTTP authorization is unchanged; no successor unblocked | `plans/closure/on-prem-user-access/018-status.md`, implementation `8e2b66a`, CLI fixture `3be5d83` | | On-prem user access M017 — managed client credential reconciliation | closed — managed local config uses an explicitly supplied active personal token; keyless remote profiles remain read-only; client/helper suites and default/no-default serial workspace suites pass; M022 promoted to ready | `plans/closure/on-prem-user-access/017-status.md`, implementation `d726bce`, `218df45` | | On-prem user access M016 — final PR #7 integration and merge-readiness reconciliation | closed — local Rust/tooling, Q007, release-doc, browser and installed release-binary evidence pass; exact PR head `3d68c715` hosted CI, Linux/macOS installer and cargo-deny pass; merge held because no maintainer PR review has been submitted; no future plan depends on M016 | `plans/closure/on-prem-user-access/016-status.md`, PR #7, base `9a0a903c`, head `3d68c715`, CI `38721601336`, installer `38721601305`, audit `38721601318` | @@ -531,3 +531,5 @@ future separately scoped controlled-I/O/media/budget work; none is registered. Unblock audit (on-prem user access M017 closed, `plans/closure/on-prem-user-access/017-status.md`): searched `plans/implementation/on-prem-user-access/`, the roadmap dependency graph, and the dependency-ready, active, and blocked registry tables. M022's sole hard dependency, M017, is closed and its shared client/wire and hard-budget proof interfaces were already stable, so M022 moves `blocked` → `ready`. M020 remains blocked on M019; M021 remains blocked on M020; M023 remains blocked on M022 and its ordinary-server operational evidence. No other registered plan became eligible. M018 and M019 were already independent ready plans. No 49-user, team-capacity, or generated-output cap claim is introduced by this audit. Unblock audit (on-prem user access M018 closed, `plans/closure/on-prem-user-access/018-status.md`): searched the on-prem implementation plans, roadmap dependency graph, and dependency-ready, active, and blocked registry tables. No registered plan has a hard M018 dependency, so no blocked row is promoted. M019 and M022 remain ready; M020 remains blocked on M019, M021 on M020, and M023 on M022 plus ordinary-server operational evidence. No capacity or generated-output cap claim is introduced. + +Unblock audit (on-prem user access M019 closed, `plans/closure/on-prem-user-access/019-status.md`): searched the on-prem implementation plans, roadmap dependency graph, and dependency-ready, active, and blocked registry tables. M020's hard dependencies M017 and M019 are both closed, so M020 moves `blocked` → `ready`. M021 remains blocked on M020. M023 remains blocked on M022 and ordinary-server operational evidence. M022 remains ready; no other plan is promoted and no capacity claim is introduced. diff --git a/plans/subsystems/on-prem-user-access-roadmap.md b/plans/subsystems/on-prem-user-access-roadmap.md index 4de49586..27944f17 100644 --- a/plans/subsystems/on-prem-user-access-roadmap.md +++ b/plans/subsystems/on-prem-user-access-roadmap.md @@ -314,7 +314,7 @@ tokens browser/admin rights. Prefer existing owner-protected local control channel; new privilege decisions require ADR review. Source: `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md`. -#### M019 — Login transport and solo LAN operability +#### M019 — Login transport and solo LAN operability (closed) Class: invariant. Ready independently. Prevent remote plaintext `GET /login` rendering credential fields, verify proxy/CSRF invariants, @@ -324,7 +324,7 @@ Source: `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-c #### M020 — Managed transition and headless administration -Class: capability. Blocked on hard M017/M019 closure. Unify +Class: capability. Ready after hard M017/M019 closure. Unify `enable-managed` with member-create transition requirements and add local CLI create/disable/enable/show member workflows; preserve audit, CSRF, password-stdin, last-admin and atomic creation guarantees. @@ -356,7 +356,7 @@ Source: `plans/implementation/on-prem-user-access/023-short-window-fairness-cont ### Post-merge dependency and hardware policy -- M017 and M018 are closed. M019 and M022 are ready; neither depends on a Pi/Le +- M017–M019 are closed. M020 and M022 are ready; neither depends on a Pi/Le Potato run. M022 became ready after the M017 closure. - Hard: M017 + M019 -> M020 -> M021; M017 -> M022 -> M023. - Interface: M018's privileged local transport must not broaden HTTP rights; @@ -455,8 +455,8 @@ A ready plan is not an implemented capability. | M016 | closed | `plans/implementation/on-prem-user-access/016-final-pr7-merge-reconciliation.md` | `plans/closure/on-prem-user-access/016-status.md` | Exact candidate checks pass; merge held pending maintainer PR review. PR #7 subsequently merged at `88f58bbf`; historical M016 closure remains unchanged. | | M017 | closed | `plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md` | `plans/closure/on-prem-user-access/017-status.md` | Managed credential selection and keyless remote export; serial/default/no-default suites pass. | | M018 | closed | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | `plans/closure/on-prem-user-access/018-status.md` | Private local control readback restores status after identity cutover; no HTTP privilege change; no successor unblocked. | -| M019 | ready | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | — | Independent; insecure remote GET password form. | -| M020 | blocked | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | — | Hard: M017 and M019 closure. | +| M019 | closed | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | `plans/closure/on-prem-user-access/019-status.md` | Unsafe remote HTTP login GET returns no credential form; M020 unblocked. | +| M020 | ready | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | — | Hard dependencies M017 and M019 are closed. | | M021 | blocked | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | — | Hard: M020 closure. | | M022 | ready | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | — | Hard M017 closure is satisfied; preserve hard budget proof. | | M023 | blocked | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Hard: M022 closure; ordinary-server evidence operational. | From c8d8d0e961225d92afe351c89263a585cd483d47 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 12:42:04 +0000 Subject: [PATCH 34/62] Activate headless administration plan 020 --- .../020-managed-transition-headless-admin.md | 2 +- plans/registry.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md b/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md index 4e74227e..6ff51079 100644 --- a/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md +++ b/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md @@ -1,6 +1,6 @@ # On-Prem User Access M020 — Managed transition consistency and headless administration -Status: ready +Status: active Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index 946cfb17..e6fca20a 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -67,13 +67,13 @@ No new runtime, schema or performance claim lands with these plans. |---|---|---|---|---| | Persistence | M011 — post-M010 roadmap and documentation reconciliation | ready | `plans/implementation/persistence/011-post-m010-roadmap-documentation-reconciliation.md` | Hard: M009/M010 closures accepted. Documentation/tooling only; no runtime or physical target dependency. | | On-prem user access | M022 — hard-budget agent interoperability | ready | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | Hard M017 is closed; preserve M008/M015 proof-or-reject guarantees. | -| On-prem user access | M020 — managed transition and headless administration | ready | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | Hard M017 and M019 closures accepted; add local member lifecycle commands. | | On-prem concurrency performance | M002 — ordinary-server team baseline | ready | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | Independent evidence work; ordinary Linux + durable SSD/NVMe required for team-capacity claims, no Pi/microSD release gate. | ## Active implementation plans | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| +| On-prem user access | M020 — managed transition and headless administration | active | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | Hard M017/M019 satisfied; align managed transition and local member lifecycle commands. | ## Blocked work From 8a875a1b6ea2d3accca9783c6e24a601844aea9d Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 13:10:42 +0000 Subject: [PATCH 35/62] Implement headless member lifecycle and managed transition --- architecture/deep-dive-security.md | 12 + docs/cli-reference.md | 14 +- docs/deployment.md | 29 +- rust/src/cli.rs | 36 ++- rust/src/identity.rs | 137 +++++++-- rust/src/runtime.rs | 104 ++++++- rust/tests/cli_contract.rs | 32 ++ rust/tests/on_prem_admin_ui.rs | 11 +- rust/tests/on_prem_headless_admin.rs | 282 ++++++++++++++++++ rust/tests/on_prem_identity.rs | 82 +++++ rust/tests/on_prem_sessions.rs | 29 +- tests/fixtures/cli/contract-matrix.json | 2 +- .../test_on_prem_credential_guidance.py | 2 - 13 files changed, 712 insertions(+), 60 deletions(-) create mode 100644 rust/tests/on_prem_headless_admin.rs diff --git a/architecture/deep-dive-security.md b/architecture/deep-dive-security.md index db58fe53..d394fdc1 100644 --- a/architecture/deep-dive-security.md +++ b/architecture/deep-dive-security.md @@ -69,6 +69,18 @@ values fall back to the proxy peer. No other forwarded header establishes trust. `security.allow_insecure_web_auth` is an explicit development override and emits a startup warning. + +The local operator CLI uses the data-directory owner's filesystem authority. +`users add` commits a member, optional initial profile, the transition from +`admin_initialized` to `managed`, and audit entries in one transaction; a +failed profile assignment leaves the original state intact. The explicit +`users enable-managed` path uses the same transition helper and records its +own audit event. A remaining configured shared key is rejected after +identity initialization, and `dashboard.public` cannot expose managed routes; +removing those stale config values is cleanup, not a precondition for the +atomic cutover. Disabling a user revokes sessions and tokens and advances the +auth epoch. Re-enabling does not un-revoke credentials, and the last active +administrator remains protected. Session-authenticated responses carry `Cache-Control: no-store, max-age=0` so account and administration HTML containing identity, budget, or CSRF data is not retained by shared or browser caches. Privileged browser mutations also diff --git a/docs/cli-reference.md b/docs/cli-reference.md index c94a9e42..bfd0dfc8 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -55,11 +55,15 @@ Command list verified against `eggpool help` (0.8.2). | `eggpool getkey` | Print the server API key | | `eggpool newkey` | Generate and write a new server API key (`--show-old`, `--show-secrets`) | | `eggpool users bootstrap` | Create the initial local administrator (`--password-stdin` for unattended input) | +| `eggpool users add USER` | Create a member with a TTY-hidden or `--password-stdin` temporary password; optional `--profile-id PROFILE`; `--actor ADMIN` records the active administrator | +| `eggpool users show USER` | Show one user's role, state and forced-password-change status without secrets | | `eggpool users list` | List local usernames, role and status | +| `eggpool users disable USER` | Disable a user and revoke their sessions and personal tokens | +| `eggpool users enable USER` | Reactivate a user; previously revoked tokens remain revoked | | `eggpool users token USER` | Issue a personal inference/discovery token (`--label` optional; `--show-secret` allows non-TTY output) | | `eggpool users revoke USER` | Revoke all personal API tokens for a user | | `eggpool users reset-password USER` | Reset a local password and revoke that user's API tokens and browser sessions (`--password-stdin` supported) | -| `eggpool users enable-managed` | Retire the legacy key path after clearing `[server].api_key`/`api_key_env` and setting `dashboard.public = false` | +| `eggpool users enable-managed` | Explicitly record managed access (`--actor ADMIN`, default `admin`); works even when retired legacy settings remain | | `eggpool users policy create ID NAME allow\|deny` | Create an exact-rule model profile | | `eggpool users policy rule PROFILE MODEL allow\|deny` | Add or replace one exact model or `model/provider` rule | | `eggpool users policy assign USER PROFILE` | Assign a profile to a local user | @@ -77,6 +81,14 @@ and wildcards are rejected. `--actor ADMIN` records the active administrator responsible for each change. Restricted profiles should use `deny` as their default and add explicit allowed models. +Local member lifecycle commands run from the machine that owns the EggPool +data directory. The first successful `users add` moves +`admin_initialized` to `managed` in the same SQLite transaction as the user, +optional initial profile, and audit records. A failed profile assignment does +not leave a user or change the access state. Disabling revokes active sessions +and personal tokens; re-enabling does not restore those tokens, so issue a new +one with `users token USER`. The last active administrator cannot be disabled. + Budget commands are local operator controls. At least one positive limit is required; each rule is keyed by user, optional exact model, and UTC calendar window. Cost limits apply to estimated microdollars, not provider invoices. diff --git a/docs/deployment.md b/docs/deployment.md index 41f55637..1ce868c7 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -56,12 +56,16 @@ authentication behavior until an operator explicitly initializes local user access. Manage local identities from the machine that owns the data directory with -`eggpool users list`, `eggpool users token USER`, `eggpool users revoke USER`, -and `eggpool users reset-password USER`. Personal tokens are for inference, +`eggpool users add USER --password-stdin`, `eggpool users show USER`, +`eggpool users disable USER`, `eggpool users enable USER`, +`eggpool users token USER`, `eggpool users revoke USER`, and +`eggpool users reset-password USER`. Member passwords are read from a hidden TTY prompt or stdin; passwords +are never accepted in argv. Add `--profile-id PROFILE` to assign the initial +model policy in the same transaction. Personal tokens are for inference, model discovery, and the integration profile; they do not authorize dashboard or admin pages. The configured server API key remains a legacy shared -inference credential during the staged migration. Do not share it with new -users; issue an individual token instead. +inference credential only while the database is `legacy_unmigrated`. Do not +share it with new users; issue an individual token instead. With `[dashboard].enabled = true`, an administrator can manage users, profiles, budgets, tokens, and audit history in the embedded dashboard. New members @@ -436,12 +440,17 @@ After the first local administrator is created, the shared key is no longer accepted; personal tokens authenticate agent surfaces, while dashboard and admin routes require a browser session regardless of `dashboard.public`. -For managed operation, remove `[server].api_key` and unset its configured -environment variable, then set `dashboard.public = false` and run -`eggpool users enable-managed`. That command refuses the transition while the -shared key or public-dashboard setting remains. Non-loopback listeners may -then run without a shared server key because protected routes require a -personal token or browser session. +The first successful `eggpool users add USER` moves an initialized instance +from `admin_initialized` to `managed` atomically with member creation, profile +assignment, and audit. `eggpool users enable-managed` records that transition +explicitly and requires an active local administrator. Both paths apply the +same database transition rules. After identity initialization, EggPool +rejects the configured legacy shared key and ignores `dashboard.public` for +privileged routes. Remove `[server].api_key`/`api_key_env` and set +`dashboard.public = false` as configuration hygiene; stale values do not +restore legacy access or block the identity cutover. Non-loopback listeners +can run without a shared key because protected routes require a personal token +or browser session. Browser password login is allowed over loopback HTTP. For remote browser use, terminate TLS at an operator-managed proxy and add the proxy's literal peer IP diff --git a/rust/src/cli.rs b/rust/src/cli.rs index c5f0b706..0445b5b6 100644 --- a/rust/src/cli.rs +++ b/rust/src/cli.rs @@ -145,6 +145,14 @@ pub struct OnboardArgs { pub enum UsersCommand { /// Create the first administrator and personal inference credential. Bootstrap(UserPasswordArgs), + /// Create a member with a temporary password and optional initial profile. + Add(UserAddArgs), + /// Disable a user, revoking sessions and personal credentials. + Disable(UserLifecycleArgs), + /// Reactivate a disabled user; issue a new personal token separately. + Enable(UserLifecycleArgs), + /// Show one local user's non-secret status. + Show(UserNameArgs), /// Show local account names and status without secrets. List, /// Reset a user's password and revoke their API credentials. @@ -154,7 +162,7 @@ pub enum UsersCommand { /// Revoke every API credential owned by one user. Revoke(UserNameArgs), /// Retire legacy access after public dashboard access is disabled. - EnableManaged, + EnableManaged(UserActorArgs), /// Create and assign model-policy profiles during the CLI-only M004 phase. #[command(subcommand)] Policy(UsersPolicyCommand), @@ -308,6 +316,32 @@ pub struct UserPasswordArgs { pub show_secret: bool, } +#[derive(Debug, clap::Args)] +pub struct UserAddArgs { + pub username: String, + /// Read the temporary password from stdin instead of prompting on a TTY. + #[arg(long)] + pub password_stdin: bool, + /// Assign an existing model-policy profile in the same transaction. + #[arg(long)] + pub profile_id: Option, + #[arg(long, default_value = "admin")] + pub actor: String, +} + +#[derive(Debug, clap::Args)] +pub struct UserLifecycleArgs { + pub username: String, + #[arg(long, default_value = "admin")] + pub actor: String, +} + +#[derive(Debug, clap::Args)] +pub struct UserActorArgs { + #[arg(long, default_value = "admin")] + pub actor: String, +} + #[derive(Debug, clap::Args)] pub struct UserTokenArgs { pub username: String, diff --git a/rust/src/identity.rs b/rust/src/identity.rs index 28f32212..61b4e78c 100644 --- a/rust/src/identity.rs +++ b/rust/src/identity.rs @@ -331,6 +331,34 @@ pub async fn list_users(database: &Database) -> Result, Identit .map_err(IdentityError::Database) } +pub async fn get_user_by_name( + database: &Database, + username: &str, +) -> Result, IdentityError> { + let username = username.to_owned(); + database + .call(move |connection| { + connection + .query_row( + "SELECT id, username, role, state, must_change_password FROM users \ + WHERE normalized_username = ?1", + [username.to_ascii_lowercase()], + |row| { + Ok(UserSummary { + id: row.get(0)?, + username: row.get(1)?, + role: row.get(2)?, + state: row.get(3)?, + must_change_password: row.get::<_, i64>(4)? != 0, + }) + }, + ) + .optional() + }) + .await + .map_err(IdentityError::Database) +} + /// Create a member with a one-time temporary password. The forced password /// change, managed-mode transition, and administrator audit row commit with /// the new user. @@ -343,6 +371,24 @@ pub async fn create_member( create_member_with_fence(database, actor_id, username, temporary_password, None, None).await } +pub async fn create_member_with_profile( + database: &Database, + actor_id: &str, + username: &str, + temporary_password: &str, + profile_id: Option<&str>, +) -> Result { + create_member_with_fence( + database, + actor_id, + username, + temporary_password, + None, + profile_id, + ) + .await +} + pub async fn create_member_from_browser( database: &Database, principal: &crate::web_session::Principal, @@ -415,16 +461,22 @@ async fn create_member_with_fence( if taken { return Ok(AdminMutation::Conflict); } + let transition = transition_to_managed(connection)?; + if transition == ManagedTransition::Invalid { + return Ok(AdminMutation::InvalidTransition); + } + if transition == ManagedTransition::Changed { + connection.execute( + "INSERT INTO user_admin_audit(actor_id, action, target_kind, target_id) \ + VALUES (?1, 'enable_managed', 'instance', 'singleton')", + [&actor_id], + )?; + } connection.execute( "INSERT INTO users(id, username, normalized_username, role, password_hash, must_change_password) \ VALUES (?1, ?2, ?3, 'member', ?4, 1)", (&id, &username, &normalized, &password_hash), )?; - connection.execute( - "UPDATE user_access_state SET state = 'managed', updated_at = CURRENT_TIMESTAMP \ - WHERE singleton = 1 AND state IN ('admin_initialized', 'managed')", - [], - )?; connection.execute( "INSERT INTO user_admin_audit(actor_id, action, target_kind, target_id) \ VALUES (?1, 'create_user', 'user', ?2)", @@ -455,6 +507,7 @@ async fn create_member_with_fence( AdminMutation::User(user) => Ok(user), AdminMutation::Forbidden => Err(IdentityError::AdministratorRequired), AdminMutation::Conflict => Err(IdentityError::UsernameConflict), + AdminMutation::InvalidTransition => Err(IdentityError::InvalidAccessTransition), _ => Err(IdentityError::UserNotFound), } } @@ -763,6 +816,38 @@ enum AdminMutation { LastAdmin, Missing, Conflict, + InvalidTransition, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum ManagedTransition { + Changed, + AlreadyManaged, + Invalid, +} + +fn transition_to_managed( + connection: &tokio_rusqlite::rusqlite::Connection, +) -> Result { + let changed = connection.execute( + "UPDATE user_access_state SET state = 'managed', updated_at = CURRENT_TIMESTAMP \ + WHERE singleton = 1 AND state = 'admin_initialized' \ + AND EXISTS (SELECT 1 FROM users WHERE role = 'admin' AND state = 'active')", + [], + )?; + if changed == 1 { + return Ok(ManagedTransition::Changed); + } + let state: String = connection.query_row( + "SELECT state FROM user_access_state WHERE singleton = 1", + [], + |row| row.get(0), + )?; + Ok(if state == "managed" { + ManagedTransition::AlreadyManaged + } else { + ManagedTransition::Invalid + }) } pub async fn admin_audit( @@ -970,22 +1055,38 @@ pub async fn access_state(database: &Database) -> Result .map_err(IdentityError::Database) } -pub async fn enable_managed(database: &Database) -> Result<(), IdentityError> { - let changed = database - .with_transaction(|connection| { - connection.execute( - "UPDATE user_access_state SET state = 'managed', updated_at = CURRENT_TIMESTAMP \ - WHERE singleton = 1 AND state = 'admin_initialized' \ - AND EXISTS (SELECT 1 FROM users WHERE role = 'admin' AND state = 'active')", - [], +pub async fn enable_managed(database: &Database, actor_id: &str) -> Result<(), IdentityError> { + let actor_id = actor_id.to_owned(); + let outcome = database + .with_transaction(move |connection| { + let actor_is_admin: bool = connection.query_row( + "SELECT EXISTS(SELECT 1 FROM users WHERE id = ?1 AND role = 'admin' AND state = 'active')", + [&actor_id], + |row| row.get(0), )?; - Ok(connection.changes()) + if !actor_is_admin { + return Ok(AdminMutation::Forbidden); + } + let transition = transition_to_managed(connection)?; + match transition { + ManagedTransition::Changed => { + connection.execute( + "INSERT INTO user_admin_audit(actor_id, action, target_kind, target_id) \ + VALUES (?1, 'enable_managed', 'instance', 'singleton')", + [&actor_id], + )?; + Ok(AdminMutation::Done) + } + ManagedTransition::AlreadyManaged => Ok(AdminMutation::Done), + ManagedTransition::Invalid => Ok(AdminMutation::InvalidTransition), + } }) .await?; - if changed == 1 || access_state(database).await?.as_str() == "managed" { - Ok(()) - } else { - Err(IdentityError::InvalidAccessTransition) + match outcome { + AdminMutation::Done => Ok(()), + AdminMutation::Forbidden => Err(IdentityError::AdministratorRequired), + AdminMutation::InvalidTransition => Err(IdentityError::InvalidAccessTransition), + _ => Err(IdentityError::InvalidAccessTransition), } } diff --git a/rust/src/runtime.rs b/rust/src/runtime.rs index e9b1ee5d..3f8c776a 100644 --- a/rust/src/runtime.rs +++ b/rust/src/runtime.rs @@ -681,21 +681,15 @@ async fn open_operational_database( async fn users(path: &Path, command: crate::cli::UsersCommand) -> Result<(), BootstrapError> { let config = config::Config::from_toml(path)?; - if matches!(&command, crate::cli::UsersCommand::EnableManaged) - && (config.resolved_server_api_key().is_some() || config.dashboard.public) - { - return Err(command_error( - EXIT_VALIDATION, - "retire server.api_key/api_key_env and set dashboard.public = false before enabling managed access", - )); - } let database = open_operational_database(&config, false).await?; match command { - crate::cli::UsersCommand::EnableManaged => { - crate::identity::enable_managed(&database) + crate::cli::UsersCommand::EnableManaged(args) => { + let actor = policy_actor_id(&database, &args.actor).await?; + crate::identity::enable_managed(&database, &actor) .await .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))?; println!("Managed browser and personal-token access is enabled."); + warn_managed_config_hygiene(&config); } crate::cli::UsersCommand::List => { let users = crate::identity::list_users(&database) @@ -713,7 +707,7 @@ async fn users(path: &Path, command: crate::cli::UsersCommand) -> Result<(), Boo } } crate::cli::UsersCommand::Bootstrap(args) => { - let password = read_user_password(args.password_stdin, true)?; + let password = read_user_password(args.password_stdin, true, "Administrator password")?; let (user, credential) = crate::identity::bootstrap_admin(&database, &args.username, &password) .await @@ -721,8 +715,69 @@ async fn users(path: &Path, command: crate::cli::UsersCommand) -> Result<(), Boo println!("Administrator {} created.", user.username); print_one_time_secret(&credential.token, args.show_secret); } + crate::cli::UsersCommand::Add(args) => { + let actor = policy_actor_id(&database, &args.actor).await?; + let password = + read_user_password(args.password_stdin, false, "Temporary member password")?; + let user = crate::identity::create_member_with_profile( + &database, + &actor, + &args.username, + &password, + args.profile_id.as_deref(), + ) + .await + .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))?; + println!( + "Member {} created; the user must change the temporary password at first login.", + user.username + ); + println!( + "Issue a personal inference token with `eggpool users token {}`.", + user.username + ); + warn_managed_config_hygiene(&config); + } + crate::cli::UsersCommand::Disable(args) => { + let actor = policy_actor_id(&database, &args.actor).await?; + let user = crate::identity::get_user_by_name(&database, &args.username) + .await + .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))? + .ok_or_else(|| command_error(EXIT_VALIDATION, "user was not found"))?; + crate::identity::set_user_enabled(&database, &actor, &user.id, false) + .await + .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))?; + println!( + "User {} disabled; browser sessions and personal tokens were revoked.", + user.username + ); + } + crate::cli::UsersCommand::Enable(args) => { + let actor = policy_actor_id(&database, &args.actor).await?; + let user = crate::identity::get_user_by_name(&database, &args.username) + .await + .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))? + .ok_or_else(|| command_error(EXIT_VALIDATION, "user was not found"))?; + crate::identity::set_user_enabled(&database, &actor, &user.id, true) + .await + .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))?; + println!( + "User {} enabled. Previously revoked credentials stay revoked; issue a new token with `eggpool users token {}`.", + user.username, user.username + ); + } + crate::cli::UsersCommand::Show(args) => { + let user = crate::identity::get_user_by_name(&database, &args.username) + .await + .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))? + .ok_or_else(|| command_error(EXIT_VALIDATION, "user was not found"))?; + println!("username: {}", user.username); + println!("role: {}", user.role); + println!("state: {}", user.state); + println!("must_change_password: {}", user.must_change_password); + } crate::cli::UsersCommand::ResetPassword(args) => { - let password = read_user_password(args.password_stdin, true)?; + let password = read_user_password(args.password_stdin, true, "Administrator password")?; crate::identity::reset_password(&database, &args.username, &password) .await .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))?; @@ -939,7 +994,11 @@ async fn policy_actor_id( .ok_or_else(|| command_error(EXIT_VALIDATION, "active administrator was not found")) } -fn read_user_password(from_stdin: bool, confirm: bool) -> Result { +fn read_user_password( + from_stdin: bool, + confirm: bool, + prompt: &str, +) -> Result { let password = if from_stdin { let mut value = String::new(); io::stdin() @@ -955,10 +1014,11 @@ fn read_user_password(from_stdin: bool, confirm: bool) -> Result Result Result<(), Boots }; let admin_password = if bootstrap_admin { println!("Create the local administrator for this new installation."); - Some(read_user_password(args.password_stdin, true)?) + Some(read_user_password( + args.password_stdin, + true, + "Administrator password", + )?) } else { None }; diff --git a/rust/tests/cli_contract.rs b/rust/tests/cli_contract.rs index 47a93941..0edc5731 100644 --- a/rust/tests/cli_contract.rs +++ b/rust/tests/cli_contract.rs @@ -323,3 +323,35 @@ fn local_user_budget_commands_parse_with_bounded_windows() { .is_err() ); } + +#[test] +fn headless_member_lifecycle_commands_parse_with_secret_safe_inputs() { + assert!( + Cli::try_parse_from([ + "eggpool", + "users", + "add", + "member-a", + "--password-stdin", + "--profile-id", + "restricted", + "--actor", + "admin", + ]) + .is_ok() + ); + assert!(Cli::try_parse_from(["eggpool", "users", "disable", "member-a"]).is_ok()); + assert!(Cli::try_parse_from(["eggpool", "users", "enable", "member-a"]).is_ok()); + assert!(Cli::try_parse_from(["eggpool", "users", "show", "member-a"]).is_ok()); + assert!( + Cli::try_parse_from([ + "eggpool", + "users", + "add", + "member-a", + "--password", + "plaintext-in-argv", + ]) + .is_err() + ); +} diff --git a/rust/tests/on_prem_admin_ui.rs b/rust/tests/on_prem_admin_ui.rs index 941b9f15..b19be909 100644 --- a/rust/tests/on_prem_admin_ui.rs +++ b/rust/tests/on_prem_admin_ui.rs @@ -208,7 +208,9 @@ async fn two_user_browser_journey_fences_admin_data_and_personal_credentials() { identity::bootstrap_admin(&database, "admin", "M006 strong admin password passphrase") .await .unwrap(); - identity::enable_managed(&database).await.unwrap(); + identity::enable_managed(&database, &admin.id) + .await + .unwrap(); let process = ProcessRuntime::new(database.clone()).expect("process runtime"); let mut config = Config::default(); @@ -694,10 +696,13 @@ async fn managed_server( async fn browser_sign_in_accepts_the_origin_and_referer_pair_a_real_browser_sends() { let (_directory, database) = database().await; identity::mark_fresh_uninitialized(&database).await.unwrap(); - identity::bootstrap_admin(&database, "admin", "M011 strong admin password passphrase") + let (admin, _) = + identity::bootstrap_admin(&database, "admin", "M011 strong admin password passphrase") + .await + .unwrap(); + identity::enable_managed(&database, &admin.id) .await .unwrap(); - identity::enable_managed(&database).await.unwrap(); let (address, handle, task) = managed_server(&database).await; let origin = format!("http://{address}"); let form = "username=admin&password=M011+strong+admin+password+passphrase"; diff --git a/rust/tests/on_prem_headless_admin.rs b/rust/tests/on_prem_headless_admin.rs new file mode 100644 index 00000000..8fa883ae --- /dev/null +++ b/rust/tests/on_prem_headless_admin.rs @@ -0,0 +1,282 @@ +use std::{fs, process::Stdio}; + +use eggpool::{ + Config, + db::{Database, DatabaseConfig, MigrationRunner}, + identity::{self, access_state}, + model_policy::ModelPolicyService, +}; +use tempfile::tempdir; +use tokio::{io::AsyncWriteExt, process::Command}; + +async fn run_cli( + config_path: &std::path::Path, + args: &[&str], + password: Option<&str>, +) -> std::process::Output { + let mut child = Command::new(env!("CARGO_BIN_EXE_eggpool")) + .args(["--config", config_path.to_str().expect("config path")]) + .args(args) + .env("EGGPOOL_CONFIG", config_path) + .env_remove("EGGPOOL_API_KEY") + .stdin(if password.is_some() { + Stdio::piped() + } else { + Stdio::null() + }) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .expect("CLI starts"); + if let Some(password) = password { + child + .stdin + .take() + .expect("stdin pipe") + .write_all(format!("{password}\n").as_bytes()) + .await + .expect("password is sent through stdin"); + } + child.wait_with_output().await.expect("CLI completes") +} + +async fn open_database(path: &std::path::Path) -> Database { + let config = DatabaseConfig { + path: path.display().to_string(), + ..DatabaseConfig::default() + }; + Database::open(config).await.expect("database opens") +} + +#[tokio::test] +async fn headless_member_lifecycle_is_atomic_audited_and_works_with_stale_config() { + let directory = tempdir().expect("temporary directory"); + let database_path = directory.path().join("eggpool.sqlite3"); + let config_path = directory.path().join("config.toml"); + let mut config = Config::default(); + config.database.path = database_path.display().to_string(); + config.server.api_key = Some("legacy-shared-key-for-test-only".to_owned()); + config.dashboard.public = true; + fs::write( + &config_path, + toml::to_string(&config).expect("config serializes"), + ) + .expect("config file is written"); + + let database = open_database(&database_path).await; + MigrationRunner::new(&database) + .run() + .await + .expect("schema migrates"); + identity::mark_fresh_uninitialized(&database) + .await + .expect("fresh state is marked"); + let (admin, admin_credential) = + identity::bootstrap_admin(&database, "admin", "M020 strong admin password passphrase") + .await + .expect("first administrator bootstraps"); + ModelPolicyService::new(database.clone()) + .create_profile("restricted", "Restricted", "deny", &admin.id) + .await + .expect("profile is created"); + database.close().await.expect("database closes before CLI"); + + let invalid_profile = run_cli( + &config_path, + &[ + "users", + "add", + "member", + "--password-stdin", + "--profile-id", + "missing-profile", + ], + Some("M020 temporary member password"), + ) + .await; + assert!(!invalid_profile.status.success()); + let database = open_database(&database_path).await; + assert_eq!(access_state(&database).await.unwrap(), "admin_initialized"); + assert_eq!(identity::list_users(&database).await.unwrap().len(), 1); + assert!( + identity::verify_api_token(&database, &admin_credential.token) + .await + .unwrap() + ); + let failed_transition_audit = identity::admin_audit(&database, 20).await.unwrap(); + assert!(!failed_transition_audit.iter().any(|entry| { + matches!( + entry.action.as_str(), + "enable_managed" | "create_user" | "assign_profile" + ) + })); + database.close().await.expect("database closes"); + + let created = run_cli( + &config_path, + &[ + "users", + "add", + "member", + "--password-stdin", + "--profile-id", + "restricted", + ], + Some("M020 temporary member password"), + ) + .await; + assert!( + created.status.success(), + "{}", + String::from_utf8_lossy(&created.stderr) + ); + assert!(!String::from_utf8_lossy(&created.stdout).contains("M020 temporary member password")); + assert!(String::from_utf8_lossy(&created.stderr).contains("configuration hygiene")); + + let shown = run_cli(&config_path, &["users", "show", "member"], None).await; + assert!(shown.status.success()); + let shown_stdout = String::from_utf8_lossy(&shown.stdout); + assert!(shown_stdout.contains("role: member")); + assert!(shown_stdout.contains("state: active")); + assert!(shown_stdout.contains("must_change_password: true")); + assert!(!shown_stdout.contains("password_hash")); + + let token = run_cli( + &config_path, + &["users", "token", "member", "--show-secret"], + None, + ) + .await; + assert!(token.status.success()); + let token_stdout = String::from_utf8_lossy(&token.stdout); + let personal_token = token_stdout + .split_whitespace() + .find(|value| value.starts_with("epk1_")) + .expect("one-time member token is printed") + .to_owned(); + + let disabled = run_cli(&config_path, &["users", "disable", "member"], None).await; + assert!(disabled.status.success()); + let enabled = run_cli(&config_path, &["users", "enable", "member"], None).await; + assert!(enabled.status.success()); + assert!( + String::from_utf8_lossy(&enabled.stdout) + .contains("Previously revoked credentials stay revoked") + ); + + let duplicate = run_cli( + &config_path, + &["users", "add", "member", "--password-stdin"], + Some("M020 another member password"), + ) + .await; + assert!(!duplicate.status.success()); + let last_admin = run_cli(&config_path, &["users", "disable", "admin"], None).await; + assert!(!last_admin.status.success()); + + let idempotent_transition = run_cli(&config_path, &["users", "enable-managed"], None).await; + assert!( + idempotent_transition.status.success(), + "{}", + String::from_utf8_lossy(&idempotent_transition.stderr) + ); + + let database = open_database(&database_path).await; + assert_eq!(access_state(&database).await.unwrap(), "managed"); + let member = identity::get_user_by_name(&database, "member") + .await + .unwrap() + .expect("member is present"); + assert_eq!(member.state, "active"); + assert!(member.must_change_password); + assert!( + !identity::verify_api_token(&database, &personal_token) + .await + .unwrap(), + "disabled credentials remain revoked after reactivation" + ); + let (assigned_profile, audit_actions) = database + .call(move |connection| { + let profile: String = connection.query_row( + "SELECT profile_id FROM user_profile_assignments WHERE user_id = ?1", + [&member.id], + |row| row.get(0), + )?; + let mut statement = + connection.prepare("SELECT action FROM user_admin_audit ORDER BY id")?; + let actions = statement + .query_map([], |row| row.get::<_, String>(0))? + .collect::, _>>()?; + Ok((profile, actions)) + }) + .await + .expect("profile and audit evidence are readable"); + assert_eq!(assigned_profile, "restricted"); + assert!( + audit_actions + .iter() + .any(|action| action == "enable_managed") + ); + assert!(audit_actions.iter().any(|action| action == "create_user")); + assert!( + audit_actions + .iter() + .any(|action| action == "assign_profile") + ); + assert!(audit_actions.iter().any(|action| action == "disable_user")); + assert!( + audit_actions + .iter() + .any(|action| action == "reactivate_user") + ); + database.close().await.expect("database closes"); +} + +#[tokio::test] +async fn enable_managed_uses_the_same_cutover_with_stale_config_settings() { + let directory = tempdir().expect("temporary directory"); + let database_path = directory.path().join("eggpool.sqlite3"); + let config_path = directory.path().join("config.toml"); + let mut config = Config::default(); + config.database.path = database_path.display().to_string(); + config.server.api_key = Some("legacy-shared-key-for-test-only".to_owned()); + config.dashboard.public = true; + fs::write( + &config_path, + toml::to_string(&config).expect("config serializes"), + ) + .expect("config file is written"); + + let database = open_database(&database_path).await; + MigrationRunner::new(&database) + .run() + .await + .expect("schema migrates"); + identity::mark_fresh_uninitialized(&database) + .await + .expect("fresh state is marked"); + identity::bootstrap_admin(&database, "owner", "M020 strong admin password passphrase") + .await + .expect("administrator bootstraps"); + database.close().await.expect("database closes before CLI"); + + let output = run_cli( + &config_path, + &["users", "enable-managed", "--actor", "owner"], + None, + ) + .await; + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + assert!(String::from_utf8_lossy(&output.stderr).contains("configuration hygiene")); + + let database = open_database(&database_path).await; + assert_eq!(access_state(&database).await.unwrap(), "managed"); + let audit = identity::admin_audit(&database, 10).await.unwrap(); + assert_eq!(audit.len(), 1); + assert_eq!(audit[0].action, "enable_managed"); + database.close().await.expect("database closes"); +} diff --git a/rust/tests/on_prem_identity.rs b/rust/tests/on_prem_identity.rs index 24eff5c8..1771b264 100644 --- a/rust/tests/on_prem_identity.rs +++ b/rust/tests/on_prem_identity.rs @@ -79,6 +79,88 @@ async fn bootstrap_is_atomic_and_personal_token_is_one_way() { database.close().await.unwrap(); } +#[tokio::test] +async fn managed_transition_requires_an_active_admin_and_audits_once() { + let database = database().await; + identity::mark_fresh_uninitialized(&database).await.unwrap(); + let (admin, _) = identity::bootstrap_admin(&database, "admin", "M020 strong admin password") + .await + .unwrap(); + + assert!(matches!( + identity::enable_managed(&database, "missing-admin").await, + Err(IdentityError::AdministratorRequired) + )); + assert_eq!( + identity::access_state(&database).await.unwrap(), + "admin_initialized" + ); + + identity::enable_managed(&database, &admin.id) + .await + .unwrap(); + identity::enable_managed(&database, &admin.id) + .await + .unwrap(); + assert_eq!(identity::access_state(&database).await.unwrap(), "managed"); + let audit = identity::admin_audit(&database, 10).await.unwrap(); + assert_eq!( + audit + .iter() + .filter(|entry| entry.action == "enable_managed") + .count(), + 1 + ); + database.close().await.unwrap(); +} + +#[tokio::test] +async fn concurrent_first_member_creation_commits_one_transition_and_one_user() { + let database = database().await; + identity::mark_fresh_uninitialized(&database).await.unwrap(); + let (admin, _) = identity::bootstrap_admin(&database, "admin", "M020 strong admin password") + .await + .unwrap(); + + let (first, second) = tokio::join!( + identity::create_member( + &database, + &admin.id, + "ConcurrentMember", + "M020 concurrent member password A" + ), + identity::create_member( + &database, + &admin.id, + "concurrentmember", + "M020 concurrent member password B" + ) + ); + assert!(matches!( + (&first, &second), + (Ok(_), Err(IdentityError::UsernameConflict)) + | (Err(IdentityError::UsernameConflict), Ok(_)) + )); + assert_eq!(identity::access_state(&database).await.unwrap(), "managed"); + assert_eq!(identity::list_users(&database).await.unwrap().len(), 2); + let audit = identity::admin_audit(&database, 20).await.unwrap(); + assert_eq!( + audit + .iter() + .filter(|entry| entry.action == "enable_managed") + .count(), + 1 + ); + assert_eq!( + audit + .iter() + .filter(|entry| entry.action == "create_user") + .count(), + 1 + ); + database.close().await.unwrap(); +} + #[tokio::test] async fn simultaneous_bootstrap_commits_only_one_admin() { let database = database().await; diff --git a/rust/tests/on_prem_sessions.rs b/rust/tests/on_prem_sessions.rs index 8844edc1..39ef058e 100644 --- a/rust/tests/on_prem_sessions.rs +++ b/rust/tests/on_prem_sessions.rs @@ -98,7 +98,9 @@ async fn password_reset_revokes_browser_sessions() { async fn revoked_admin_session_cannot_commit_privileged_identity_or_budget_writes() { let (_directory, database) = database().await; let (admin_id, _) = create_admin(&database).await; - identity::enable_managed(&database).await.unwrap(); + identity::enable_managed(&database, &admin_id) + .await + .unwrap(); let issued = web_session::issue(&database, &admin_id).await.unwrap(); let principal = web_session::authenticate(&database, &issued.session_token) .await @@ -357,8 +359,8 @@ fn cookie(response: &str, name: &str) -> String { #[tokio::test] async fn managed_browser_login_account_csrf_and_logout_work_over_loopback() { let (_directory, database) = database().await; - let (_user_id, agent_token) = create_admin(&database).await; - identity::enable_managed(&database).await.unwrap(); + let (user_id, agent_token) = create_admin(&database).await; + identity::enable_managed(&database, &user_id).await.unwrap(); let process = ProcessRuntime::new(database.clone()).expect("process runtime"); let mut config = Config::default(); @@ -508,8 +510,8 @@ async fn managed_browser_login_account_csrf_and_logout_work_over_loopback() { #[tokio::test] async fn trusted_reverse_proxy_can_complete_secure_browser_login() { let (_directory, database) = database().await; - let (_user_id, _) = create_admin(&database).await; - identity::enable_managed(&database).await.unwrap(); + let (user_id, _) = create_admin(&database).await; + identity::enable_managed(&database, &user_id).await.unwrap(); let process = ProcessRuntime::new(database.clone()).expect("process runtime"); let mut config = Config::default(); @@ -555,7 +557,9 @@ async fn trusted_reverse_proxy_can_complete_secure_browser_login() { async fn browser_sessions_can_read_scoped_user_budget_api() { let (_directory, database) = database().await; let (admin_id, _) = create_admin(&database).await; - identity::enable_managed(&database).await.unwrap(); + identity::enable_managed(&database, &admin_id) + .await + .unwrap(); identity::reset_password(&database, "admin", "M015 budget api admin password") .await .unwrap(); @@ -724,8 +728,10 @@ async fn browser_sessions_can_read_scoped_user_budget_api() { #[tokio::test] async fn token_guess_throttle_runs_before_access_state_and_database_lookup() { let (_directory, database) = database().await; - let (_admin_id, _) = create_admin(&database).await; - identity::enable_managed(&database).await.unwrap(); + let (admin_id, _) = create_admin(&database).await; + identity::enable_managed(&database, &admin_id) + .await + .unwrap(); let process = ProcessRuntime::new(database.clone()).expect("process runtime"); let mut config = Config::default(); @@ -758,6 +764,13 @@ async fn token_guess_throttle_runs_before_access_state_and_database_lookup() { before_missing, "missing credentials are rejected before the access-state read when the shared key is configured" ); + let stale_shared_key = http_request(address, &format!( + "GET /v1/models HTTP/1.1\r\nHost: {address}\r\nAuthorization: Bearer legacy-shared-key-for-test-only\r\nConnection: close\r\n\r\n" + )).await; + assert!( + stale_shared_key.starts_with("HTTP/1.1 401"), + "{stale_shared_key}" + ); for index in 0..30_u64 { let token = format!("epk1_{index:016x}_{:064x}", index + 1); diff --git a/tests/fixtures/cli/contract-matrix.json b/tests/fixtures/cli/contract-matrix.json index ef53455a..65c53312 100644 --- a/tests/fixtures/cli/contract-matrix.json +++ b/tests/fixtures/cli/contract-matrix.json @@ -68,7 +68,7 @@ {"path": "uninstall", "owner": "O009", "options": ["--yes", "--keep-data", "--keep-config", "--keep-path", "--deploy-artifacts"]}, {"path": "update", "owner": "O008", "options": ["--check", "--from-source"]}, {"path": "users", "owner": "O004", "options": []}, - {"path": "users enable-managed", "owner": "M002", "options": []}, + {"path": "users enable-managed", "owner": "M002", "options": ["--actor"]}, {"path": "users bootstrap", "owner": "O004", "options": ["--password-stdin", "--show-secret"]}, {"path": "users list", "owner": "O004", "options": []}, {"path": "users reset-password", "owner": "O004", "options": ["--password-stdin", "--show-secret"]}, diff --git a/tests/tooling/test_on_prem_credential_guidance.py b/tests/tooling/test_on_prem_credential_guidance.py index 7de4e494..ab52ed45 100644 --- a/tests/tooling/test_on_prem_credential_guidance.py +++ b/tests/tooling/test_on_prem_credential_guidance.py @@ -1,6 +1,5 @@ from pathlib import Path - ROOT = Path(__file__).resolve().parents[2] @@ -9,4 +8,3 @@ def test_first_request_uses_personal_token_after_managed_bootstrap() -> None: assert "eggpool users token USER --show-secret" in readme assert 'export EGGPOOL_API_KEY="$(eggpool getkey)"' not in readme assert "it does not grant managed-user access" in readme - From c0828f9b968a9f10ffdadc47b7dd46968f4a9b7c Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 13:10:53 +0000 Subject: [PATCH 36/62] Start closing headless administration plan 020 --- .../020-managed-transition-headless-admin.md | 2 +- plans/registry.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md b/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md index 6ff51079..5b527a5a 100644 --- a/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md +++ b/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md @@ -1,6 +1,6 @@ # On-Prem User Access M020 — Managed transition consistency and headless administration -Status: active +Status: closing Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index e6fca20a..132d972d 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -73,7 +73,7 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| -| On-prem user access | M020 — managed transition and headless administration | active | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | Hard M017/M019 satisfied; align managed transition and local member lifecycle commands. | +| On-prem user access | M020 — managed transition and headless administration | closing | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | State transition, headless lifecycle, atomic profile assignment and full local gates passed; closure audit in progress. | ## Blocked work From c32f141ed71b5e22418a55c1c5b942f1cea929e5 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 13:12:20 +0000 Subject: [PATCH 37/62] Close headless administration plan and unblock defaults --- .../closure/on-prem-user-access/020-status.md | 98 +++++++++++++++++++ .../020-managed-transition-headless-admin.md | 2 +- .../021-default-member-policy-provisioning.md | 2 +- plans/registry.md | 8 +- .../subsystems/on-prem-user-access-roadmap.md | 12 +-- 5 files changed, 111 insertions(+), 11 deletions(-) create mode 100644 plans/closure/on-prem-user-access/020-status.md diff --git a/plans/closure/on-prem-user-access/020-status.md b/plans/closure/on-prem-user-access/020-status.md new file mode 100644 index 00000000..bbc1391a --- /dev/null +++ b/plans/closure/on-prem-user-access/020-status.md @@ -0,0 +1,98 @@ +# On-Prem User Access M020 — Closure Status + +Status: closed + +Source implementation plan: + +- `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` + +Source subsystem roadmap: + +- `plans/subsystems/on-prem-user-access-roadmap.md#m020--managed-transition-and-headless-administration` + +Repository baseline reviewed: post-merge `88f58bbf`; implementation +`8a875a1` after M017/M019 closure and M020 activation. + +## 1. Executive finding + +The CLI and browser member-creation path now use one database transaction for +the access-state transition, member, optional initial profile, and audit +entries. The first successful member add transitions `admin_initialized` to +`managed`; invalid profile assignment leaves the original state, users, audit, +and existing admin credential unchanged. The explicit `users enable-managed` +command uses the same transition helper and requires an active admin actor. +Stale shared-key and public-dashboard settings are reported as cleanup work; +they neither block the cutover nor restore legacy authentication. + +Local operators can add, inspect, disable, and reactivate members. Passwords +come from a hidden TTY prompt or stdin, never argv. Disable revokes current +sessions and personal tokens; reactivation does not restore them. The last +active administrator remains protected. + +## 2. Requirement-to-evidence matrix + +| Requirement | Evidence | Result | Notes | +|---|---|---|---| +| CLI/browser use the same managed-state transition rules | `identity.rs::transition_to_managed`; CLI `enable_managed`; `create_member_with_fence`; `on_prem_identity` | pass | The transition remains atomic with member creation on the first-member path. | +| Failed initial profile assignment leaves no partial user or state | `on_prem_headless_admin::headless_member_lifecycle_is_atomic_audited_and_works_with_stale_config` | pass | State stays `admin_initialized`; no member/transition/profile audit is committed; admin token remains valid. | +| Headless add/show/disable/enable lifecycle is usable | `on_prem_headless_admin`; `cli_contract::headless_member_lifecycle_commands_parse_with_secret_safe_inputs` | pass | Temporary password delivered by stdin; profile assignment, show output, disable, re-enable, and token reissue guidance exercised. | +| Shared key is rejected after cutover even if stale in config | `on_prem_sessions::token_guess_throttle_runs_before_access_state_and_database_lookup`; headless stale-config fixture | pass | A request bearing the configured legacy key receives 401 after managed transition. | +| Access transition requires an active admin and records audit once | `on_prem_identity::managed_transition_requires_an_active_admin_and_audits_once`; stale-settings CLI fixture | pass | Repeated enable is idempotent and does not duplicate transition audit. | +| Concurrent first-member attempts preserve uniqueness and transition integrity | `on_prem_identity::concurrent_first_member_creation_commits_one_transition_and_one_user` | pass | Same normalized username raced; one creation and one transition commit, other attempt conflicts. | +| Disable/re-enable preserves revocation and last-admin protection | Headless CLI fixture; `on_prem_sessions`; `on_prem_admin_ui` | pass | Old token remains invalid after reactivation; last active admin disable is rejected. | +| Existing browser role/session and migration behavior stays intact | `on_prem_admin_ui`, `on_prem_sessions`, `database_compatibility` within workspace suite | pass | No schema migration or role change was added. | + +## 3. Implementation evidence + +- `rust/src/identity.rs` provides bounded username lookup, shared managed + transition logic, local member creation with optional profile assignment, + actor checks, last-admin fencing, credential/session revocation, and audit. +- `rust/src/cli.rs` and `rust/src/runtime.rs` add `users add`, `show`, + `disable`, and `enable`; explicit `enable-managed` accepts `--actor` and + works with stale legacy configuration. Temporary passwords are hidden in a + TTY or read from stdin. +- Member creation, the first managed transition, optional profile assignment, + and audit rows commit or roll back together. No new DB worker or table was + added. +- `tests/fixtures/cli/contract-matrix.json`, CLI/deployment docs, and the + security deep dive now describe the supported operator interface and + configuration-hygiene boundary. + +## 4. Verification executed + +- `cargo test --manifest-path rust/Cargo.toml --test cli_contract -- --test-threads=1` — 7 passed. +- `cargo test --manifest-path rust/Cargo.toml --test on_prem_identity -- --test-threads=1` — 5 passed. +- `cargo test --manifest-path rust/Cargo.toml --test on_prem_sessions -- --test-threads=1` — 10 passed. +- `cargo test --manifest-path rust/Cargo.toml --test on_prem_admin_ui -- --test-threads=1` — 3 passed. +- `cargo test --manifest-path rust/Cargo.toml --test on_prem_headless_admin -- --test-threads=1` — 2 passed. +- Default serial workspace: 1,035 passed, 1 ignored across 80 suites. +- No-default serial workspace: 1,036 passed, 1 ignored across 80 suites. +- Strict default/no-default Clippy, no-default workspace check, formatting, Ruff format/check, Pyright, and full tooling pytest passed (192 passed, 1 skipped). +- These are local results. Hosted CI was not run in this closure pass. + +## 5. Invariant and recovery review + +No migration was introduced; access-state labels and the existing schema stay +compatible. The configured shared key remains rejected after identity +initialization, even if its stale value is still loaded from config. The +public-dashboard flag cannot bypass managed role/session checks. A failed +profile foreign-key insert rolls back the transition, user, and audit rows. +Password hashing occurs before entering the serialized SQLite transaction. +Reactivation advances auth epoch and does not restore revoked credentials. + +## 6. Unresolved findings + +| Severity | Finding | Impact | Required action | +|---|---|---|---| +| none | No unresolved M020 implementation finding | — | — | + +## 7. Roadmap and unblock disposition + +M020 is closed. The dependency audit searched the on-prem implementation +plans, roadmap graph, and ready/active/blocked registry tables. M021's sole +hard dependency, M020, is closed, so M021 moves `blocked` → `ready`. M023 +remains blocked on M022 and ordinary-server operational evidence. M022 remains +ready. No other registered plan is promoted and no capacity claim is added. + +The registry and roadmap record M020 closed and M021 ready. Historical +M001–M019 closure records were not modified. diff --git a/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md b/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md index 5b527a5a..5ef7b8a6 100644 --- a/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md +++ b/plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md @@ -1,6 +1,6 @@ # On-Prem User Access M020 — Managed transition consistency and headless administration -Status: closing +Status: implemented Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md b/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md index c7b9f1bd..4c5b3d4d 100644 --- a/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md +++ b/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md @@ -1,6 +1,6 @@ # On-Prem User Access M021 — Default member policy and provisioning UX -Status: blocked +Status: ready Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index 132d972d..f07274df 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -58,7 +58,7 @@ No new runtime, schema or performance claim lands with these plans. | Persistence | active | `plans/subsystems/persistence-roadmap.md` | M011 ready — post-M010 roadmap/documentation reconciliation | M009/M010 closures accepted. Documentation/tooling-only corrective; production remains WAL/NORMAL. No technical persistence successor is registered. | | Deployment and packaging | closed | `plans/subsystems/deployment-packaging-roadmap.md` | M005 closed — active PATH integration detection corrective | M001–M005 closed; no successor registered. | | Dashboard | closed | `plans/subsystems/dashboard-roadmap.md` | M012 closed — lifecycle closure and documentation polish | M001–M012 closed; no registered successor; future dashboard work requires a new bounded plan. | -| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M020/M022 ready — headless administration and hard-budget compatibility | M017–M019 closed; M021/M023 retain their hard dependencies. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | +| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M021/M022 ready — default member provisioning and hard-budget compatibility | M017–M020 closed; M023 retains its hard dependency/evidence gates. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | | On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 ready — ordinary-server small-team synthetic capacity baseline | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 is independent measurement on ordinary on-prem Linux with durable SSD/NVMe (operational target required), not an authorization to change WAL/SQLite topology. | ## Dependency-ready implementation plans @@ -67,20 +67,19 @@ No new runtime, schema or performance claim lands with these plans. |---|---|---|---|---| | Persistence | M011 — post-M010 roadmap and documentation reconciliation | ready | `plans/implementation/persistence/011-post-m010-roadmap-documentation-reconciliation.md` | Hard: M009/M010 closures accepted. Documentation/tooling only; no runtime or physical target dependency. | | On-prem user access | M022 — hard-budget agent interoperability | ready | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | Hard M017 is closed; preserve M008/M015 proof-or-reject guarantees. | +| On-prem user access | M021 — default member profile and provisioning | ready | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | Hard M020 closure accepted; define safe default assignment and provisioning behavior. | | On-prem concurrency performance | M002 — ordinary-server team baseline | ready | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | Independent evidence work; ordinary Linux + durable SSD/NVMe required for team-capacity claims, no Pi/microSD release gate. | ## Active implementation plans | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| -| On-prem user access | M020 — managed transition and headless administration | closing | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | State transition, headless lifecycle, atomic profile assignment and full local gates passed; closure audit in progress. | ## Blocked work | Subsystem | Milestone | Blocker | |---|---|---| -| On-prem user access | M021 — default member profile and provisioning | Hard: M020 closure. | | On-prem user access | M023 — short-window team fairness | Hard: M022 closure; ordinary-server evidence operational. | @@ -113,6 +112,7 @@ Historical M007 blocker assessment at baseline `7e241ad` (`plans/closure/persist | Subsystem / plan | Disposition | Evidence | |---|---|---| +| On-prem user access M020 — managed transition and headless administration | closed — first member transition, profile assignment and audit are atomic; headless add/show/disable/enable workflow and stale-config auth rejection pass; M021 promoted to ready | `plans/closure/on-prem-user-access/020-status.md`, implementation `8a875a1` | | On-prem user access M019 — login transport and solo LAN operability corrective | closed — remote plaintext login GET returns no credential form; loopback and configured HTTPS-proxy paths pass; solo SSH tunnel and team proxy guidance documented; M020 promoted to ready | `plans/closure/on-prem-user-access/019-status.md`, implementation `b18a561` | | On-prem user access M018 — local operator status corrective | closed — status and runtime status use bounded private control-socket readback without a server API key; managed HTTP authorization is unchanged; no successor unblocked | `plans/closure/on-prem-user-access/018-status.md`, implementation `8e2b66a`, CLI fixture `3be5d83` | | On-prem user access M017 — managed client credential reconciliation | closed — managed local config uses an explicitly supplied active personal token; keyless remote profiles remain read-only; client/helper suites and default/no-default serial workspace suites pass; M022 promoted to ready | `plans/closure/on-prem-user-access/017-status.md`, implementation `d726bce`, `218df45` | @@ -533,3 +533,5 @@ Unblock audit (on-prem user access M017 closed, `plans/closure/on-prem-user-acce Unblock audit (on-prem user access M018 closed, `plans/closure/on-prem-user-access/018-status.md`): searched the on-prem implementation plans, roadmap dependency graph, and dependency-ready, active, and blocked registry tables. No registered plan has a hard M018 dependency, so no blocked row is promoted. M019 and M022 remain ready; M020 remains blocked on M019, M021 on M020, and M023 on M022 plus ordinary-server operational evidence. No capacity or generated-output cap claim is introduced. Unblock audit (on-prem user access M019 closed, `plans/closure/on-prem-user-access/019-status.md`): searched the on-prem implementation plans, roadmap dependency graph, and dependency-ready, active, and blocked registry tables. M020's hard dependencies M017 and M019 are both closed, so M020 moves `blocked` → `ready`. M021 remains blocked on M020. M023 remains blocked on M022 and ordinary-server operational evidence. M022 remains ready; no other plan is promoted and no capacity claim is introduced. + +Unblock audit (on-prem user access M020 closed, `plans/closure/on-prem-user-access/020-status.md`): searched the on-prem implementation plans, roadmap dependency graph, and dependency-ready, active, and blocked registry tables. M021's sole hard dependency, M020, is closed, so M021 moves `blocked` → `ready`. M023 remains blocked on M022 and ordinary-server operational evidence; M022 remains ready. No other plan is promoted and no team-capacity claim is introduced. diff --git a/plans/subsystems/on-prem-user-access-roadmap.md b/plans/subsystems/on-prem-user-access-roadmap.md index 27944f17..d43fd396 100644 --- a/plans/subsystems/on-prem-user-access-roadmap.md +++ b/plans/subsystems/on-prem-user-access-roadmap.md @@ -322,9 +322,9 @@ and document a clear SSH/loopback solo SBC browser path plus ordinary-server TLS-proxy team path. No native TLS server or insecure LAN default. Source: `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md`. -#### M020 — Managed transition and headless administration +#### M020 — Managed transition and headless administration (closed) -Class: capability. Ready after hard M017/M019 closure. Unify +Class: capability. Closed after hard M017/M019 closure. Unify `enable-managed` with member-create transition requirements and add local CLI create/disable/enable/show member workflows; preserve audit, CSRF, password-stdin, last-admin and atomic creation guarantees. @@ -332,7 +332,7 @@ Source: `plans/implementation/on-prem-user-access/020-managed-transition-headles #### M021 — Default member policy and provisioning UX -Class: capability. Blocked on hard M020 closure. Add explicit reusable +Class: capability. Ready after hard M020 closure. Add explicit reusable member default profile (and optional budget template) with safe fallback, transactional assignment, admin/CLI consistency and understandable catalog-empty/deny feedback. Never grant unaudited model access. @@ -356,7 +356,7 @@ Source: `plans/implementation/on-prem-user-access/023-short-window-fairness-cont ### Post-merge dependency and hardware policy -- M017–M019 are closed. M020 and M022 are ready; neither depends on a Pi/Le +- M017–M020 are closed. M021 and M022 are ready; neither depends on a Pi/Le Potato run. M022 became ready after the M017 closure. - Hard: M017 + M019 -> M020 -> M021; M017 -> M022 -> M023. - Interface: M018's privileged local transport must not broaden HTTP rights; @@ -456,7 +456,7 @@ A ready plan is not an implemented capability. | M017 | closed | `plans/implementation/on-prem-user-access/017-managed-client-credential-reconciliation.md` | `plans/closure/on-prem-user-access/017-status.md` | Managed credential selection and keyless remote export; serial/default/no-default suites pass. | | M018 | closed | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | `plans/closure/on-prem-user-access/018-status.md` | Private local control readback restores status after identity cutover; no HTTP privilege change; no successor unblocked. | | M019 | closed | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | `plans/closure/on-prem-user-access/019-status.md` | Unsafe remote HTTP login GET returns no credential form; M020 unblocked. | -| M020 | ready | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | — | Hard dependencies M017 and M019 are closed. | -| M021 | blocked | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | — | Hard: M020 closure. | +| M020 | closed | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | `plans/closure/on-prem-user-access/020-status.md` | Headless member lifecycle and first-member transition are atomic and audited; M021 unblocked. | +| M021 | ready | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | — | Hard dependency M020 is closed. | | M022 | ready | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | — | Hard M017 closure is satisfied; preserve hard budget proof. | | M023 | blocked | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Hard: M022 closure; ordinary-server evidence operational. | From 2e99cdd2962494d38ee005124f6cbd41c9bf9830 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 13:12:43 +0000 Subject: [PATCH 38/62] Activate default member provisioning plan 021 --- .../021-default-member-policy-provisioning.md | 2 +- plans/registry.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md b/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md index 4c5b3d4d..5a39693e 100644 --- a/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md +++ b/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md @@ -1,6 +1,6 @@ # On-Prem User Access M021 — Default member policy and provisioning UX -Status: ready +Status: active Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index f07274df..db193aa9 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -67,13 +67,13 @@ No new runtime, schema or performance claim lands with these plans. |---|---|---|---|---| | Persistence | M011 — post-M010 roadmap and documentation reconciliation | ready | `plans/implementation/persistence/011-post-m010-roadmap-documentation-reconciliation.md` | Hard: M009/M010 closures accepted. Documentation/tooling only; no runtime or physical target dependency. | | On-prem user access | M022 — hard-budget agent interoperability | ready | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | Hard M017 is closed; preserve M008/M015 proof-or-reject guarantees. | -| On-prem user access | M021 — default member profile and provisioning | ready | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | Hard M020 closure accepted; define safe default assignment and provisioning behavior. | | On-prem concurrency performance | M002 — ordinary-server team baseline | ready | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | Independent evidence work; ordinary Linux + durable SSD/NVMe required for team-capacity claims, no Pi/microSD release gate. | ## Active implementation plans | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| +| On-prem user access | M021 — default member profile and provisioning | active | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | Hard M020 closure accepted; implement explicit deny-by-default profile provisioning. | ## Blocked work From b76d359016315b213c533f0080737ec54a3a5d58 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 13:46:45 +0000 Subject: [PATCH 39/62] Implement default member policy provisioning --- architecture/deep-dive-database.md | 2 +- architecture/deep-dive-security.md | 8 +- docs/backup-restore.md | 9 +- docs/cli-reference.md | 11 +- docs/deployment.md | 11 +- docs/model-limits.md | 4 + docs/model-routing.md | 6 + .../021-default-member-policy-provisioning.md | 2 +- .../0063_default_member_profile.sql | 10 ++ rust/assets/db/migrations/checksums.json | 3 +- rust/assets/runtime-manifest.json | 4 +- rust/src/cli.rs | 23 ++++ rust/src/coordinator/endpoints.rs | 1 + rust/src/db/migrations.rs | 4 +- rust/src/identity.rs | 64 ++++++++- rust/src/model_policy.rs | 105 +++++++++++++- rust/src/runtime.rs | 87 ++++++++++-- rust/src/server/dashboard/admin.rs | 58 +++++++- rust/src/server/dashboard/mod.rs | 6 +- rust/src/server/dashboard/render/admin.rs | 23 +++- rust/src/server/mod.rs | 14 +- rust/tests/build_manifest.rs | 4 +- rust/tests/cli_contract.rs | 27 ++++ rust/tests/database_compatibility.rs | 25 +++- rust/tests/on_prem_admin_ui.rs | 79 ++++++++++- rust/tests/on_prem_model_policy.rs | 129 ++++++++++++++++++ 26 files changed, 662 insertions(+), 57 deletions(-) create mode 100644 rust/assets/db/migrations/0063_default_member_profile.sql diff --git a/architecture/deep-dive-database.md b/architecture/deep-dive-database.md index 5d8dc61f..246686c1 100644 --- a/architecture/deep-dive-database.md +++ b/architecture/deep-dive-database.md @@ -18,7 +18,7 @@ Commit/rollback ambiguity fails closed: body failure maps to `Transaction`/`Sqli ## Migrations -Migrations are embedded from `rust/assets/db/migrations/` (zero-padded `0001_*.sql`…`0062_*.sql` = versions 1–62) with `checksums.json` (`files` map, 62 entries), validated by `MigrationRunner::validate_embedded_checksums` before any database use. Migration 55 adds downstream identity and credential storage without repurposing upstream `accounts`; migration 56 adds browser sessions and CSRF digests; migration 57 adds user/auth-epoch attribution to request rows and selector-parent linkage; migration 58 adds local model-policy profiles, exact rules, per-user overrides, instance denials, and a monotonic policy revision; migration 59 adds user budget rules, request holds, attempt usage, audit, and bounded user/model/time rollups; migration 60 adds bounded secret-free administrative audit events; migration 61 adds the hard/estimated enforcement mode, worst-case hold facts, and price provenance to the budget tables; migration 62 adds the instance-global budget scope (rules, request holds, per-model usage buckets, audit, policy-revision triggers) with no user reference. Schema history is never rewritten or renumbered. `MigrationRunner::run` keeps the `_migrations` ledger, rejects unknown versions and ledger-name mismatches, refuses read-only migration, and applies all pending migrations in one atomic transaction. Restore validation additionally checks the staged ledger against this inventory. +Migrations are embedded from `rust/assets/db/migrations/` (zero-padded `0001_*.sql`…`0063_*.sql` = versions 1–63) with `checksums.json` (`files` map, 63 entries), validated by `MigrationRunner::validate_embedded_checksums` before any database use. Migration 55 adds downstream identity and credential storage without repurposing upstream `accounts`; migration 56 adds browser sessions and CSRF digests; migration 57 adds user/auth-epoch attribution to request rows and selector-parent linkage; migration 58 adds local model-policy profiles, exact rules, per-user overrides, instance denials, and a monotonic policy revision; migration 59 adds user budget rules, request holds, attempt usage, audit, and bounded user/model/time rollups; migration 60 adds bounded secret-free administrative audit events; migration 61 adds the hard/estimated enforcement mode, worst-case hold facts, and price provenance to the budget tables; migration 62 adds the instance-global budget scope (rules, request holds, per-model usage buckets, audit, policy-revision triggers) with no user reference; migration 63 adds one audited, instance-local default member profile without changing existing assignments. Schema history is never rewritten or renumbered. `MigrationRunner::run` keeps the `_migrations` ledger, rejects unknown versions and ledger-name mismatches, refuses read-only migration, and applies all pending migrations in one atomic transaction. Restore validation additionally checks the staged ledger against this inventory. `users`, `user_credentials`, and `user_access_state` hold usernames, password hashes, downstream roles, token digests, scopes, and explicit transition state. diff --git a/architecture/deep-dive-security.md b/architecture/deep-dive-security.md index d394fdc1..2c3179b4 100644 --- a/architecture/deep-dive-security.md +++ b/architecture/deep-dive-security.md @@ -71,14 +71,18 @@ trust. emits a startup warning. The local operator CLI uses the data-directory owner's filesystem authority. -`users add` commits a member, optional initial profile, the transition from +`users add` commits a member, an explicit or configured default initial profile, +the transition from `admin_initialized` to `managed`, and audit entries in one transaction; a failed profile assignment leaves the original state intact. The explicit `users enable-managed` path uses the same transition helper and records its own audit event. A remaining configured shared key is rejected after identity initialization, and `dashboard.public` cannot expose managed routes; removing those stale config values is cleanup, not a precondition for the -atomic cutover. Disabling a user revokes sessions and tokens and advances the +atomic cutover. Default member profile selection is audited and revisioned, +affects only future member creation, and does not alter existing assignments. +Clearing restores deny-by-default for future unassigned members. No budget +template is copied; budget scopes remain independently configured. Disabling a user revokes sessions and tokens and advances the auth epoch. Re-enabling does not un-revoke credentials, and the last active administrator remains protected. Session-authenticated responses carry `Cache-Control: no-store, max-age=0` so diff --git a/docs/backup-restore.md b/docs/backup-restore.md index 68d746cf..0fde1048 100644 --- a/docs/backup-restore.md +++ b/docs/backup-restore.md @@ -186,9 +186,10 @@ sudo systemctl start eggpool Migrations are idempotent and safe to run multiple times. -Schema migrations 55–62 add local downstream identity, browser sessions, +Schema migrations 55–63 add local downstream identity, browser sessions, policy profiles, user budgets with enforceable hold/settlement state -(migration 61), instance-global budgets (migration 62), and administrative +(migration 61), instance-global budgets (migration 62), the future-member +default profile (migration 63), and administrative audit tables. Backups made after those migrations include usernames, password hashes, personal-token digests, session/CSRF digests, policy and budget state, bounded usage records, and secret-free audit facts. They do not contain plaintext passwords, API @@ -199,13 +200,13 @@ requests interrupt, active holds release without folding usage) exactly like a crashed live database. After restoring an older backup, the migrations are applied by `eggpool migrate` or normal server startup; existing provider accounts and keys remain owned by TOML/environment as before. -A schema-54 image restores and migrates forward through 55–62 with legacy +A schema-54 image restores and migrates forward through 55–63 with legacy requests, usage history, quarantine, and catalog state preserved; historical shared-key requests stay unattributed (`legacy_shared_key`, never converted to a fabricated human identity), and the upgraded database reports `legacy_unmigrated` until an administrator bootstraps. SQLite migrations are forward-only: an older executable may reject a database -that has reached schema 62. Roll back by restoring a protected backup made +that has reached schema 63. Roll back by restoring a protected backup made before the upgrade together with the matching older binary; do not point an older executable at the upgraded database. Capture the protected backup before running `migrate`, not after. diff --git a/docs/cli-reference.md b/docs/cli-reference.md index bfd0dfc8..db333d6d 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -55,7 +55,10 @@ Command list verified against `eggpool help` (0.8.2). | `eggpool getkey` | Print the server API key | | `eggpool newkey` | Generate and write a new server API key (`--show-old`, `--show-secrets`) | | `eggpool users bootstrap` | Create the initial local administrator (`--password-stdin` for unattended input) | -| `eggpool users add USER` | Create a member with a TTY-hidden or `--password-stdin` temporary password; optional `--profile-id PROFILE`; `--actor ADMIN` records the active administrator | +| `eggpool users add USER` | Create a member with a TTY-hidden or `--password-stdin` temporary password; optional `--profile-id PROFILE` or `--no-profile`; otherwise uses the configured new-member default; `--actor ADMIN` records the active administrator | +| `eggpool users defaults show` | Show the default member profile and budget-template behavior | +| `eggpool users defaults set PROFILE` | Select the profile for future members created without an explicit profile | +| `eggpool users defaults clear` | Remove the default; future unassigned members remain deny-by-default | | `eggpool users show USER` | Show one user's role, state and forced-password-change status without secrets | | `eggpool users list` | List local usernames, role and status | | `eggpool users disable USER` | Disable a user and revoke their sessions and personal tokens | @@ -84,8 +87,10 @@ default and add explicit allowed models. Local member lifecycle commands run from the machine that owns the EggPool data directory. The first successful `users add` moves `admin_initialized` to `managed` in the same SQLite transaction as the user, -optional initial profile, and audit records. A failed profile assignment does -not leave a user or change the access state. Disabling revokes active sessions +optional or default initial profile, and audit records. A failed profile assignment does +not leave a user or change the access state. The default applies to future +members only; setting or clearing it never reassigns existing users. There is +no default budget template. Disabling revokes active sessions and personal tokens; re-enabling does not restore those tokens, so issue a new one with `users token USER`. The last active administrator cannot be disabled. diff --git a/docs/deployment.md b/docs/deployment.md index 1ce868c7..2722fb66 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -61,7 +61,12 @@ Manage local identities from the machine that owns the data directory with `eggpool users token USER`, `eggpool users revoke USER`, and `eggpool users reset-password USER`. Member passwords are read from a hidden TTY prompt or stdin; passwords are never accepted in argv. Add `--profile-id PROFILE` to assign the initial -model policy in the same transaction. Personal tokens are for inference, +model policy in the same transaction. `eggpool users defaults show|set PROFILE|clear` +manages the instance default used only when future members are created without +an explicit profile; `users add USER --no-profile` explicitly keeps a new +member unassigned. Existing assignments are never changed by setting or +clearing the default. No budget template is copied: user and instance budget +rules remain independently configured. Personal tokens are for inference, model discovery, and the integration profile; they do not authorize dashboard or admin pages. The configured server API key remains a legacy shared inference credential only while the database is `legacy_unmigrated`. Do not @@ -71,7 +76,9 @@ With `[dashboard].enabled = true`, an administrator can manage users, profiles, budgets, tokens, and audit history in the embedded dashboard. New members receive a temporary password and must change it at first login. Members can manage their personal tokens and view only their own usage and accessible -models. A browser-issued token is shown once in a no-store response; set it as +models. The users page can use the instance default, select a specific profile, +or explicitly leave a member without a profile. A browser-issued token is shown +once in a no-store response; set it as `EGGPOOL_API_KEY` in the environment that launches the desktop client. Remote browser login must use the operator's TLS proxy and trusted-proxy setup below: LAN reachability alone does not encrypt passwords or cookies. diff --git a/docs/model-limits.md b/docs/model-limits.md index 8a5c054a..09578cdf 100644 --- a/docs/model-limits.md +++ b/docs/model-limits.md @@ -173,3 +173,7 @@ the per-user contract above. Global counters never depend on a user row, so disabling or deleting a user preserves history and remaining headroom. Members see only their own rules and usage; instance ceilings and aggregate usage are admin-visible only. + +Default member profiles do not create or copy budget rules. Configure user and +instance ceilings separately; changing the default model profile leaves all +existing budget rules and usage unchanged. diff --git a/docs/model-routing.md b/docs/model-routing.md index bafce8dc..b7fff9dd 100644 --- a/docs/model-routing.md +++ b/docs/model-routing.md @@ -144,6 +144,12 @@ exact provider-qualified IDs in `model/provider` form. Instance-wide hard denials always win, then per-user overrides, profile rules, and the profile default. Restricted profiles default-deny models without an explicit allow; an administrator without an assigned profile retains unrestricted access. +An administrator may select one instance-local default member profile. CLI and +dashboard member creation apply it transactionally only when no profile was +selected explicitly. It affects future members only; clearing it leaves future +unassigned members deny-by-default, and existing assignments do not change. +Administrators can explicitly create a member without a profile. There is no +default budget template: per-user and instance budget rules remain separate. For a virtual alias, EggPool checks the alias, selector model, and resolved concrete route against the same snapshot. A selector, repair result, sticky diff --git a/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md b/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md index 5a39693e..c2f5d1b8 100644 --- a/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md +++ b/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md @@ -30,7 +30,7 @@ Default deny remains the internal safe fallback; instance-deny always overrides ## 5. Scope -In: optional operator-configured default profile/budget template, explicit create-time profile confirmation, policy visibility/reason codes, admin+CLI consistent defaults. Out: arbitrary role hierarchy, orgs, broad wildcard rules, billing/invoices, prompt/content retention, model policy language redesign. +In: optional operator-configured default profile, explicit create-time profile choice, policy visibility/reason codes, admin+CLI consistent defaults. This pass does not add a default budget template: user and instance rules remain independently configured, and member creation copies no budget state. Out: arbitrary role hierarchy, orgs, broad wildcard rules, billing/invoices, prompt/content retention, model policy language redesign. ## 6. Production changes diff --git a/rust/assets/db/migrations/0063_default_member_profile.sql b/rust/assets/db/migrations/0063_default_member_profile.sql new file mode 100644 index 00000000..d1392cdf --- /dev/null +++ b/rust/assets/db/migrations/0063_default_member_profile.sql @@ -0,0 +1,10 @@ +ALTER TABLE policy_state + ADD COLUMN default_member_profile_id TEXT + REFERENCES policy_profiles(id) ON DELETE RESTRICT; + +CREATE TRIGGER policy_state_default_member_profile_revision_update +AFTER UPDATE OF default_member_profile_id ON policy_state +WHEN OLD.default_member_profile_id IS NOT NEW.default_member_profile_id +BEGIN + UPDATE policy_state SET revision = revision + 1 WHERE singleton = 1; +END; diff --git a/rust/assets/db/migrations/checksums.json b/rust/assets/db/migrations/checksums.json index c43ef577..2455ba77 100644 --- a/rust/assets/db/migrations/checksums.json +++ b/rust/assets/db/migrations/checksums.json @@ -61,6 +61,7 @@ "0059_user_budgets.sql": "6637f6242750639cf292359ab3f476ff1f43ee8d9cb7de99ca67a84088f49bc2", "0060_user_admin_audit.sql": "fe15aebc5a344aa25e049aaf554e8d7eaac69bd16134abbb41e24c390df8891c", "0061_user_budget_enforcement.sql": "d19229166fed11622a3c924642705d78563e95bd02605123b7b2b9483ecf1a3f", - "0062_instance_budgets.sql": "bcb0999e1be7944512f082164f763671bb5ce1b6e82c071470d3ff5f03e5ebac" + "0062_instance_budgets.sql": "bcb0999e1be7944512f082164f763671bb5ce1b6e82c071470d3ff5f03e5ebac", + "0063_default_member_profile.sql": "853cc2e66f6e667f7010c88471b525d2b459393b148aab3fe051b789cec02558" } } diff --git a/rust/assets/runtime-manifest.json b/rust/assets/runtime-manifest.json index 353e53ea..5356fc3f 100644 --- a/rust/assets/runtime-manifest.json +++ b/rust/assets/runtime-manifest.json @@ -5,7 +5,7 @@ "path": "db/migrations/checksums.json", "category": "sqlite-migrations", "source": "rust/assets/db/migrations/checksums.json", - "sha256": "6391e4ddd06466dfb899dd2baabb6669c7299b8562b27854224b0d8dfe415e00", + "sha256": "5f911c22721620a6bd02968df0d815d7592895a1305218eb7d9b334d2fb890b5", "identity": "exact" }, { @@ -51,7 +51,7 @@ "identity": "exact" } ], - "schema_migration_count": 62, + "schema_migration_count": 63, "schema_change": true, "ownership": "Rust build/runtime embeds these assets; source paths are canonical repository inputs", "rollback": "restore the source tree from Git history" diff --git a/rust/src/cli.rs b/rust/src/cli.rs index 0445b5b6..b1fc3f57 100644 --- a/rust/src/cli.rs +++ b/rust/src/cli.rs @@ -173,6 +173,26 @@ pub enum UsersCommand { /// per-user rule. Requires an active administrator. #[command(subcommand)] GlobalBudget(UsersGlobalBudgetCommand), + /// Show, select, or clear the default profile for newly created members. + #[command(subcommand)] + Defaults(UsersDefaultsCommand), +} + +#[derive(Debug, Subcommand)] +pub enum UsersDefaultsCommand { + /// Show the default member profile, if one is selected. + Show, + /// Apply a profile to members created from now on. + Set(UserDefaultProfileSetArgs), + /// Leave future members without a profile (deny by default). + Clear(UserActorArgs), +} + +#[derive(Debug, clap::Args)] +pub struct UserDefaultProfileSetArgs { + pub profile_id: String, + #[arg(long, default_value = "admin")] + pub actor: String, } #[derive(Debug, Subcommand)] @@ -325,6 +345,9 @@ pub struct UserAddArgs { /// Assign an existing model-policy profile in the same transaction. #[arg(long)] pub profile_id: Option, + /// Create with no profile even when an instance default exists. + #[arg(long, conflicts_with = "profile_id")] + pub no_profile: bool, #[arg(long, default_value = "admin")] pub actor: String, } diff --git a/rust/src/coordinator/endpoints.rs b/rust/src/coordinator/endpoints.rs index 1eac8ec1..7c7bc4f1 100644 --- a/rust/src/coordinator/endpoints.rs +++ b/rust/src/coordinator/endpoints.rs @@ -693,6 +693,7 @@ async fn resolve_concrete( } crate::model_policy::ModelPolicyError::InvalidPolicyInput | crate::model_policy::ModelPolicyError::PreferredModelDenied + | crate::model_policy::ModelPolicyError::DefaultProfileNotFound | crate::model_policy::ModelPolicyError::Database(_) => EndpointError::Internal, })?, ), diff --git a/rust/src/db/migrations.rs b/rust/src/db/migrations.rs index 47c74cfd..bdbcc85c 100644 --- a/rust/src/db/migrations.rs +++ b/rust/src/db/migrations.rs @@ -203,9 +203,9 @@ mod tests { #[test] fn canonical_inventory_is_complete_and_immutable() { MigrationRunner::validate_embedded_checksums().expect("canonical checksums match"); - assert_eq!(MigrationRunner::migrations().len(), 62); + assert_eq!(MigrationRunner::migrations().len(), 63); assert_eq!(MigrationRunner::migrations().first().unwrap().version, 1); - assert_eq!(MigrationRunner::migrations().last().unwrap().version, 62); + assert_eq!(MigrationRunner::migrations().last().unwrap().version, 63); } #[test] diff --git a/rust/src/identity.rs b/rust/src/identity.rs index 61b4e78c..58c9dfc7 100644 --- a/rust/src/identity.rs +++ b/rust/src/identity.rs @@ -368,7 +368,16 @@ pub async fn create_member( username: &str, temporary_password: &str, ) -> Result { - create_member_with_fence(database, actor_id, username, temporary_password, None, None).await + create_member_with_fence( + database, + actor_id, + username, + temporary_password, + None, + None, + true, + ) + .await } pub async fn create_member_with_profile( @@ -385,6 +394,25 @@ pub async fn create_member_with_profile( temporary_password, None, profile_id, + true, + ) + .await +} + +pub async fn create_member_without_profile( + database: &Database, + actor_id: &str, + username: &str, + temporary_password: &str, +) -> Result { + create_member_with_fence( + database, + actor_id, + username, + temporary_password, + None, + None, + false, ) .await } @@ -402,6 +430,25 @@ pub async fn create_member_from_browser( temporary_password, Some(crate::web_session::SessionFence::from_principal(principal)), None, + true, + ) + .await +} + +pub async fn create_member_without_profile_from_browser( + database: &Database, + principal: &crate::web_session::Principal, + username: &str, + temporary_password: &str, +) -> Result { + create_member_with_fence( + database, + &principal.user_id, + username, + temporary_password, + Some(crate::web_session::SessionFence::from_principal(principal)), + None, + false, ) .await } @@ -420,6 +467,7 @@ pub async fn create_member_with_profile_from_browser( temporary_password, Some(crate::web_session::SessionFence::from_principal(principal)), Some(profile_id), + true, ) .await } @@ -431,6 +479,7 @@ async fn create_member_with_fence( temporary_password: &str, session_fence: Option, profile_id: Option<&str>, + use_default_profile: bool, ) -> Result { let username = validate_username(username)?; let password_hash = hash_password(temporary_password).await?; @@ -472,6 +521,17 @@ async fn create_member_with_fence( [&actor_id], )?; } + let selected_profile_id = if profile_id.is_some() { + profile_id.clone() + } else if use_default_profile { + connection.query_row( + "SELECT default_member_profile_id FROM policy_state WHERE singleton = 1", + [], + |row| row.get::<_, Option>(0), + )? + } else { + None + }; connection.execute( "INSERT INTO users(id, username, normalized_username, role, password_hash, must_change_password) \ VALUES (?1, ?2, ?3, 'member', ?4, 1)", @@ -482,7 +542,7 @@ async fn create_member_with_fence( VALUES (?1, 'create_user', 'user', ?2)", (&actor_id, &id), )?; - if let Some(profile_id) = profile_id.as_ref() { + if let Some(profile_id) = selected_profile_id.as_ref() { connection.execute( "INSERT INTO user_profile_assignments(user_id, profile_id, assigned_by) \ VALUES (?1, ?2, ?3)", diff --git a/rust/src/model_policy.rs b/rust/src/model_policy.rs index b223f25a..848579c6 100644 --- a/rust/src/model_policy.rs +++ b/rust/src/model_policy.rs @@ -38,6 +38,8 @@ pub enum ModelPolicyError { InvalidPolicyInput, #[error("preferred model is not allowed by its profile")] PreferredModelDenied, + #[error("member default profile was not found")] + DefaultProfileNotFound, #[error(transparent)] Database(#[from] crate::db::DatabaseError), } @@ -62,6 +64,7 @@ pub struct PolicyProfileSummary { pub version: i64, pub assigned_users: i64, pub model_rules: i64, + pub is_default_member_profile: bool, } #[derive(Debug, Clone, PartialEq, Eq)] @@ -77,6 +80,12 @@ impl ModelPolicySnapshot { self.revision } + /// Whether a member has an assigned profile policy. Administrators may + /// intentionally be unprofiled and retain the administrator default. + pub fn has_profile(&self) -> bool { + self.default_effect.is_some() + } + pub fn preferred_model_id(&self) -> Option<&str> { self.preferred_model_id.as_deref() } @@ -223,7 +232,8 @@ impl ModelPolicyService { let mut statement = connection.prepare( "SELECT p.id, p.name, p.default_effect, p.preferred_model_id, p.version, \ (SELECT COUNT(*) FROM user_profile_assignments a WHERE a.profile_id = p.id), \ - (SELECT COUNT(*) FROM profile_model_rules r WHERE r.profile_id = p.id) \ + (SELECT COUNT(*) FROM profile_model_rules r WHERE r.profile_id = p.id), \ + COALESCE(p.id = (SELECT default_member_profile_id FROM policy_state WHERE singleton = 1), 0) \ FROM policy_profiles p ORDER BY p.name COLLATE NOCASE, p.id LIMIT 200", )?; statement @@ -236,6 +246,7 @@ impl ModelPolicyService { version: row.get(4)?, assigned_users: row.get(5)?, model_rules: row.get(6)?, + is_default_member_profile: row.get(7)?, }) })? .collect() @@ -267,6 +278,91 @@ impl ModelPolicyService { .map_err(ModelPolicyError::Database) } + pub async fn default_member_profile(&self) -> Result, ModelPolicyError> { + self.database + .call(|connection| { + connection.query_row( + "SELECT default_member_profile_id FROM policy_state WHERE singleton = 1", + [], + |row| row.get(0), + ) + }) + .await + .map_err(ModelPolicyError::Database) + } + + pub async fn set_default_member_profile( + &self, + profile_id: Option<&str>, + actor_id: &str, + ) -> Result { + let profile_id = profile_id + .map(|id| { + let id = id.trim(); + if id.is_empty() || id.len() > 64 { + Err(ModelPolicyError::InvalidPolicyInput) + } else { + Ok(id.to_owned()) + } + }) + .transpose()?; + let actor_id = actor_id.to_owned(); + let mutation_fence = self.mutation_fence.clone(); + self.database + .with_transaction(move |connection| { + validate_mutation_actor(connection, &actor_id, mutation_fence.as_ref())?; + if let Some(profile_id) = profile_id.as_deref() { + let exists = connection.query_row( + "SELECT EXISTS(SELECT 1 FROM policy_profiles WHERE id = ?1)", + [profile_id], + |row| row.get::<_, bool>(0), + )?; + if !exists { + return Err(tokio_rusqlite::rusqlite::Error::QueryReturnedNoRows); + } + } + let current: Option = connection.query_row( + "SELECT default_member_profile_id FROM policy_state WHERE singleton = 1", + [], + |row| row.get(0), + )?; + if current == profile_id { + return Ok(false); + } + connection.execute( + "UPDATE policy_state SET default_member_profile_id = ?1 WHERE singleton = 1", + [profile_id.as_deref()], + )?; + connection.execute( + "INSERT INTO user_admin_audit(actor_id, action, target_kind, target_id) \ + VALUES (?1, ?2, ?3, ?4)", + ( + &actor_id, + if profile_id.is_some() { + "set_default_member_profile" + } else { + "clear_default_member_profile" + }, + if profile_id.is_some() { "profile" } else { "instance" }, + profile_id.as_deref().unwrap_or("singleton"), + ), + )?; + Ok(true) + }) + .await + .map_err(|error| { + if matches!( + &error, + crate::db::DatabaseError::Transaction { source } + if matches!(source.as_ref(), tokio_rusqlite::rusqlite::Error::QueryReturnedNoRows) + ) { + ModelPolicyError::DefaultProfileNotFound + } else { + ModelPolicyError::Database(error) + } + }) + } + pub async fn remove_profile_rule( &self, profile_id: &str, @@ -362,7 +458,12 @@ impl ModelPolicyService { [&id], |row| row.get(0), )?; - if assigned { + let is_default: bool = connection.query_row( + "SELECT EXISTS(SELECT 1 FROM policy_state WHERE singleton = 1 AND default_member_profile_id = ?1)", + [&id], + |row| row.get(0), + )?; + if assigned || is_default { return Ok(false); } let changed = connection.execute( diff --git a/rust/src/runtime.rs b/rust/src/runtime.rs index 3f8c776a..d7a71898 100644 --- a/rust/src/runtime.rs +++ b/rust/src/runtime.rs @@ -719,14 +719,24 @@ async fn users(path: &Path, command: crate::cli::UsersCommand) -> Result<(), Boo let actor = policy_actor_id(&database, &args.actor).await?; let password = read_user_password(args.password_stdin, false, "Temporary member password")?; - let user = crate::identity::create_member_with_profile( - &database, - &actor, - &args.username, - &password, - args.profile_id.as_deref(), - ) - .await + let user = if args.no_profile { + crate::identity::create_member_without_profile( + &database, + &actor, + &args.username, + &password, + ) + .await + } else { + crate::identity::create_member_with_profile( + &database, + &actor, + &args.username, + &password, + args.profile_id.as_deref(), + ) + .await + } .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))?; println!( "Member {} created; the user must change the temporary password at first login.", @@ -844,6 +854,67 @@ async fn users(path: &Path, command: crate::cli::UsersCommand) -> Result<(), Boo } } } + crate::cli::UsersCommand::Defaults(command) => { + use crate::cli::UsersDefaultsCommand as Defaults; + let service = crate::model_policy::ModelPolicyService::new(database.clone()); + match command { + Defaults::Show => { + let default_id = service + .default_member_profile() + .await + .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))?; + if let Some(default_id) = default_id { + let profiles = service + .list_profiles() + .await + .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))?; + if let Some(profile) = + profiles.iter().find(|profile| profile.id == default_id) + { + println!("default_member_profile: {} ({})", profile.id, profile.name); + println!("new members only: true"); + println!("existing assignments changed: false"); + } else { + return Err(command_error( + EXIT_VALIDATION, + "default member profile is missing", + )); + } + } else { + println!("default_member_profile: none"); + println!( + "new members remain deny-by-default unless a profile is selected explicitly." + ); + } + println!( + "default budget template: none; user and instance budget rules are unchanged." + ); + } + Defaults::Set(args) => { + let actor = policy_actor_id(&database, &args.actor).await?; + service + .set_default_member_profile(Some(&args.profile_id), &actor) + .await + .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))?; + println!( + "Default member profile set to {}; it applies to new members only.", + args.profile_id + ); + println!("Existing assignments and all budget rules are unchanged."); + } + Defaults::Clear(args) => { + let actor = policy_actor_id(&database, &args.actor).await?; + service + .set_default_member_profile(None, &actor) + .await + .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))?; + println!( + "Default member profile cleared; future members without an explicit profile remain deny-by-default." + ); + println!("Existing assignments and all budget rules are unchanged."); + } + } + } crate::cli::UsersCommand::Budget(command) => { use crate::cli::UsersBudgetCommand as Budget; match command { diff --git a/rust/src/server/dashboard/admin.rs b/rust/src/server/dashboard/admin.rs index 8232c0ef..040da1d3 100644 --- a/rust/src/server/dashboard/admin.rs +++ b/rust/src/server/dashboard/admin.rs @@ -70,6 +70,12 @@ pub(in crate::server) struct ProfileDeleteForm { csrf: String, } +#[derive(Deserialize)] +pub(in crate::server) struct DefaultMemberProfileForm { + profile_id: Option, + csrf: String, +} + #[derive(Deserialize)] pub(in crate::server) struct ProfileRuleDeleteForm { profile_id: String, @@ -379,13 +385,24 @@ pub(in crate::server) async fn my_models_page( Ok(snapshot) => snapshot, Err(_) => return degraded("model list unavailable"), }; - let models = inference - .authorized_discovery_model_ids(&snapshot) + let available_models = inference.authorized_discovery_model_ids(&snapshot); + let explanation = if available_models.is_empty() { + if inference.discovery_model_ids().is_empty() { + "No models are currently available on this instance; contact the admin about provider setup." + } else if !snapshot.has_profile() { + "You have no member model profile; ask an admin to assign a profile." + } else { + "Your profile currently permits no available models; ask an admin to review the profile." + } + } else { + "This list is filtered by your current model policy." + }; + let models = available_models .into_iter() .map(|model| format!("
  • {}
  • ", html_escape(&model))) .collect::(); html_response(format!( - "My models

    My accessible models

    This list is filtered by your current model policy.

      {models}

    My account

    " + "My models

    My accessible models

    {explanation}

      {models}

    My account

    " )) } @@ -402,7 +419,15 @@ pub(in crate::server) async fn create_user( return (StatusCode::FORBIDDEN, "Administrator access required.").into_response(); } let profile_id = form.profile_id.as_deref().filter(|id| !id.is_empty()); - let created = if let Some(profile_id) = profile_id { + let created = if form.profile_id.as_deref() == Some("__none") { + crate::identity::create_member_without_profile_from_browser( + &state.database, + &session, + &form.username, + &form.password, + ) + .await + } else if let Some(profile_id) = profile_id { crate::identity::create_member_with_profile_from_browser( &state.database, &session, @@ -427,6 +452,31 @@ pub(in crate::server) async fn create_user( mutation_redirect("/users") } +pub(in crate::server) async fn set_default_member_profile( + State(state): State, + Extension(session): Extension, + headers: HeaderMap, + Form(form): Form, +) -> Response { + if !csrf_valid(&session, &headers, &form.csrf) { + return (StatusCode::FORBIDDEN, "Invalid request token.").into_response(); + } + if !admin_only(&session) { + return (StatusCode::FORBIDDEN, "Administrator access required.").into_response(); + } + let result = crate::model_policy::ModelPolicyService::new(state.database.clone()) + .with_browser_admin_session(&session) + .set_default_member_profile( + form.profile_id.as_deref().filter(|id| !id.is_empty()), + &session.user_id, + ) + .await; + if let Err(error) = result { + return mutation_error(error, "Could not update the default member profile."); + } + mutation_redirect("/profiles") +} + pub(in crate::server) async fn set_user_state( State(state): State, Extension(session): Extension, diff --git a/rust/src/server/dashboard/mod.rs b/rust/src/server/dashboard/mod.rs index b35f107f..73d9da50 100644 --- a/rust/src/server/dashboard/mod.rs +++ b/rust/src/server/dashboard/mod.rs @@ -144,9 +144,9 @@ pub(super) use admin::{ assign_user_profile, audit_page, create_personal_token, create_profile, create_user, delete_profile, issue_admin_token, my_account_page, my_models_page, profiles_page, remove_instance_budget, remove_profile_rule, remove_user_override, reset_user_password, - revoke_personal_token, revoke_user_tokens, set_instance_budget, set_profile_preference, - set_profile_rule, set_user_budget, set_user_override, set_user_state, update_profile, - usage_page, users_page, + revoke_personal_token, revoke_user_tokens, set_default_member_profile, set_instance_budget, + set_profile_preference, set_profile_rule, set_user_budget, set_user_override, set_user_state, + update_profile, usage_page, users_page, }; pub(super) use api::{ grouped_timeseries_api, stats_cache_observability, stats_cache_stability, diff --git a/rust/src/server/dashboard/render/admin.rs b/rust/src/server/dashboard/render/admin.rs index 255867c1..c2be0f1b 100644 --- a/rust/src/server/dashboard/render/admin.rs +++ b/rust/src/server/dashboard/render/admin.rs @@ -6,6 +6,18 @@ pub(in crate::server::dashboard) fn render_users_admin( instance_rules: &[crate::instance_budget::InstanceBudgetRule], csrf: &str, ) -> String { + let default_profile = profiles + .iter() + .find(|profile| profile.is_default_member_profile); + let initial_profile = default_profile.map_or_else( + || "".to_owned(), + |profile| { + format!( + "", + html_escape(&profile.name) + ) + }, + ); let options = profiles .iter() .map(|profile| { @@ -89,7 +101,7 @@ pub(in crate::server::dashboard) fn render_users_admin( html_escape(csrf) ); format!( - "{instance_section}

    Users

    Create local member accounts and assign policy profiles. Temporary passwords are shown only to the administrator who enters them; users must change them at first login.

    Local accounts

    {rows}
    UsernameRoleStatePasswordActions
    ", + "{instance_section}

    Users

    Create local member accounts and assign policy profiles. The instance default applies only to new members; existing assignments are unchanged. There is no default budget template: user and instance budget rules stay independent. Temporary passwords are shown only to the administrator who enters them; users must change them at first login.

    Local accounts

    {rows}
    UsernameRoleStatePasswordActions
    ", html_escape(csrf) ) } @@ -113,8 +125,13 @@ pub(in crate::server::dashboard) fn render_profiles_admin( ) }) .collect::(); + let default_control = if profile.is_default_member_profile { + format!("

    Default for newly created members

    ", html_escape(csrf)) + } else { + format!("
    ", html_escape(csrf), html_escape(&profile.id)) + }; format!( - "

    {}

    ID: {} · default: {} · version: {} · assigned users: {} · exact model rules: {}

      {profile_rules}
    ", + "

    {}

    ID: {} · default: {} · version: {} · assigned users: {} · exact model rules: {}

    {default_control}
      {profile_rules}
    ", html_escape(&profile.name), html_escape(&profile.id), html_escape(&profile.default_effect), profile.version, profile.assigned_users, profile.model_rules, html_escape(csrf), html_escape(&profile.id), profile.version, html_escape(&profile.name), @@ -127,7 +144,7 @@ pub(in crate::server::dashboard) fn render_profiles_admin( }) .collect::(); format!( - "

    Profiles

    Profiles control model access and client recommendations. Provider topology and upstream secrets remain in operator-owned configuration.

    {rows}", + "

    Profiles

    Profiles control model access and client recommendations. The default member profile affects future members only. Budget rules remain separately configured per user and instance; no budget template is copied. Provider topology and upstream secrets remain in operator-owned configuration.

    {rows}", html_escape(csrf) ) } diff --git a/rust/src/server/mod.rs b/rust/src/server/mod.rs index d46a3728..a030661c 100644 --- a/rust/src/server/mod.rs +++ b/rust/src/server/mod.rs @@ -19,12 +19,13 @@ use dashboard::{ grouped_timeseries_api, issue_admin_token, latency_page, model_detail_page, models_page, my_account_page, my_models_page, overview, pings_page, profiles_page, reliability_page, remove_instance_budget, remove_profile_rule, remove_user_override, reset_user_password, - revoke_personal_token, revoke_user_tokens, routing_page, runtime_page, set_instance_budget, - set_profile_preference, set_profile_rule, set_user_budget, set_user_override, set_user_state, - static_chart_js, static_css, static_favicon, static_js, stats_cache_observability, - stats_cache_stability, stats_request_segmentation, stats_request_shaping, stats_transcoding, - summary, sync_accounts, theme_css, timeseries_api, timeseries_page, traces_page, - update_profile, usage_page, users_page, + revoke_personal_token, revoke_user_tokens, routing_page, runtime_page, + set_default_member_profile, set_instance_budget, set_profile_preference, set_profile_rule, + set_user_budget, set_user_override, set_user_state, static_chart_js, static_css, + static_favicon, static_js, stats_cache_observability, stats_cache_stability, + stats_request_segmentation, stats_request_shaping, stats_transcoding, summary, sync_accounts, + theme_css, timeseries_api, timeseries_page, traces_page, update_profile, usage_page, + users_page, }; use health::{ healthz, integration_profile, models_api, readyz, runtime_status, status_api, update_status, @@ -1222,6 +1223,7 @@ pub fn build_router(state: AppState) -> Router { post(remove_instance_budget), ) .route("/admin/profiles/create", post(create_profile)) + .route("/admin/profiles/default", post(set_default_member_profile)) .route("/admin/profiles/update", post(update_profile)) .route("/admin/profiles/delete", post(delete_profile)) .route("/admin/profiles/rule", post(set_profile_rule)) diff --git a/rust/tests/build_manifest.rs b/rust/tests/build_manifest.rs index 87a44019..1696ecae 100644 --- a/rust/tests/build_manifest.rs +++ b/rust/tests/build_manifest.rs @@ -28,7 +28,7 @@ fn semantically_equivalent_reformatted_manifest_is_accepted() { let reformatted = serde_json::to_string(&value).expect("compact JSON"); let parsed = build_support::parse_checksums(&reformatted); - assert_eq!(parsed.len(), 62); + assert_eq!(parsed.len(), 63); assert_eq!( parsed.get("0001_initial.sql").map(String::as_str), Some("2aa5800147fc3ef8cc35591c44564244dba138b4dbc92fc1779e4f88813912ad") @@ -55,7 +55,7 @@ fn rust_owned_runtime_assets_are_complete_and_hash_locked() { serde_json::from_str(include_str!("../assets/runtime-manifest.json")) .expect("runtime asset manifest is valid JSON"); assert_eq!(manifest.manifest_version, "runtime-assets.v1"); - assert_eq!(manifest.schema_migration_count, 62); + assert_eq!(manifest.schema_migration_count, 63); assert!(manifest.schema_change); let root = PathBuf::from(env!("CARGO_MANIFEST_DIR")); diff --git a/rust/tests/cli_contract.rs b/rust/tests/cli_contract.rs index 0edc5731..a97d12d7 100644 --- a/rust/tests/cli_contract.rs +++ b/rust/tests/cli_contract.rs @@ -343,6 +343,33 @@ fn headless_member_lifecycle_commands_parse_with_secret_safe_inputs() { assert!(Cli::try_parse_from(["eggpool", "users", "disable", "member-a"]).is_ok()); assert!(Cli::try_parse_from(["eggpool", "users", "enable", "member-a"]).is_ok()); assert!(Cli::try_parse_from(["eggpool", "users", "show", "member-a"]).is_ok()); + assert!( + Cli::try_parse_from([ + "eggpool", + "users", + "add", + "member-b", + "--password-stdin", + "--no-profile" + ]) + .is_ok() + ); + assert!( + Cli::try_parse_from([ + "eggpool", + "users", + "add", + "member-b", + "--password-stdin", + "--no-profile", + "--profile-id", + "restricted" + ]) + .is_err() + ); + assert!(Cli::try_parse_from(["eggpool", "users", "defaults", "show"]).is_ok()); + assert!(Cli::try_parse_from(["eggpool", "users", "defaults", "set", "restricted"]).is_ok()); + assert!(Cli::try_parse_from(["eggpool", "users", "defaults", "clear"]).is_ok()); assert!( Cli::try_parse_from([ "eggpool", diff --git a/rust/tests/database_compatibility.rs b/rust/tests/database_compatibility.rs index 09b80bfe..f8524e10 100644 --- a/rust/tests/database_compatibility.rs +++ b/rust/tests/database_compatibility.rs @@ -40,8 +40,19 @@ async fn fresh_database_migrates_idempotently_and_preserves_pragmas() { .run() .await .expect("migrations apply"); - assert_eq!(first.applied_versions.len(), 62); - assert_eq!(first.applied_this_run.len(), 62); + assert_eq!(first.applied_versions.len(), 63); + assert_eq!(first.applied_this_run.len(), 63); + let default_member_profile: Option = database + .call(|connection| { + connection.query_row( + "SELECT default_member_profile_id FROM policy_state WHERE singleton = 1", + [], + |row| row.get(0), + ) + }) + .await + .expect("fresh member default is readable"); + assert_eq!(default_member_profile, None); database .quick_check() .await @@ -259,7 +270,7 @@ async fn python_historical_fixture_upgrades_and_repositories_round_trip() { .await .expect("current migrations apply"); assert_eq!(state.applied_this_run.first(), Some(&12)); - assert_eq!(state.applied_this_run.last(), Some(&62)); + assert_eq!(state.applied_this_run.last(), Some(&63)); let account = AccountRepository::new(&database) .get_by_name("historical-account") @@ -511,7 +522,7 @@ async fn current_database_can_be_opened_read_only_without_writes() { // forward, and recovers identity/policy/budgets without fabricating // history. The frozen image is `tests/fixtures/database/schema54- // pre-upgrade.sql` (SHA-256 recorded in the M010 closure); tests never run -// migrations 1..54 themselves — only the production 55..62 upgrade path. +// migrations 1..54 themselves — only the production 55..63 upgrade path. // --------------------------------------------------------------------------- use eggpool::{ @@ -628,14 +639,14 @@ async fn schema54_image_upgrades_with_legacy_state_preserved() { .archive; BackupService::validate_archive(&archive).expect("pre-upgrade archive validates"); - // Forward-only production upgrade: exactly migrations 55..62. + // Forward-only production upgrade: exactly migrations 55..63. let state = MigrationRunner::new(&database) .run() .await .expect("production upgrade applies"); assert_eq!(state.applied_this_run.first(), Some(&55)); - assert_eq!(state.applied_this_run.last(), Some(&62)); - assert_eq!(state.applied_this_run.len(), 8); + assert_eq!(state.applied_this_run.last(), Some(&63)); + assert_eq!(state.applied_this_run.len(), 9); database .quick_check() .await diff --git a/rust/tests/on_prem_admin_ui.rs b/rust/tests/on_prem_admin_ui.rs index b19be909..516a4e27 100644 --- a/rust/tests/on_prem_admin_ui.rs +++ b/rust/tests/on_prem_admin_ui.rs @@ -280,9 +280,39 @@ async fn two_user_browser_journey_fences_admin_data_and_personal_credentials() { "{profile_created}" ); - let create_form = format!( - "username=member&password=M006+temporary+member+password&profile_id=team&csrf={csrf}" + let default_form = format!("profile_id=team&csrf={csrf}"); + let default_saved = http_request( + address, + "POST", + "/admin/profiles/default", + &default_form, + Some(&admin_cookies), + Some(&origin), + None, + ) + .await; + assert!(default_saved.starts_with("HTTP/1.1 303"), "{default_saved}"); + let profiles_page = http_request( + address, + "GET", + "/profiles", + "", + Some(&admin_cookies), + None, + None, + ) + .await; + assert!( + profiles_page.contains("Default for newly created members"), + "{profiles_page}" ); + assert!( + profiles_page.contains("Clear default for future members"), + "{profiles_page}" + ); + + let create_form = + format!("username=member&password=M006+temporary+member+password&profile_id=&csrf={csrf}"); let created = http_request( address, "POST", @@ -294,6 +324,51 @@ async fn two_user_browser_journey_fences_admin_data_and_personal_credentials() { ) .await; assert!(created.starts_with("HTTP/1.1 303"), "{created}"); + let assigned_default: bool = database + .call(|connection| { + connection.query_row( + "SELECT EXISTS(SELECT 1 FROM user_profile_assignments a JOIN users u ON u.id = a.user_id WHERE u.username = 'member' AND a.profile_id = 'team')", + [], + |row| row.get(0), + ) + }) + .await + .unwrap(); + assert!( + assigned_default, + "browser creation applies the selected instance default" + ); + let explicit_none_form = format!( + "username=without-profile&password=M006+temporary+member+password&profile_id=__none&csrf={csrf}" + ); + let explicit_none_created = http_request( + address, + "POST", + "/admin/users/create", + &explicit_none_form, + Some(&admin_cookies), + Some(&origin), + None, + ) + .await; + assert!( + explicit_none_created.starts_with("HTTP/1.1 303"), + "{explicit_none_created}" + ); + let explicitly_unassigned: bool = database + .call(|connection| { + connection.query_row( + "SELECT EXISTS(SELECT 1 FROM user_profile_assignments a JOIN users u ON u.id = a.user_id WHERE u.username = 'without-profile')", + [], + |row| row.get(0), + ) + }) + .await + .unwrap(); + assert!( + !explicitly_unassigned, + "explicit no-profile choice bypasses default" + ); let user_page = http_request( address, "GET", diff --git a/rust/tests/on_prem_model_policy.rs b/rust/tests/on_prem_model_policy.rs index 1c79cd03..279e7434 100644 --- a/rust/tests/on_prem_model_policy.rs +++ b/rust/tests/on_prem_model_policy.rs @@ -173,6 +173,135 @@ async fn concurrent_policy_edit_keeps_each_snapshot_consistent_and_refreshes_new database.close().await.unwrap(); } +#[tokio::test] +async fn default_profile_provisions_future_members_only_and_is_audited() { + let database = database().await; + identity::mark_fresh_uninitialized(&database).await.unwrap(); + let (admin, _) = + identity::bootstrap_admin(&database, "admin-a", "M021 strong administrator password") + .await + .unwrap(); + identity::enable_managed(&database, &admin.id) + .await + .unwrap(); + let policy = ModelPolicyService::new(database.clone()); + policy + .create_profile("team", "Team", "allow", &admin.id) + .await + .unwrap(); + policy + .set_profile_rule("team", "model-a/provider-a", "deny", &admin.id) + .await + .unwrap(); + policy + .create_profile("unused", "Unused", "deny", &admin.id) + .await + .unwrap(); + + let before_default = identity::create_member_with_profile( + &database, + &admin.id, + "before-default", + "M021 temporary password", + None, + ) + .await + .unwrap(); + let before = policy.snapshot(&before_default.id).await.unwrap(); + assert!(!before.has_profile()); + assert!(!before.allows("model-a", None)); + + policy + .set_default_member_profile(Some("team"), &admin.id) + .await + .unwrap(); + assert!( + policy + .snapshot(&before_default.id) + .await + .unwrap() + .revision() + > before.revision() + ); + let profiles = policy.list_profiles().await.unwrap(); + assert!( + profiles + .iter() + .any(|profile| profile.id == "team" && profile.is_default_member_profile) + ); + assert!(!policy.delete_profile("team", 2, &admin.id).await.unwrap()); + + let created = identity::create_member_with_profile( + &database, + &admin.id, + "after-default", + "M021 temporary password", + None, + ) + .await + .unwrap(); + let provisioned = policy.snapshot(&created.id).await.unwrap(); + assert!(provisioned.has_profile()); + assert!(provisioned.allows("model-a", None)); + assert!(!provisioned.allows("model-a", Some("provider-a"))); + assert!(!before.has_profile(), "existing snapshot stays unchanged"); + let still_unassigned = policy.snapshot(&before_default.id).await.unwrap(); + assert!( + !still_unassigned.has_profile(), + "default does not reassign existing members" + ); + let explicitly_unassigned = identity::create_member_without_profile( + &database, + &admin.id, + "explicit-no-profile", + "M021 temporary password", + ) + .await + .unwrap(); + assert!( + !policy + .snapshot(&explicitly_unassigned.id) + .await + .unwrap() + .has_profile(), + "explicit no-profile bypasses the instance default" + ); + + policy + .set_default_member_profile(None, &admin.id) + .await + .unwrap(); + let no_default_member = identity::create_member_with_profile( + &database, + &admin.id, + "after-clear", + "M021 temporary password", + None, + ) + .await + .unwrap(); + assert!( + !policy + .snapshot(&no_default_member.id) + .await + .unwrap() + .has_profile() + ); + assert!(policy.delete_profile("unused", 1, &admin.id).await.unwrap()); + let audit = identity::admin_audit(&database, 100).await.unwrap(); + assert!( + audit + .iter() + .any(|entry| entry.action == "set_default_member_profile") + ); + assert!( + audit + .iter() + .any(|entry| entry.action == "clear_default_member_profile") + ); + database.close().await.unwrap(); +} + #[tokio::test] async fn revoked_admin_session_cannot_commit_a_policy_mutation() { let database = database().await; From 4db0ead6e342342e99515caa4980e65d3eb2f661 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 13:46:58 +0000 Subject: [PATCH 40/62] Start closing default member policy plan 021 --- .../021-default-member-policy-provisioning.md | 2 +- plans/registry.md | 4 ++-- plans/subsystems/on-prem-user-access-roadmap.md | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md b/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md index c2f5d1b8..4e4f9910 100644 --- a/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md +++ b/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md @@ -1,6 +1,6 @@ # On-Prem User Access M021 — Default member policy and provisioning UX -Status: active +Status: closing Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index db193aa9..69b82e29 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -58,7 +58,7 @@ No new runtime, schema or performance claim lands with these plans. | Persistence | active | `plans/subsystems/persistence-roadmap.md` | M011 ready — post-M010 roadmap/documentation reconciliation | M009/M010 closures accepted. Documentation/tooling-only corrective; production remains WAL/NORMAL. No technical persistence successor is registered. | | Deployment and packaging | closed | `plans/subsystems/deployment-packaging-roadmap.md` | M005 closed — active PATH integration detection corrective | M001–M005 closed; no successor registered. | | Dashboard | closed | `plans/subsystems/dashboard-roadmap.md` | M012 closed — lifecycle closure and documentation polish | M001–M012 closed; no registered successor; future dashboard work requires a new bounded plan. | -| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M021/M022 ready — default member provisioning and hard-budget compatibility | M017–M020 closed; M023 retains its hard dependency/evidence gates. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | +| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M021 closing; M022 ready — default member provisioning and hard-budget compatibility | M017–M020 closed; M023 retains its hard dependency/evidence gates. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | | On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 ready — ordinary-server small-team synthetic capacity baseline | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 is independent measurement on ordinary on-prem Linux with durable SSD/NVMe (operational target required), not an authorization to change WAL/SQLite topology. | ## Dependency-ready implementation plans @@ -73,7 +73,7 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| -| On-prem user access | M021 — default member profile and provisioning | active | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | Hard M020 closure accepted; implement explicit deny-by-default profile provisioning. | +| On-prem user access | M021 — default member profile and provisioning | closing | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | Implementation landed; verifying the profile-default, no-profile, migration, and default/no-default contracts. | ## Blocked work diff --git a/plans/subsystems/on-prem-user-access-roadmap.md b/plans/subsystems/on-prem-user-access-roadmap.md index d43fd396..6a0394e9 100644 --- a/plans/subsystems/on-prem-user-access-roadmap.md +++ b/plans/subsystems/on-prem-user-access-roadmap.md @@ -457,6 +457,6 @@ A ready plan is not an implemented capability. | M018 | closed | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | `plans/closure/on-prem-user-access/018-status.md` | Private local control readback restores status after identity cutover; no HTTP privilege change; no successor unblocked. | | M019 | closed | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | `plans/closure/on-prem-user-access/019-status.md` | Unsafe remote HTTP login GET returns no credential form; M020 unblocked. | | M020 | closed | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | `plans/closure/on-prem-user-access/020-status.md` | Headless member lifecycle and first-member transition are atomic and audited; M021 unblocked. | -| M021 | ready | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | — | Hard dependency M020 is closed. | +| M021 | closing | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | — | Implementation landed; closure verification and dependency audit in progress. | | M022 | ready | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | — | Hard M017 closure is satisfied; preserve hard budget proof. | | M023 | blocked | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Hard: M022 closure; ordinary-server evidence operational. | From ee563be5b23059f0a14eb724bbaa9a675653d3d7 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 13:47:45 +0000 Subject: [PATCH 41/62] Verify default member CLI workflow --- rust/tests/on_prem_headless_admin.rs | 66 +++++++++++++++++++++++----- 1 file changed, 56 insertions(+), 10 deletions(-) diff --git a/rust/tests/on_prem_headless_admin.rs b/rust/tests/on_prem_headless_admin.rs index 8fa883ae..105b2ed2 100644 --- a/rust/tests/on_prem_headless_admin.rs +++ b/rust/tests/on_prem_headless_admin.rs @@ -112,16 +112,24 @@ async fn headless_member_lifecycle_is_atomic_audited_and_works_with_stale_config })); database.close().await.expect("database closes"); + let default_set = run_cli( + &config_path, + &["users", "defaults", "set", "restricted"], + None, + ) + .await; + assert!( + default_set.status.success(), + "{}", + String::from_utf8_lossy(&default_set.stderr) + ); + let default_shown = run_cli(&config_path, &["users", "defaults", "show"], None).await; + assert!(default_shown.status.success()); + assert!(String::from_utf8_lossy(&default_shown.stdout).contains("restricted")); + let created = run_cli( &config_path, - &[ - "users", - "add", - "member", - "--password-stdin", - "--profile-id", - "restricted", - ], + &["users", "add", "member", "--password-stdin"], Some("M020 temporary member password"), ) .await; @@ -164,6 +172,35 @@ async fn headless_member_lifecycle_is_atomic_audited_and_works_with_stale_config .contains("Previously revoked credentials stay revoked") ); + let no_profile = run_cli( + &config_path, + &[ + "users", + "add", + "explicit-none", + "--password-stdin", + "--no-profile", + ], + Some("M021 explicit no-profile password"), + ) + .await; + assert!( + no_profile.status.success(), + "{}", + String::from_utf8_lossy(&no_profile.stderr) + ); + let default_clear = run_cli(&config_path, &["users", "defaults", "clear"], None).await; + assert!( + default_clear.status.success(), + "{}", + String::from_utf8_lossy(&default_clear.stderr) + ); + let default_shown = run_cli(&config_path, &["users", "defaults", "show"], None).await; + assert!(default_shown.status.success()); + assert!( + String::from_utf8_lossy(&default_shown.stdout).contains("default_member_profile: none") + ); + let duplicate = run_cli( &config_path, &["users", "add", "member", "--password-stdin"], @@ -195,23 +232,32 @@ async fn headless_member_lifecycle_is_atomic_audited_and_works_with_stale_config .unwrap(), "disabled credentials remain revoked after reactivation" ); - let (assigned_profile, audit_actions) = database + let (assigned_profile, explicit_none_assignment, audit_actions) = database .call(move |connection| { let profile: String = connection.query_row( "SELECT profile_id FROM user_profile_assignments WHERE user_id = ?1", [&member.id], |row| row.get(0), )?; + let explicit_none: bool = connection.query_row( + "SELECT EXISTS(SELECT 1 FROM user_profile_assignments a JOIN users u ON u.id = a.user_id WHERE u.username = 'explicit-none')", + [], + |row| row.get(0), + )?; let mut statement = connection.prepare("SELECT action FROM user_admin_audit ORDER BY id")?; let actions = statement .query_map([], |row| row.get::<_, String>(0))? .collect::, _>>()?; - Ok((profile, actions)) + Ok((profile, explicit_none, actions)) }) .await .expect("profile and audit evidence are readable"); assert_eq!(assigned_profile, "restricted"); + assert!( + !explicit_none_assignment, + "--no-profile bypasses the configured default" + ); assert!( audit_actions .iter() From 3f1ca547c77d8594d6a1a95dab728d9d41c8adb9 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 13:48:24 +0000 Subject: [PATCH 42/62] Cover member default policy transition race --- rust/tests/on_prem_model_policy.rs | 66 ++++++++++++++++++++++++++++++ 1 file changed, 66 insertions(+) diff --git a/rust/tests/on_prem_model_policy.rs b/rust/tests/on_prem_model_policy.rs index 279e7434..1747390f 100644 --- a/rust/tests/on_prem_model_policy.rs +++ b/rust/tests/on_prem_model_policy.rs @@ -302,6 +302,72 @@ async fn default_profile_provisions_future_members_only_and_is_audited() { database.close().await.unwrap(); } +#[tokio::test] +async fn member_creation_racing_default_change_observes_one_complete_policy() { + let database = database().await; + identity::mark_fresh_uninitialized(&database).await.unwrap(); + let (admin, _) = identity::bootstrap_admin( + &database, + "admin-race", + "M021 strong administrator password", + ) + .await + .unwrap(); + identity::enable_managed(&database, &admin.id) + .await + .unwrap(); + let policy = ModelPolicyService::new(database.clone()); + policy + .create_profile("before", "Before", "deny", &admin.id) + .await + .unwrap(); + policy + .create_profile("after", "After", "allow", &admin.id) + .await + .unwrap(); + policy + .set_default_member_profile(Some("before"), &admin.id) + .await + .unwrap(); + + let (changed, created) = tokio::join!( + policy.set_default_member_profile(Some("after"), &admin.id), + identity::create_member_with_profile( + &database, + &admin.id, + "racing-member", + "M021 temporary password", + None, + ), + ); + changed.unwrap(); + let created = created.unwrap(); + let assigned = database + .call(move |connection| { + let mut statement = connection + .prepare("SELECT profile_id FROM user_profile_assignments WHERE user_id = ?1")?; + statement + .query_map([created.id], |row| row.get::<_, String>(0))? + .collect::, _>>() + }) + .await + .unwrap(); + assert_eq!(assigned.len(), 1); + assert!(assigned[0] == "before" || assigned[0] == "after"); + let member_policy = policy + .snapshot( + &identity::get_user_by_name(&database, "racing-member") + .await + .unwrap() + .unwrap() + .id, + ) + .await + .unwrap(); + assert!(member_policy.has_profile()); + database.close().await.unwrap(); +} + #[tokio::test] async fn revoked_admin_session_cannot_commit_a_policy_mutation() { let database = database().await; From aa59f1d622a0fec08285b3ceb5e167ca7302a402 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 13:55:40 +0000 Subject: [PATCH 43/62] Close default member policy plan 021 --- .../closure/on-prem-user-access/021-status.md | 157 ++++++++++++++++++ .../021-default-member-policy-provisioning.md | 2 +- plans/registry.md | 6 +- .../subsystems/on-prem-user-access-roadmap.md | 17 +- 4 files changed, 171 insertions(+), 11 deletions(-) create mode 100644 plans/closure/on-prem-user-access/021-status.md diff --git a/plans/closure/on-prem-user-access/021-status.md b/plans/closure/on-prem-user-access/021-status.md new file mode 100644 index 00000000..e52be7a2 --- /dev/null +++ b/plans/closure/on-prem-user-access/021-status.md @@ -0,0 +1,157 @@ +# On-Prem User Access M021 — Closure Status + +Status: closed + +Source implementation plan: + +- `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` + +Source subsystem roadmap: + +- `plans/subsystems/on-prem-user-access-roadmap.md#m021--default-member-policy-and-provisioning-ux` + +Repository baseline reviewed: post-merge `88f58bbf`; implementation commits +`b76d3590`, `ee563be`, and `3f1ca54` after M020 closure. Status transitions: +`2e99cdd` activation, `4db0ead` active → closing. + +## 1. Executive finding + +One audited instance-local default profile now applies transactionally to +future member creation through both the local CLI and browser admin. Explicit +profile selection takes precedence, and an explicit no-profile choice bypasses +the default. Setting or clearing the default never edits existing assignments. +When no profile is selected, members remain deny-by-default; a single +administrator without a profile retains the existing unrestricted behavior. + +This pass does not copy budget rules or introduce a default budget template. +Per-user and instance budgets remain separate and unchanged. + +## 2. Requirement-to-evidence matrix + +| Requirement | Evidence | Result | Notes | +|---|---|---|---| +| Select or clear one default profile with audit and policy revision invalidation | `ModelPolicyService::set_default_member_profile`; `policy_state_default_member_profile_revision_update`; `on_prem_model_policy::default_profile_provisions_future_members_only_and_is_audited` | pass | Selection and clearing require an active admin; browser mutations also revalidate the session inside the write transaction. Repeating the current choice is idempotent. | +| Apply defaults atomically to future CLI and browser members | `identity::create_member_with_fence`; `on_prem_headless_admin::headless_member_lifecycle_is_atomic_audited_and_works_with_stale_config`; `on_prem_admin_ui::two_user_browser_journey_fences_admin_data_and_personal_credentials` | pass | The default read, member insert, assignment, access transition, and audit writes share one SQLite transaction. CLI and browser paths exercise default selection. | +| Explicit profile and no-profile choices override the configured default | `users add --profile-id` / `--no-profile`; browser `profile_id` handling; `on_prem_headless_admin`; `on_prem_admin_ui`; `cli_contract` | pass | Both CLI and browser integration journeys prove the explicit no-profile path does not inherit the default. | +| Existing users are not retroactively changed; concurrent changes are consistent | `on_prem_model_policy::default_profile_provisions_future_members_only_and_is_audited`; `on_prem_model_policy::member_creation_racing_default_change_observes_one_complete_policy` | pass | Existing assignments remain empty when a default is added; concurrent creation sees exactly one complete old-or-new profile assignment. | +| Profile policy and instance denials still govern visible and requested models | `on_prem_model_policy` qualified allow/deny precedence; `on_prem_admin_ui`; existing coordinator policy admission tests | pass | Provider-qualified denial remains in force. Generic model-forbidden response behavior is unchanged. | +| Empty member model lists explain setup or policy state without private upstream details | `server/dashboard/admin.rs::my_models_page` | pass | Distinguishes no instance models, no member profile, and a profile with no available grants; renders no provider/account data. | +| Existing admin-only personal mode and budgets are unchanged | `on_prem_model_policy`; `on_prem_budgets`; `on_prem_global_budgets`; model-limits docs | pass | No default budget template is added or copied. | +| Migration is additive, checksummed, and default-null | migration `0063_default_member_profile.sql`; checksums/runtime manifest; `database_compatibility::fresh_database_migrates_idempotently_and_preserves_pragmas`; upgrade compatibility target | pass | Existing rows receive no profile value or reassignment. Historical migrations remain immutable. | + +## 3. Production implementation evidence + +Migration 0063 adds nullable `policy_state.default_member_profile_id` with a +restricting profile foreign key and a revision trigger. Policy administration +lists the selected profile, audits set/clear, increments the shared policy +revision, and prevents deleting a selected default. + +`identity.rs` resolves the default in the same transaction as member creation +and its audit rows. The CLI adds `users defaults show|set|clear` and +`users add --no-profile`; the admin UI can use the default, choose a named +profile, or explicitly leave the member unassigned. The member model page +reports safe empty-catalog explanations. + +## 4. Verification executed + +### Commands run + +```bash +rtk cargo fmt --manifest-path rust/Cargo.toml --all -- --check +rtk cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings +rtk cargo check --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features +rtk cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- -D warnings +rtk cargo test --manifest-path rust/Cargo.toml --workspace --all-targets -- --test-threads=1 +rtk cargo test --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- --test-threads=1 +rtk cargo test --manifest-path rust/Cargo.toml --test on_prem_model_policy -- --test-threads=1 +rtk cargo test --manifest-path rust/Cargo.toml --test on_prem_headless_admin -- --test-threads=1 +rtk cargo test --manifest-path rust/Cargo.toml --test on_prem_admin_ui -- --test-threads=1 +rtk cargo test --manifest-path rust/Cargo.toml --test database_compatibility -- --test-threads=1 +rtk cargo test --manifest-path rust/Cargo.toml --test cli_contract -- --test-threads=1 +rtk uv run ruff format --check scripts/ tests/tooling/ +rtk uv run ruff check scripts/ tests/tooling/ +rtk uv run pyright scripts/ +rtk uv run pytest tests/tooling/ -q --tb=short --maxfail=1 +rtk uv run python scripts/validate_release_docs.py +rtk uv run python scripts/validate_runtime_package_boundary.py +rtk git diff --check +``` + +### Results + +- Default workspace: 1,037 passed, 1 ignored across 80 suites. +- No-default workspace: 1,038 passed, 1 ignored across 80 suites. +- Focused final targets: policy 6, headless admin 2, browser admin 3, + database compatibility 11, and CLI contract 7 passed. +- Strict Clippy passed with and without default features; no-default workspace + check passed; formatting passed. +- Tooling: Ruff format/check passed, Pyright reported 0 errors, and pytest + reported 192 passed and 1 skipped. +- Release-doc and runtime-package-boundary validators passed; `git diff --check` + passed. + +## 5. Invariant review + +- No profile remains the safe deny-by-default member fallback; the existing + administrator default is unchanged. +- Instance denials override profile or user allows; model identifiers remain + exact and provider-qualified checks remain enforced. +- Defaults apply only to future members. Revision changes refresh new policy + snapshots without mutating admitted snapshots or existing assignments. +- No provider secrets, prompts, request bodies, or hidden upstream facts enter + the new policy state, audit records, or member page. +- No request-time network or DB hold across provider I/O was added. + +## 6. Failure and recovery review + +Default lookup, member creation, optional assignment, access transition, and +audit are one SQLite transaction. A failed assignment rolls back the member +and transition as covered by the headless admin lifecycle test. Concurrent +default mutation and creation serialize at the existing SQLite write gate and +the race test accepts only one complete old-or-new assignment. Clearing is +idempotent; a selected profile cannot be removed before clearing it. + +## 7. Migration and compatibility review + +Migration 0063 is append-only and nullable, with a checksum entry and runtime +asset hash update. It does not backfill existing user assignments. Fresh +databases have no default selected. Compatibility tests cover historical and +production-schema forward upgrades through version 63, and the no-default +build retains the same identity/policy behavior. + +## 8. Security review + +CLI writes require an active administrator actor. Browser writes require +same-origin CSRF validation, administrator role, and an in-transaction session +fence. Invalid profile IDs fail without partial member creation. Empty-model +messages do not expose provider topology, accounts, or credentials. Per-user +and instance budget boundaries remain unchanged. + +## 9. Documentation and operations + +Updated `docs/deployment.md`, `docs/cli-reference.md`, +`docs/model-routing.md`, `docs/model-limits.md`, `docs/backup-restore.md`, +`architecture/deep-dive-security.md`, and +`architecture/deep-dive-database.md`. CLI and browser help explain future-only +default behavior and separate budget configuration. + +## 10. Unresolved findings + +| Severity | Finding | Impact | Required action | +|---|---|---|---| +| none | None | No known correctness, security, compatibility, or operational issue remains within M021. | None | + +## 11. Roadmap disposition + +M021 is closed. M022 was already ready because its hard M017 dependency had +closed; this closure does not add a new dependency or change its readiness. +M023 remains blocked on M022 closure and ordinary-server operational evidence +from the independent on-prem concurrency M002 workstream. M002 is registered +ready but no qualifying ordinary Linux/SSD-or-NVMe evidence was added here. + +## 12. Registry updates + +In the closure commit, M021 moves `closing` → `implemented` and is added to +recent closures in `plans/registry.md`; its roadmap row moves to `closed`. +M022 remains ready and is the next plan in the user-requested sequence. M023 +remains blocked; no newly-ready plan was found by the closure audit. diff --git a/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md b/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md index 4e4f9910..4b568f06 100644 --- a/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md +++ b/plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md @@ -1,6 +1,6 @@ # On-Prem User Access M021 — Default member policy and provisioning UX -Status: closing +Status: implemented Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index 69b82e29..14417c98 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -58,7 +58,7 @@ No new runtime, schema or performance claim lands with these plans. | Persistence | active | `plans/subsystems/persistence-roadmap.md` | M011 ready — post-M010 roadmap/documentation reconciliation | M009/M010 closures accepted. Documentation/tooling-only corrective; production remains WAL/NORMAL. No technical persistence successor is registered. | | Deployment and packaging | closed | `plans/subsystems/deployment-packaging-roadmap.md` | M005 closed — active PATH integration detection corrective | M001–M005 closed; no successor registered. | | Dashboard | closed | `plans/subsystems/dashboard-roadmap.md` | M012 closed — lifecycle closure and documentation polish | M001–M012 closed; no registered successor; future dashboard work requires a new bounded plan. | -| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M021 closing; M022 ready — default member provisioning and hard-budget compatibility | M017–M020 closed; M023 retains its hard dependency/evidence gates. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | +| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M022 ready — hard-budget compatibility; M021 closed | M017–M021 closed; M023 retains its hard dependency/evidence gates. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | | On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 ready — ordinary-server small-team synthetic capacity baseline | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 is independent measurement on ordinary on-prem Linux with durable SSD/NVMe (operational target required), not an authorization to change WAL/SQLite topology. | ## Dependency-ready implementation plans @@ -73,7 +73,6 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| -| On-prem user access | M021 — default member profile and provisioning | closing | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | Implementation landed; verifying the profile-default, no-profile, migration, and default/no-default contracts. | ## Blocked work @@ -112,6 +111,7 @@ Historical M007 blocker assessment at baseline `7e241ad` (`plans/closure/persist | Subsystem / plan | Disposition | Evidence | |---|---|---| +| On-prem user access M021 — default member policy and provisioning | closed — transactional future-only default profile selection, CLI/browser creation, explicit no-profile override, additive schema 63 migration, policy visibility and budget-separation behavior pass; M022 remains ready; M023 remains blocked on M022 and ordinary-server evidence | `plans/closure/on-prem-user-access/021-status.md`, implementation `b76d359`, CLI journey `ee563be`, race test `3f1ca54` | | On-prem user access M020 — managed transition and headless administration | closed — first member transition, profile assignment and audit are atomic; headless add/show/disable/enable workflow and stale-config auth rejection pass; M021 promoted to ready | `plans/closure/on-prem-user-access/020-status.md`, implementation `8a875a1` | | On-prem user access M019 — login transport and solo LAN operability corrective | closed — remote plaintext login GET returns no credential form; loopback and configured HTTPS-proxy paths pass; solo SSH tunnel and team proxy guidance documented; M020 promoted to ready | `plans/closure/on-prem-user-access/019-status.md`, implementation `b18a561` | | On-prem user access M018 — local operator status corrective | closed — status and runtime status use bounded private control-socket readback without a server API key; managed HTTP authorization is unchanged; no successor unblocked | `plans/closure/on-prem-user-access/018-status.md`, implementation `8e2b66a`, CLI fixture `3be5d83` | @@ -535,3 +535,5 @@ Unblock audit (on-prem user access M018 closed, `plans/closure/on-prem-user-acce Unblock audit (on-prem user access M019 closed, `plans/closure/on-prem-user-access/019-status.md`): searched the on-prem implementation plans, roadmap dependency graph, and dependency-ready, active, and blocked registry tables. M020's hard dependencies M017 and M019 are both closed, so M020 moves `blocked` → `ready`. M021 remains blocked on M020. M023 remains blocked on M022 and ordinary-server operational evidence. M022 remains ready; no other plan is promoted and no capacity claim is introduced. Unblock audit (on-prem user access M020 closed, `plans/closure/on-prem-user-access/020-status.md`): searched the on-prem implementation plans, roadmap dependency graph, and dependency-ready, active, and blocked registry tables. M021's sole hard dependency, M020, is closed, so M021 moves `blocked` → `ready`. M023 remains blocked on M022 and ordinary-server operational evidence; M022 remains ready. No other plan is promoted and no team-capacity claim is introduced. + +Unblock audit (on-prem user access M021 closed, `plans/closure/on-prem-user-access/021-status.md`): searched the on-prem implementation plans, roadmap dependency graph, and dependency-ready, active, and blocked registry tables. M022 was already ready after M017 closed and remains ready; M021 was not a prerequisite. M023 remains blocked on hard M022 closure and the operational ordinary-server M002 evidence; M002's ready status does not satisfy that evidence gate. No newly-ready plan is promoted and no team-capacity claim is introduced. diff --git a/plans/subsystems/on-prem-user-access-roadmap.md b/plans/subsystems/on-prem-user-access-roadmap.md index 6a0394e9..24782aeb 100644 --- a/plans/subsystems/on-prem-user-access-roadmap.md +++ b/plans/subsystems/on-prem-user-access-roadmap.md @@ -330,12 +330,13 @@ CLI create/disable/enable/show member workflows; preserve audit, CSRF, password-stdin, last-admin and atomic creation guarantees. Source: `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md`. -#### M021 — Default member policy and provisioning UX +#### M021 — Default member policy and provisioning UX (closed) -Class: capability. Ready after hard M020 closure. Add explicit reusable -member default profile (and optional budget template) with safe fallback, -transactional assignment, admin/CLI consistency and understandable -catalog-empty/deny feedback. Never grant unaudited model access. +Class: capability. Closed after hard M020 closure. One audited, instance-local +default profile applies transactionally to future members only, with explicit +profile and no-profile choices, admin/CLI consistency, and understandable +catalog-empty/deny feedback. No budget template is copied; existing budget +scopes remain independently configured. Never grant unaudited model access. Source: `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md`. #### M022 — Hard-budget agent interoperability @@ -356,8 +357,8 @@ Source: `plans/implementation/on-prem-user-access/023-short-window-fairness-cont ### Post-merge dependency and hardware policy -- M017–M020 are closed. M021 and M022 are ready; neither depends on a Pi/Le - Potato run. M022 became ready after the M017 closure. +- M017–M021 are closed. M022 is ready and does not depend on a Pi/Le Potato + run; it became ready after the M017 closure. - Hard: M017 + M019 -> M020 -> M021; M017 -> M022 -> M023. - Interface: M018's privileged local transport must not broaden HTTP rights; any new auth authority stops for ADR review. M022 hard-cost proof contract @@ -457,6 +458,6 @@ A ready plan is not an implemented capability. | M018 | closed | `plans/implementation/on-prem-user-access/018-local-operator-status-corrective.md` | `plans/closure/on-prem-user-access/018-status.md` | Private local control readback restores status after identity cutover; no HTTP privilege change; no successor unblocked. | | M019 | closed | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | `plans/closure/on-prem-user-access/019-status.md` | Unsafe remote HTTP login GET returns no credential form; M020 unblocked. | | M020 | closed | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | `plans/closure/on-prem-user-access/020-status.md` | Headless member lifecycle and first-member transition are atomic and audited; M021 unblocked. | -| M021 | closing | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | — | Implementation landed; closure verification and dependency audit in progress. | +| M021 | closed | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | `plans/closure/on-prem-user-access/021-status.md` | Transactional future-only profile default; CLI/browser and no-profile journeys, migration, policy and budget boundaries verified. | | M022 | ready | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | — | Hard M017 closure is satisfied; preserve hard budget proof. | | M023 | blocked | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Hard: M022 closure; ordinary-server evidence operational. | From 1406f3cc271460dae5ae256d2ec8441d9b4c7198 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 13:56:04 +0000 Subject: [PATCH 44/62] Activate hard-budget compatibility plan 022 --- .../022-hard-budget-agent-compatibility.md | 2 +- plans/registry.md | 4 ++-- plans/subsystems/on-prem-user-access-roadmap.md | 6 +++--- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md b/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md index a4ec58c7..52f02cbf 100644 --- a/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md +++ b/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md @@ -1,6 +1,6 @@ # On-Prem User Access M022 — Hard-budget interoperability with coding agents -Status: ready +Status: active Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index 14417c98..3294912e 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -58,7 +58,7 @@ No new runtime, schema or performance claim lands with these plans. | Persistence | active | `plans/subsystems/persistence-roadmap.md` | M011 ready — post-M010 roadmap/documentation reconciliation | M009/M010 closures accepted. Documentation/tooling-only corrective; production remains WAL/NORMAL. No technical persistence successor is registered. | | Deployment and packaging | closed | `plans/subsystems/deployment-packaging-roadmap.md` | M005 closed — active PATH integration detection corrective | M001–M005 closed; no successor registered. | | Dashboard | closed | `plans/subsystems/dashboard-roadmap.md` | M012 closed — lifecycle closure and documentation polish | M001–M012 closed; no registered successor; future dashboard work requires a new bounded plan. | -| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M022 ready — hard-budget compatibility; M021 closed | M017–M021 closed; M023 retains its hard dependency/evidence gates. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | +| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M022 active — hard-budget agent compatibility; M021 closed | M017–M021 closed; M023 retains its hard dependency/evidence gates. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | | On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 ready — ordinary-server small-team synthetic capacity baseline | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 is independent measurement on ordinary on-prem Linux with durable SSD/NVMe (operational target required), not an authorization to change WAL/SQLite topology. | ## Dependency-ready implementation plans @@ -66,13 +66,13 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Dependencies / handoff note | |---|---|---|---|---| | Persistence | M011 — post-M010 roadmap and documentation reconciliation | ready | `plans/implementation/persistence/011-post-m010-roadmap-documentation-reconciliation.md` | Hard: M009/M010 closures accepted. Documentation/tooling only; no runtime or physical target dependency. | -| On-prem user access | M022 — hard-budget agent interoperability | ready | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | Hard M017 is closed; preserve M008/M015 proof-or-reject guarantees. | | On-prem concurrency performance | M002 — ordinary-server team baseline | ready | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | Independent evidence work; ordinary Linux + durable SSD/NVMe required for team-capacity claims, no Pi/microSD release gate. | ## Active implementation plans | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| +| On-prem user access | M022 — hard-budget agent interoperability | active | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | Hard M017 closure is satisfied; preserve M008/M015 proof-or-reject guarantees. | ## Blocked work diff --git a/plans/subsystems/on-prem-user-access-roadmap.md b/plans/subsystems/on-prem-user-access-roadmap.md index 24782aeb..d4617bd2 100644 --- a/plans/subsystems/on-prem-user-access-roadmap.md +++ b/plans/subsystems/on-prem-user-access-roadmap.md @@ -339,9 +339,9 @@ catalog-empty/deny feedback. No budget template is copied; existing budget scopes remain independently configured. Never grant unaudited model access. Source: `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md`. -#### M022 — Hard-budget agent interoperability +#### M022 — Hard-budget agent interoperability (active) -Class: polish. Ready after hard M017 closure. Measure whether client/surface +Class: polish. Active after hard M017 closure. Measure whether client/surface requests carry sufficient output-token proof; improve diagnostics and introduce output-cap insertion only where upstream and wire evidence prove a hard bound. Do not regress M008/M015 hard-cost safety. @@ -459,5 +459,5 @@ A ready plan is not an implemented capability. | M019 | closed | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | `plans/closure/on-prem-user-access/019-status.md` | Unsafe remote HTTP login GET returns no credential form; M020 unblocked. | | M020 | closed | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | `plans/closure/on-prem-user-access/020-status.md` | Headless member lifecycle and first-member transition are atomic and audited; M021 unblocked. | | M021 | closed | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | `plans/closure/on-prem-user-access/021-status.md` | Transactional future-only profile default; CLI/browser and no-profile journeys, migration, policy and budget boundaries verified. | -| M022 | ready | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | — | Hard M017 closure is satisfied; preserve hard budget proof. | +| M022 | active | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | — | Measuring client output-cap evidence while preserving hard proof-or-reject semantics. | | M023 | blocked | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Hard: M022 closure; ordinary-server evidence operational. | From dee65ae51a7f5fad5f684b8123d9dee8a585e2b4 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 14:23:35 +0000 Subject: [PATCH 45/62] Improve hard budget agent compatibility --- architecture/deep-dive-request-lifecycle.md | 2 +- docs/agent-configuration.md | 9 ++ docs/api-reference.md | 9 +- docs/model-limits.md | 31 ++++++- .../022-hard-budget-agent-compatibility.md | 8 +- rust/src/coordinator/endpoints.rs | 32 ++++++- rust/src/coordinator/mod.rs | 5 +- rust/src/server/inference.rs | 4 +- .../on_prem_budget_agent_compatibility.json | 29 +++++++ rust/tests/on_prem_budgets.rs | 83 ++++++++++++++++++- 10 files changed, 190 insertions(+), 22 deletions(-) create mode 100644 rust/tests/fixtures/on_prem_budget_agent_compatibility.json diff --git a/architecture/deep-dive-request-lifecycle.md b/architecture/deep-dive-request-lifecycle.md index 02401efb..4a02f8e5 100644 --- a/architecture/deep-dive-request-lifecycle.md +++ b/architecture/deep-dive-request-lifecycle.md @@ -147,7 +147,7 @@ adapts the response, and converges durable state. caller's explicit compatible output cap (`max_tokens`/`max_completion_tokens` on chat, `max_output_tokens` on Responses/compact, `max_tokens` on Anthropic Messages) converted at the operator-configured provider tariff — and deny - with `BudgetNotEnforceable` (429, no upstream submission) when unbounded, + with `BudgetNotEnforceable` (429, no upstream submission) when unbounded, unpriced, or overflowing. The initial hard hold covers two attempt equivalents (concrete attempt plus unattributed selector overhead); retries extend by one equivalent after re-checking. Observed provider usage above diff --git a/docs/agent-configuration.md b/docs/agent-configuration.md index 9546fe1b..40597902 100644 --- a/docs/agent-configuration.md +++ b/docs/agent-configuration.md @@ -8,6 +8,15 @@ Codex and OpenCode additionally support a managed lifecycle that installs a gene ## Supported Targets +For hard token budgets, clients must send an explicit output cap supported by +their API surface. OpenCode sends its configured model `limit.output` as +`max_output_tokens` on Responses requests; set that model limit when using a +hard token budget. The tested Codex CLI 0.162.1 Responses request did not +include an output cap, so it is rejected before upstream dispatch when a hard +token rule applies. EggPool does not invent an output cap. See the +[hard-budget compatibility matrix](model-limits.md#coding-agent-compatibility) +for supported field names and capture details. + | Target | Command | Output Format | `--write` Default | Model | |--------|---------|---------------|-------------------|-------| | OpenCode | `eggpool configsetup opencode` | JSON provider config (Responses runtime) | N/A (clipboard) | auto (all models) | diff --git a/docs/api-reference.md b/docs/api-reference.md index 65ea7807..9bc14e35 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -128,10 +128,11 @@ rules require an explicit caller output cap; USD rules require an operator-configured provider tariff) and deny otherwise, while `estimated` rules use approximate holds and are never a guaranteed cap. This API does not represent invoices or exact charges for flat-rate provider accounts. -Requests denied because a hard rule cannot be bounded return HTTP 429 with a -stable machine-readable message -(`User budget hard limit is not enforceable for this request`) before any -upstream submission. Administrator responses additionally include +Requests denied because a hard rule cannot be bounded return HTTP 429 with +machine code `hard_budget_not_enforceable` before any upstream submission. +The response gives corrective guidance for an explicit output cap and, when +applicable, an operator tariff. Exhausted rules use the code +`budget_exceeded`. Administrator responses additionally include `instance_rules` and `instance_usage`: the single instance-wide ceiling intersected with every per-user rule. Members never receive instance fields, so one member cannot observe the shared ceiling or infer other users' diff --git a/docs/model-limits.md b/docs/model-limits.md index 09578cdf..0459b2bf 100644 --- a/docs/model-limits.md +++ b/docs/model-limits.md @@ -146,10 +146,33 @@ nothing is silently upgraded to a guarantee. max_microdollars_per_million_tokens`); unpriced providers fail closed, an explicit zero tariff means verified free, and estimator fallbacks are never a hard price. Requests that cannot be bounded — unbounded output, unknown - tariff, arithmetic overflow — are denied with a stable 429 - (`User budget hard limit is not enforceable for this request`) before any - upstream submission. Observed provider usage above a hard hold is settled - visibly, never clamped. + tariff, arithmetic overflow — are denied with HTTP 429 and machine code + `hard_budget_not_enforceable` before any upstream submission. The response + advises callers to include a positive output cap supported by the API and + operators to configure a tariff when hard USD rules apply. `budget_exceeded` + identifies an exhausted rule. Observed provider usage above a hard hold is + settled visibly, never clamped. + +### Coding-agent compatibility + +The supported request fields used as output bounds are: + +| API surface | Explicit output bound fields | +|-------------|-------------------------------| +| Chat Completions | `max_completion_tokens`, `max_tokens` | +| Responses | `max_output_tokens`, `max_completion_tokens`, `max_tokens` | +| Compact Responses | `max_output_tokens`, `max_completion_tokens`, `max_tokens` | +| Anthropic Messages | `max_tokens` | + +Synthetic loopback captures of Codex CLI 0.162.1 and OpenCode 1.18.35 +confirmed that behavior depends on client model configuration: Codex CLI sent +a streaming Responses request without an output cap, while OpenCode sent +`max_output_tokens: 1024` from its configured `limit.output`. The sanitized +capture record is in +[`rust/tests/fixtures/on_prem_budget_agent_compatibility.json`](../rust/tests/fixtures/on_prem_budget_agent_compatibility.json). +An unbounded request continues to fail before dispatch under hard token rules. +For OpenCode, configure a model output limit so its Responses request carries +the bound. EggPool does not inject or infer a cap for any client. See the [API reference](api-reference.md#authentication) for the authenticated `/api/user-budget` view. diff --git a/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md b/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md index 52f02cbf..b3363f46 100644 --- a/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md +++ b/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md @@ -22,7 +22,7 @@ Hard: M017 client credential/replay integration must close first. Interface: exi ## 3. Current evidence -`rust/src/user_budget.rs` and `rust/src/instance_budget.rs` reserve worst-case hard costs; `docs/model-limits.md` documents proof-or-reject semantics. Hard token rules require explicit caller output cap. Hard USD rules also require conservative operator tariff for all eligible providers. Some supported clients may omit compatible output-cap fields for one or more OpenAI Responses, Chat, Anthropic Messages or compact surfaces; those requests correctly fail before dispatch. M015 corrected under-reserved hard-dollar proof, so a relaxed estimated fallback would regress security. +`rust/src/user_budget.rs` and `rust/src/instance_budget.rs` reserve worst-case hard costs; `docs/model-limits.md` documents proof-or-reject semantics. Hard token rules require an explicit caller output cap. Hard USD rules also require a conservative operator tariff for all eligible providers. Synthetic loopback request captures against a local mock server used Codex CLI 0.162.1 and OpenCode 1.18.35: Codex sent a streaming Responses request without an output cap, while OpenCode sent `max_output_tokens: 1024` from its configured model `limit.output`. The mock returned synthetic HTTP 400; only endpoint, version, stream and cap fields were retained. No prompt, request body, key, or provider call was used. The sanitized evidence is in `rust/tests/fixtures/on_prem_budget_agent_compatibility.json`. M015 corrected under-reserved hard-dollar proof, so a relaxed estimated fallback would regress security. ## 4. Invariants @@ -34,7 +34,7 @@ In: reproduce which clients/protocols omit bounds, clearer budget failure diagno ## 6. Production changes -Start by creating a protocol/client matrix with OpenCode, Codex and other supported clients using controlled fixture requests. Identify exact native output-limit fields. Investigate an explicit per-model/provider operator maximum output cap that the adapter can enforce in upstream request payload and prove across native/transcoded/streaming/compact paths. If impossible on a surface, preserve rejection and add clear guidance. Do not add a cross-client injected limit just to improve pass rates. +Create a protocol/client matrix with controlled synthetic requests and identify the accepted output-limit fields for Chat Completions, Responses, compact Responses and Anthropic Messages. Existing request admission and `BudgetProof` consume these fields without mutating the request. Do not add an operator cap: there is no protocol-wide evidence that an inferred limit would be preserved across all upstreams, wire translations and retries. Preserve rejection for clients omitting a bound and give a corrective hint. ## 7. Ordered work packages @@ -52,9 +52,9 @@ Acceptance: adversarial proof test for every affected surface, including malform ### C — UX and failure mapping -Return stable sanitized reason codes and meaningful corrective hints for missing output cap, unknown tariff, exhausted budget and concurrency. Preserve correct `Retry-After` behavior only for retryable conditions. Surface non-priced private-model token/request limits accurately. +Return stable sanitized codes for an exhausted budget (`budget_exceeded`) and a hard proof failure (`hard_budget_not_enforceable`), with a corrective hint for explicit output caps and operator tariffs. The current transaction result intentionally does not distinguish a missing token bound from an unknown tariff or arithmetic overflow, so the message describes both safe client/operator remedies without exposing rule contents. No `Retry-After` is added: these are policy/proof decisions rather than transient waits. Preserve non-priced private-model token/request limits. -Acceptance: a member can identify whether to change the client request, budget or provider tariff without seeing another user's budgets or accounts. +Acceptance: a member sees a stable machine code and can identify the client-request or operator-tariff correction without seeing another user's budgets or accounts. ## 8. Failure, cancellation, restart and contention diff --git a/rust/src/coordinator/endpoints.rs b/rust/src/coordinator/endpoints.rs index 7c7bc4f1..1984aede 100644 --- a/rust/src/coordinator/endpoints.rs +++ b/rust/src/coordinator/endpoints.rs @@ -119,7 +119,9 @@ pub enum EndpointError { ModelForbidden, #[error("User budget limit reached")] BudgetExceeded, - #[error("User budget hard limit is not enforceable for this request")] + #[error( + "Hard budget cannot be enforced: include a positive output cap supported by this API, and configure an operator tariff when a hard USD limit applies" + )] BudgetNotEnforceable, #[error("Routing claim failed")] Claim, @@ -218,6 +220,34 @@ pub fn endpoint_error_body(surface: ClientSurface, message: &str) -> Vec { body.into_iter().take(MAX_CLIENT_ERROR_BYTES).collect() } +/// Render stable machine-readable codes for budget decisions while keeping +/// the generic envelope used by other endpoint errors. +pub fn endpoint_error_body_for(surface: ClientSurface, error: &EndpointError) -> Vec { + let budget_code = match error { + EndpointError::BudgetExceeded => Some(("budget_exceeded", "User budget limit reached")), + EndpointError::BudgetNotEnforceable => Some(( + "hard_budget_not_enforceable", + "Hard budget cannot be enforced: include a positive output cap supported by this API, and configure an operator tariff when a hard USD limit applies", + )), + _ => None, + }; + let Some((code, message)) = budget_code else { + return endpoint_error_body(surface, &error.to_string()); + }; + let value = if surface == ClientSurface::Messages { + json!({ + "type": "error", + "error": {"type": "api_error", "message": message, "code": code} + }) + } else { + json!({ + "error": {"message": message, "type": "upstream_error", "code": code} + }) + }; + serde_json::to_vec(&value) + .unwrap_or_else(|_| b"{\"error\":{\"type\":\"upstream_error\"}}".to_vec()) +} + /// Validate the stateless Responses contract (Python parity). /// /// Returns a rejection message when stateful fields are present; `None` diff --git a/rust/src/coordinator/mod.rs b/rust/src/coordinator/mod.rs index 2b31c38d..491fe26f 100644 --- a/rust/src/coordinator/mod.rs +++ b/rust/src/coordinator/mod.rs @@ -61,8 +61,9 @@ pub use streaming::{ pub use endpoints::{ EndpointError, EndpointExecution, InferenceOperation, InferenceOutcome, InferenceState, ResolvedInference, VirtualResolution, build_stream_response_headers, endpoint_error_body, - execute_compact_finite, execute_endpoint, execute_finite, execute_stream, new_proxy_request_id, - parse_provider_qualified_model, validate_responses_stateless, + endpoint_error_body_for, execute_compact_finite, execute_endpoint, execute_finite, + execute_stream, new_proxy_request_id, parse_provider_qualified_model, + validate_responses_stateless, }; pub(crate) use endpoints::{ build_inference_state_with_shared, build_inference_state_with_shared_and_accounts, diff --git a/rust/src/server/inference.rs b/rust/src/server/inference.rs index 1d150500..c24a3555 100644 --- a/rust/src/server/inference.rs +++ b/rust/src/server/inference.rs @@ -152,7 +152,7 @@ pub(super) async fn handle_finite_compact( } Err(error) => { let status = error.status(); - let detail = endpoint_error_body(surface, &error.to_string()); + let detail = crate::coordinator::endpoint_error_body_for(surface, &error); error_body_response(status, surface, detail) } } @@ -191,7 +191,7 @@ pub(super) async fn handle_inference( } Err(error) => { let status = error.status(); - let detail = endpoint_error_body(surface, &error.to_string()); + let detail = crate::coordinator::endpoint_error_body_for(surface, &error); error_body_response(status, surface, detail) } } diff --git a/rust/tests/fixtures/on_prem_budget_agent_compatibility.json b/rust/tests/fixtures/on_prem_budget_agent_compatibility.json new file mode 100644 index 00000000..d336f483 --- /dev/null +++ b/rust/tests/fixtures/on_prem_budget_agent_compatibility.json @@ -0,0 +1,29 @@ +{ + "capture_method": "loopback synthetic request capture; retained fields only; no request bodies, prompts, keys, or provider calls", + "mock_response": "HTTP 400", + "clients": [ + { + "name": "Codex CLI", + "version": "0.162.1", + "surface": "responses", + "stream": true, + "output_cap": null, + "budget_compatibility": "unbounded; EggPool hard rules reject before dispatch under the proof contract" + }, + { + "name": "OpenCode", + "version": "1.18.35", + "surface": "responses", + "stream": true, + "output_cap": 1024, + "source": "configured model limit.output", + "hard_token_budget": "bounded_by_explicit_request_cap" + } + ], + "surface_fields": { + "chat_completions": ["max_completion_tokens", "max_tokens"], + "responses": ["max_output_tokens", "max_completion_tokens", "max_tokens"], + "compact_responses": ["max_output_tokens", "max_completion_tokens", "max_tokens"], + "anthropic_messages": ["max_tokens"] + } +} diff --git a/rust/tests/on_prem_budgets.rs b/rust/tests/on_prem_budgets.rs index 022f8bd1..1588b9fd 100644 --- a/rust/tests/on_prem_budgets.rs +++ b/rust/tests/on_prem_budgets.rs @@ -7,7 +7,7 @@ use eggpool::{ coordinator::{ CrashReconciler, DurableFinalizer, EndpointError, FinalizationData, FinalizationError, FinalizationOutcome, PublicationError, PublicationFaultInjector, PublicationInput, - PublicationOutcome, PublicationService, PublicationStage, endpoint_error_body, + PublicationOutcome, PublicationService, PublicationStage, endpoint_error_body_for, }, db::{Account, Database, DatabaseConfig, MigrationRunner}, identity::{AuthenticatedPrincipal, PrincipalKind}, @@ -957,14 +957,29 @@ async fn endpoint_error_maps_not_enforceable_to_stable_429() { assert_eq!(error.status().as_u16(), 429); assert_eq!( error.to_string(), - "User budget hard limit is not enforceable for this request" + "Hard budget cannot be enforced: include a positive output cap supported by this API, and configure an operator tariff when a hard USD limit applies" ); - let body = endpoint_error_body(ClientSurface::ChatCompletions, &error.to_string()); + let body = endpoint_error_body_for(ClientSurface::ChatCompletions, &error); let parsed: serde_json::Value = serde_json::from_slice(&body).expect("error body is JSON"); assert_eq!( parsed["error"]["message"].as_str(), - Some("User budget hard limit is not enforceable for this request") + Some( + "Hard budget cannot be enforced: include a positive output cap supported by this API, and configure an operator tariff when a hard USD limit applies" + ) + ); + assert_eq!( + parsed["error"]["code"].as_str(), + Some("hard_budget_not_enforceable") + ); + let messages = endpoint_error_body_for(ClientSurface::Messages, &error); + let parsed: serde_json::Value = serde_json::from_slice(&messages).expect("JSON error"); + assert_eq!(parsed["error"]["code"], "hard_budget_not_enforceable"); + let exceeded = endpoint_error_body_for( + ClientSurface::ChatCompletions, + &EndpointError::BudgetExceeded, ); + let parsed: serde_json::Value = serde_json::from_slice(&exceeded).expect("JSON error"); + assert_eq!(parsed["error"]["code"], "budget_exceeded"); } #[tokio::test] @@ -1042,6 +1057,66 @@ async fn admission_extracts_per_surface_output_bounds() { assert_eq!(compact.budget_facts.requested_output_tokens, Some(16)); } +#[test] +fn invalid_agent_output_caps_are_rejected_across_surfaces() { + use eggpool::request::{AdmissionOptions, admit_request}; + + for (surface, cap_field) in [ + (ClientSurface::ChatCompletions, "max_tokens"), + (ClientSurface::Responses, "max_output_tokens"), + (ClientSurface::Messages, "max_tokens"), + ] { + let mut payload = serde_json::Map::new(); + payload.insert("model".into(), serde_json::json!("synthetic-model")); + payload.insert(cap_field.into(), serde_json::json!("unbounded")); + let body = serde_json::to_vec(&payload).expect("synthetic request encodes"); + let result = admit_request( + &body, + AdmissionOptions { + client_surface: surface, + ..AdmissionOptions::default() + }, + ); + assert!( + result.is_err(), + "invalid output cap accepted for {surface:?}" + ); + } +} + +#[test] +fn sanitized_agent_budget_capture_corpus_records_supported_surfaces() { + let corpus: serde_json::Value = serde_json::from_str(include_str!( + "fixtures/on_prem_budget_agent_compatibility.json" + )) + .expect("compatibility corpus is valid JSON"); + assert_eq!(corpus["clients"][0]["version"], "0.162.1"); + assert_eq!(corpus["clients"][0]["surface"], "responses"); + assert!(corpus["clients"][0]["output_cap"].is_null()); + assert_eq!( + corpus["clients"][0]["budget_compatibility"], + "unbounded; EggPool hard rules reject before dispatch under the proof contract" + ); + assert_eq!(corpus["clients"][1]["version"], "1.18.35"); + assert_eq!(corpus["clients"][1]["output_cap"], 1024); + assert_eq!( + corpus["surface_fields"]["chat_completions"], + serde_json::json!(["max_completion_tokens", "max_tokens"]) + ); + assert_eq!( + corpus["surface_fields"]["responses"], + serde_json::json!(["max_output_tokens", "max_completion_tokens", "max_tokens"]) + ); + assert_eq!( + corpus["surface_fields"]["compact_responses"], + corpus["surface_fields"]["responses"] + ); + assert_eq!( + corpus["surface_fields"]["anthropic_messages"], + serde_json::json!(["max_tokens"]) + ); +} + #[tokio::test] async fn admission_preserves_routing_inputs_for_transcode_eligibility() { use eggpool::request::{AdmissionOptions, StaticRoutingFacts, admit_request}; From 45f43e9015cca843ecf2d3c2c4068df38975d058 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 14:23:51 +0000 Subject: [PATCH 46/62] Start closing hard budget compatibility plan 022 --- .../022-hard-budget-agent-compatibility.md | 2 +- plans/registry.md | 4 ++-- plans/subsystems/on-prem-user-access-roadmap.md | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md b/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md index b3363f46..ab2659da 100644 --- a/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md +++ b/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md @@ -1,6 +1,6 @@ # On-Prem User Access M022 — Hard-budget interoperability with coding agents -Status: active +Status: closing Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index 3294912e..a6397412 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -58,7 +58,7 @@ No new runtime, schema or performance claim lands with these plans. | Persistence | active | `plans/subsystems/persistence-roadmap.md` | M011 ready — post-M010 roadmap/documentation reconciliation | M009/M010 closures accepted. Documentation/tooling-only corrective; production remains WAL/NORMAL. No technical persistence successor is registered. | | Deployment and packaging | closed | `plans/subsystems/deployment-packaging-roadmap.md` | M005 closed — active PATH integration detection corrective | M001–M005 closed; no successor registered. | | Dashboard | closed | `plans/subsystems/dashboard-roadmap.md` | M012 closed — lifecycle closure and documentation polish | M001–M012 closed; no registered successor; future dashboard work requires a new bounded plan. | -| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M022 active — hard-budget agent compatibility; M021 closed | M017–M021 closed; M023 retains its hard dependency/evidence gates. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | +| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M022 closing — hard-budget agent compatibility; M021 closed | M017–M021 closed; M023 retains its operational evidence gate. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | | On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 ready — ordinary-server small-team synthetic capacity baseline | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 is independent measurement on ordinary on-prem Linux with durable SSD/NVMe (operational target required), not an authorization to change WAL/SQLite topology. | ## Dependency-ready implementation plans @@ -72,7 +72,7 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| -| On-prem user access | M022 — hard-budget agent interoperability | active | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | Hard M017 closure is satisfied; preserve M008/M015 proof-or-reject guarantees. | +| On-prem user access | M022 — hard-budget agent interoperability | closing | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | Client/surface evidence and full default/no-default verification are complete; preserve M008/M015 proof-or-reject guarantees. | ## Blocked work diff --git a/plans/subsystems/on-prem-user-access-roadmap.md b/plans/subsystems/on-prem-user-access-roadmap.md index d4617bd2..e5d4c4c9 100644 --- a/plans/subsystems/on-prem-user-access-roadmap.md +++ b/plans/subsystems/on-prem-user-access-roadmap.md @@ -339,7 +339,7 @@ catalog-empty/deny feedback. No budget template is copied; existing budget scopes remain independently configured. Never grant unaudited model access. Source: `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md`. -#### M022 — Hard-budget agent interoperability (active) +#### M022 — Hard-budget agent interoperability (closing) Class: polish. Active after hard M017 closure. Measure whether client/surface requests carry sufficient output-token proof; improve diagnostics and @@ -459,5 +459,5 @@ A ready plan is not an implemented capability. | M019 | closed | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | `plans/closure/on-prem-user-access/019-status.md` | Unsafe remote HTTP login GET returns no credential form; M020 unblocked. | | M020 | closed | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | `plans/closure/on-prem-user-access/020-status.md` | Headless member lifecycle and first-member transition are atomic and audited; M021 unblocked. | | M021 | closed | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | `plans/closure/on-prem-user-access/021-status.md` | Transactional future-only profile default; CLI/browser and no-profile journeys, migration, policy and budget boundaries verified. | -| M022 | active | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | — | Measuring client output-cap evidence while preserving hard proof-or-reject semantics. | +| M022 | closing | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | — | Captured Codex/OpenCode output-cap behavior; preserved hard proof-or-reject semantics and added actionable error codes. | | M023 | blocked | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Hard: M022 closure; ordinary-server evidence operational. | From 2ce15a042029ddbed2bcd4174ba2b116b3f5db9c Mon Sep 17 00:00:00 2001 From: David Bowman Date: Thu, 15 Oct 2026 14:27:01 +0000 Subject: [PATCH 47/62] Close hard budget compatibility plan 022 --- .../closure/on-prem-user-access/022-status.md | 152 ++++++++++++++++++ .../022-hard-budget-agent-compatibility.md | 2 +- plans/registry.md | 10 +- .../subsystems/on-prem-user-access-roadmap.md | 27 ++-- 4 files changed, 173 insertions(+), 18 deletions(-) create mode 100644 plans/closure/on-prem-user-access/022-status.md diff --git a/plans/closure/on-prem-user-access/022-status.md b/plans/closure/on-prem-user-access/022-status.md new file mode 100644 index 00000000..05caa787 --- /dev/null +++ b/plans/closure/on-prem-user-access/022-status.md @@ -0,0 +1,152 @@ +# On-Prem User Access M022 — Closure Status + +Status: closed + +Source implementation plan: + +- `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` + +Source subsystem roadmap: + +- `plans/subsystems/on-prem-user-access-roadmap.md#m022--hard-budget-agent-interoperability` + +Repository baseline reviewed: post-merge `88f58bbf`; implementation commit +`dee65ae`; status transition `45f43e9` (active → closing). + +## 1. Executive finding + +Hard-budget interoperability evidence is recorded for the installed Codex CLI +and OpenCode versions and for each supported request surface's explicit output +bound fields. Codex CLI 0.162.1 omitted an output bound on its captured +Responses request; OpenCode 1.18.35 sent the configured `limit.output` as +`max_output_tokens`. Hard budget proof remains caller-bound and fail-closed. +No implicit output cap or estimated fallback was added. Budget errors now +include stable codes and actionable, sanitized guidance. M022 is closed. + +## 2. Requirement-to-evidence matrix + +| Requirement | Evidence | Result | Notes | +|---|---|---|---| +| Freeze a sanitized client and surface matrix | `rust/tests/fixtures/on_prem_budget_agent_compatibility.json`; `on_prem_budgets::sanitized_agent_budget_capture_corpus_records_supported_surfaces` | pass | The client capture used synthetic requests and a local mock that returned HTTP 400. Only version, endpoint, stream and output-cap facts are retained; no body, prompt, key or provider call. | +| Measure actual coding-agent cap behavior | Same fixture and `docs/model-limits.md#coding-agent-compatibility` | pass | Codex CLI 0.162.1 omitted the cap; OpenCode 1.18.35 sent `max_output_tokens: 1024` from its configured `limit.output`. OpenCode retried the mock response; the evidence fixture records the observed field shape. | +| Preserve supported native output-cap extraction | `request/limits.rs::requested_output_tokens`; `on_prem_budgets::admission_extracts_per_surface_output_bounds` | pass | Chat accepts `max_completion_tokens`/`max_tokens`; Responses and compact Responses accept `max_output_tokens`/`max_completion_tokens`/`max_tokens`; Messages accepts `max_tokens`. | +| Reject omitted or invalid caps under hard rules before provider dispatch | `on_prem_budgets::hard_token_rule_denies_unbounded_request_without_upstream_io`, `hard_token_rule_denies_missing_proof_without_upstream_io`, `invalid_agent_output_caps_are_rejected_across_surfaces`; `on_prem_global_budgets::global_hard_token_and_usd_deny_unprovable_without_upstream_io` | pass | Existing admission proof and atomic publication gate remain authoritative. No request-body mutation or output-cap inference was added. | +| Return stable, sanitized budget failure codes and corrective guidance | `EndpointError`; `endpoint_error_body_for`; `on_prem_budgets::endpoint_error_maps_not_enforceable_to_stable_429`; `docs/api-reference.md` | pass | `hard_budget_not_enforceable` is used for missing proof, unknown tariff or overflow; `budget_exceeded` means a rule is exhausted. The proof failure message recommends a supported explicit cap and an operator tariff when hard USD applies. | +| Preserve retry settlement, private-model budgets and provider-health boundaries | Existing `on_prem_budgets`, `on_prem_global_budgets`, `on_prem_attribution`, coordinator and provider suites; request lifecycle documentation | pass | No `Retry-After` is emitted for a policy/proof decision. No retry, routing, provider health, persistence or price-accounting behavior was changed. | + +## 3. Production implementation evidence + +`request/limits.rs` continues to extract only positive, surface-compatible +caller output caps. `BudgetProof` and the publication transaction remain +unchanged: hard token rules require the explicit cap, hard USD rules also +require the operator tariff, and no proof means pre-dispatch denial. + +Inference errors now use `hard_budget_not_enforceable` and `budget_exceeded` +machine codes in OpenAI-shaped and Messages-shaped envelopes. The hard-proof +message names both safe corrections (a caller cap and, for hard USD, an +operator tariff) because the durable transaction result deliberately does not +expose whether the underlying missing proof was output, price or overflow. + +An operator-injected cap was evaluated and rejected: available protocol and +client evidence does not prove one inferred limit would be honored across +native, translated, streaming, compact, retry and fallback-provider paths. +Codex must provide a request-level cap for hard token rules. OpenCode +operators can configure `limit.output` so the client sends one. + +## 4. Verification executed + +### Commands run + +```bash +rtk cargo fmt --manifest-path rust/Cargo.toml --all -- --check +rtk cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings +rtk cargo check --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features +rtk cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- -D warnings +rtk cargo test --manifest-path rust/Cargo.toml --workspace --all-targets -- --test-threads=1 +rtk cargo test --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- --test-threads=1 +rtk cargo test --manifest-path rust/Cargo.toml --test on_prem_budgets -- --test-threads=1 +rtk cargo test --manifest-path rust/Cargo.toml --test on_prem_budgets --no-default-features -- --test-threads=1 +rtk uv run pytest tests/tooling/ -q --tb=short --maxfail=1 +rtk git diff --check +``` + +### Results + +- Final implementation workspace runs: default features 1,039 passed and 1 + ignored across 80 suites; no-default features 1,040 passed and 1 ignored + across 80 suites. +- Final focused budget target: 29 passed with default features and 29 passed + with `--no-default-features`. +- Formatting, default/no-default strict Clippy and no-default workspace check + passed. +- Tooling pytest: 192 passed, 1 skipped. `git diff --check` passed. +- Local verification only; hosted CI and hardware performance qualification + were not run for this milestone. + +## 5. Invariant review + +- No hard request/token/USD rule is weakened; no unbounded request reaches a + provider because of this work. +- Requests are not mutated to synthesize output bounds. Zero, malformed or + absent caps cannot become hard proof. +- USD proof still requires the selected provider's conservative operator + tariff; estimator fallbacks do not become proof. +- Client-facing errors disclose no user budgets, provider IDs, accounts, + prompts, tokens or request bodies. +- Policy intersection, retry holds, selector overhead, cancellation and + stream finalization retain their previous owners and behavior. + +## 6. Failure and recovery review + +This change adds no durable state, queue, background task, retry or network +call. The admission proof is immutable request data and is checked in the +existing atomic publication path before provider submission. Retry settlement +and stream/cancellation recovery are covered by the unchanged serial +coordinator and budget suites. Invalid limits fail during request admission; +missing hard proof and unknown tariff fail closed at publication. + +## 7. Migration and compatibility review + +No schema or configuration migration was needed. Existing estimated rules +retain their semantics. Existing explicit cap aliases remain supported per +surface. The new error codes are additive JSON fields; existing status remains +HTTP 429. No `Retry-After` is emitted for a budget policy decision. + +## 8. Security review + +Only synthetic requests reached a local loopback mock during client capture. +The checked-in fixture contains only sanitized version and field facts. No +real prompt, provider key, request body, or provider traffic was used. Error +messages are static and do not reveal rule scope or tariff values. No +privilege or routing boundary changed. + +## 9. Documentation and operations + +Updated `docs/model-limits.md`, `docs/agent-configuration.md`, +`docs/api-reference.md`, and `architecture/deep-dive-request-lifecycle.md`. +The client guidance tells OpenCode operators to set `limit.output` and +explains that Codex CLI requests without a cap cannot use hard token budgets. + +## 10. Unresolved findings + +| Severity | Finding | Impact | Required action | +|---|---|---|---| +| none | No M022 correctness, security or compatibility defect remains. | Hard proof behavior and client guidance are verified locally. | None. | + +## 11. Roadmap disposition + +M022 is closed. M023's hard M022 dependency is satisfied. Its remaining +ordinary-server Linux/SSD-or-NVMe measurement is an operational evidence gate +for implementation acceptance and team-capacity claims. The available runner +is a Darwin MacBook Pro, so it cannot supply that evidence; the independent +on-prem concurrency M002 plan remains ready and no qualifying hardware record +was found. M023 remains blocked pending that ordinary-server evidence. No +other on-prem user-access plan after M023 is registered as eligible. + +## 12. Registry updates + +The closure commit marks M022 `implemented` and the roadmap milestone `closed`, +adds this record to recent closures, and records the M023 unblock audit. +M023 remains `blocked` with M022 removed from its blocker; its remaining +operational gate is the ordinary-server M002 qualification. No other +dependency-ready or active plan is promoted. diff --git a/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md b/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md index ab2659da..bb9aa7a7 100644 --- a/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md +++ b/plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md @@ -1,6 +1,6 @@ # On-Prem User Access M022 — Hard-budget interoperability with coding agents -Status: closing +Status: implemented Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index a6397412..801c0f9c 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -58,7 +58,7 @@ No new runtime, schema or performance claim lands with these plans. | Persistence | active | `plans/subsystems/persistence-roadmap.md` | M011 ready — post-M010 roadmap/documentation reconciliation | M009/M010 closures accepted. Documentation/tooling-only corrective; production remains WAL/NORMAL. No technical persistence successor is registered. | | Deployment and packaging | closed | `plans/subsystems/deployment-packaging-roadmap.md` | M005 closed — active PATH integration detection corrective | M001–M005 closed; no successor registered. | | Dashboard | closed | `plans/subsystems/dashboard-roadmap.md` | M012 closed — lifecycle closure and documentation polish | M001–M012 closed; no registered successor; future dashboard work requires a new bounded plan. | -| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M022 closing — hard-budget agent compatibility; M021 closed | M017–M021 closed; M023 retains its operational evidence gate. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | +| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M023 blocked — short-window fairness controls; M022 closed | M017–M022 closed; M023 awaits ordinary-server operational evidence. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | | On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 ready — ordinary-server small-team synthetic capacity baseline | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 is independent measurement on ordinary on-prem Linux with durable SSD/NVMe (operational target required), not an authorization to change WAL/SQLite topology. | ## Dependency-ready implementation plans @@ -72,14 +72,13 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| -| On-prem user access | M022 — hard-budget agent interoperability | closing | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | Client/surface evidence and full default/no-default verification are complete; preserve M008/M015 proof-or-reject guarantees. | ## Blocked work | Subsystem | Milestone | Blocker | |---|---|---| -| On-prem user access | M023 — short-window team fairness | Hard: M022 closure; ordinary-server evidence operational. | +| On-prem user access | M023 — short-window team fairness | Ordinary-server M002 operational evidence: representative Linux server with durable SSD/NVMe and measured baseline; M022 is closed. | Unblock audit (on-prem user access reopening after M012 closure, baseline @@ -111,7 +110,8 @@ Historical M007 blocker assessment at baseline `7e241ad` (`plans/closure/persist | Subsystem / plan | Disposition | Evidence | |---|---|---| -| On-prem user access M021 — default member policy and provisioning | closed — transactional future-only default profile selection, CLI/browser creation, explicit no-profile override, additive schema 63 migration, policy visibility and budget-separation behavior pass; M022 remains ready; M023 remains blocked on M022 and ordinary-server evidence | `plans/closure/on-prem-user-access/021-status.md`, implementation `b76d359`, CLI journey `ee563be`, race test `3f1ca54` | +| On-prem user access M022 — hard-budget agent interoperability | closed — Codex CLI 0.162.1 omitted a Responses output cap; OpenCode 1.18.35 sent configured `max_output_tokens`; stable proof-failure and exhausted-budget codes added; default/no-default verification passes; M023 remains blocked on ordinary-server evidence | `plans/closure/on-prem-user-access/022-status.md`, implementation `dee65ae` | +| On-prem user access M021 — default member policy and provisioning | closed — transactional future-only default profile selection, CLI/browser creation, explicit no-profile override, additive schema 63 migration, policy visibility and budget-separation behavior pass; M022 subsequently closed; M023 remains blocked on ordinary-server evidence | `plans/closure/on-prem-user-access/021-status.md`, implementation `b76d359`, CLI journey `ee563be`, race test `3f1ca54` | | On-prem user access M020 — managed transition and headless administration | closed — first member transition, profile assignment and audit are atomic; headless add/show/disable/enable workflow and stale-config auth rejection pass; M021 promoted to ready | `plans/closure/on-prem-user-access/020-status.md`, implementation `8a875a1` | | On-prem user access M019 — login transport and solo LAN operability corrective | closed — remote plaintext login GET returns no credential form; loopback and configured HTTPS-proxy paths pass; solo SSH tunnel and team proxy guidance documented; M020 promoted to ready | `plans/closure/on-prem-user-access/019-status.md`, implementation `b18a561` | | On-prem user access M018 — local operator status corrective | closed — status and runtime status use bounded private control-socket readback without a server API key; managed HTTP authorization is unchanged; no successor unblocked | `plans/closure/on-prem-user-access/018-status.md`, implementation `8e2b66a`, CLI fixture `3be5d83` | @@ -537,3 +537,5 @@ Unblock audit (on-prem user access M019 closed, `plans/closure/on-prem-user-acce Unblock audit (on-prem user access M020 closed, `plans/closure/on-prem-user-access/020-status.md`): searched the on-prem implementation plans, roadmap dependency graph, and dependency-ready, active, and blocked registry tables. M021's sole hard dependency, M020, is closed, so M021 moves `blocked` → `ready`. M023 remains blocked on M022 and ordinary-server operational evidence; M022 remains ready. No other plan is promoted and no team-capacity claim is introduced. Unblock audit (on-prem user access M021 closed, `plans/closure/on-prem-user-access/021-status.md`): searched the on-prem implementation plans, roadmap dependency graph, and dependency-ready, active, and blocked registry tables. M022 was already ready after M017 closed and remains ready; M021 was not a prerequisite. M023 remains blocked on hard M022 closure and the operational ordinary-server M002 evidence; M002's ready status does not satisfy that evidence gate. No newly-ready plan is promoted and no team-capacity claim is introduced. + +Unblock audit (on-prem user access M022 closed, `plans/closure/on-prem-user-access/022-status.md`): searched the on-prem implementation plans, roadmap dependency graph, and dependency-ready, active, and blocked registry tables. M023's hard M022 dependency is satisfied, but its source-plan stop condition still requires measured ordinary-server Linux with durable SSD/NVMe storage. The available runner is a Darwin MacBook Pro and no qualifying M002 record exists; M002 remains ready but cannot be completed from this environment. M023 therefore remains blocked pending that evidence. No other registered on-prem user-access plan after M023 is eligible; no capacity claim is introduced. diff --git a/plans/subsystems/on-prem-user-access-roadmap.md b/plans/subsystems/on-prem-user-access-roadmap.md index e5d4c4c9..6b517b16 100644 --- a/plans/subsystems/on-prem-user-access-roadmap.md +++ b/plans/subsystems/on-prem-user-access-roadmap.md @@ -339,26 +339,27 @@ catalog-empty/deny feedback. No budget template is copied; existing budget scopes remain independently configured. Never grant unaudited model access. Source: `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md`. -#### M022 — Hard-budget agent interoperability (closing) +#### M022 — Hard-budget agent interoperability (closed) -Class: polish. Active after hard M017 closure. Measure whether client/surface -requests carry sufficient output-token proof; improve diagnostics and -introduce output-cap insertion only where upstream and wire evidence prove a -hard bound. Do not regress M008/M015 hard-cost safety. +Class: polish. Closed after measuring supported client/surface output-cap +behavior and adding actionable hard-budget diagnostics. No output cap is +injected; M008/M015 proof-or-reject guarantees remain in force. Source: `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md`. #### M023 — Short-window team fairness controls -Class: capability. Blocked on hard M022 closure. Optional per-principal -short-window admission throttle and stable 429/Retry-After, bounded in-memory -state, no unbounded queues or new SQLite write pressure; qualify on ordinary -server hardware while preserving solo no-rule behavior. +Class: capability. Hard M022 dependency is closed. Remains blocked by the +source plan's stop condition until ordinary-server Linux/SSD-or-NVMe evidence +is recorded through concurrency M002. Then implement an optional +per-principal short-window admission throttle and stable 429/Retry-After, +bounded in-memory state, no unbounded queues or new SQLite write pressure; +preserve solo no-rule behavior. Source: `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md`. ### Post-merge dependency and hardware policy -- M017–M021 are closed. M022 is ready and does not depend on a Pi/Le Potato - run; it became ready after the M017 closure. +- M017–M022 are closed. M022 did not depend on a Pi/Le Potato run; it became + ready after M017 closed. - Hard: M017 + M019 -> M020 -> M021; M017 -> M022 -> M023. - Interface: M018's privileged local transport must not broaden HTTP rights; any new auth authority stops for ADR review. M022 hard-cost proof contract @@ -459,5 +460,5 @@ A ready plan is not an implemented capability. | M019 | closed | `plans/implementation/on-prem-user-access/019-login-transport-solo-lan-corrective.md` | `plans/closure/on-prem-user-access/019-status.md` | Unsafe remote HTTP login GET returns no credential form; M020 unblocked. | | M020 | closed | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | `plans/closure/on-prem-user-access/020-status.md` | Headless member lifecycle and first-member transition are atomic and audited; M021 unblocked. | | M021 | closed | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | `plans/closure/on-prem-user-access/021-status.md` | Transactional future-only profile default; CLI/browser and no-profile journeys, migration, policy and budget boundaries verified. | -| M022 | closing | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | — | Captured Codex/OpenCode output-cap behavior; preserved hard proof-or-reject semantics and added actionable error codes. | -| M023 | blocked | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Hard: M022 closure; ordinary-server evidence operational. | +| M022 | closed | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | `plans/closure/on-prem-user-access/022-status.md` | Captured Codex/OpenCode output-cap behavior; preserved hard proof-or-reject semantics and added actionable error codes. | +| M023 | blocked | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Hard M022 is closed; ordinary-server M002 Linux/SSD-or-NVMe operational evidence remains outstanding. | From e284b457d4c2425ab60c3bfab7eae08c7f0de8d2 Mon Sep 17 00:00:00 2001 From: wr3n-ai Date: Fri, 16 Oct 2026 00:54:27 +0000 Subject: [PATCH 48/62] Add recovery evidence to ordinary-server qualification --- docs/deployment.md | 6 +- scripts/qualification_ordinary_server.py | 59 ++++++++++++++++++ .../test_qualification_ordinary_server.py | 62 +++++++++++++++++++ 3 files changed, 125 insertions(+), 2 deletions(-) diff --git a/docs/deployment.md b/docs/deployment.md index 180c4865..388125e9 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -518,8 +518,10 @@ To repeat the frozen corpus from a release binary, run `uv run python scripts/qualification_ordinary_server.py --binary `. The driver writes its manifest before collection, creates a fresh qualification root for each run, requires three repetitions per condition, validates p99 -sample counts and settlement, and derives `summary.json` from the raw Q008 -reports. It refuses non-Linux hosts and non-durable or rotational qualification +sample counts and settlement, derives `summary.json` from the raw Q008 +reports, and runs deterministic Q009 provider-fault and abrupt-restart phases +against the same binary. The summary requires the matching `lifecycle.json` +artifact. It refuses non-Linux hosts and non-durable or rotational qualification storage, and stops before load when initial full I/O pressure exceeds 25% over the 10-second window. Qualification does not send requests to a real provider or charge a provider account. diff --git a/scripts/qualification_ordinary_server.py b/scripts/qualification_ordinary_server.py index eefe009a..47d235f7 100644 --- a/scripts/qualification_ordinary_server.py +++ b/scripts/qualification_ordinary_server.py @@ -29,6 +29,7 @@ ) SCHEMA_VERSION = "on-prem-ordinary-server-matrix.v1" SUMMARY_VERSION = "on-prem-ordinary-server-summary.v1" +LIFECYCLE_ARTIFACT = "lifecycle.json" REPETITIONS = 3 REQUESTS_PER_CELL = 1000 SEQUENTIAL_SAMPLES = 30 @@ -165,6 +166,32 @@ def summarize(manifest_path: Path) -> dict[str, Any]: if not isinstance(candidate, dict): raise ValueError("candidate identity is missing") candidate = cast("dict[str, Any]", candidate) + lifecycle_name = manifest.get("lifecycle_artifact") + if lifecycle_name != LIFECYCLE_ARTIFACT: + raise ValueError("ordinary-server lifecycle artifact is missing") + lifecycle = _load(manifest_path.parent / LIFECYCLE_ARTIFACT) + if ( + lifecycle.get("status") != "pass" + or lifecycle.get("schema_version") != "runtime-q009.v1" + or not isinstance(lifecycle.get("candidate"), dict) + or lifecycle["candidate"].get("sha256") != candidate.get("sha256") + ): + raise ValueError("ordinary-server lifecycle qualification did not pass") + lifecycle_phases = lifecycle.get("phases") + if not isinstance(lifecycle_phases, list): + raise ValueError("ordinary-server lifecycle phases are missing") + lifecycle_phases = cast("list[Any]", lifecycle_phases) + required_phases = {"faults", "restart"} + observed_phases: set[str] = set() + for raw_phase in lifecycle_phases: + if not isinstance(raw_phase, dict): + continue + phase = cast("dict[str, Any]", raw_phase) + name = phase.get("phase") + if isinstance(name, str) and phase.get("status") == "pass": + observed_phases.add(name) + if not required_phases <= observed_phases: + raise ValueError("ordinary-server lifecycle phases are incomplete") output_dir = manifest_path.parent conditions = manifest.get("conditions") if not isinstance(conditions, list): @@ -293,6 +320,13 @@ def timing_median( "candidate": candidate, "target": manifest.get("target"), "workload": manifest.get("workload"), + "lifecycle": { + "artifact": LIFECYCLE_ARTIFACT, + "status": "pass", + "phases": sorted(required_phases), + "provider": lifecycle.get("provider"), + "ownership_convergence": lifecycle.get("ownership_convergence"), + }, "interpretation": ( "Descriptive loopback capacity characterization for this host and " "synthetic workload; not an SLA or provider-throughput promise." @@ -350,6 +384,31 @@ def run(binary: Path, output_dir: Path) -> int: json.dumps(manifest, indent=2) + "\n", encoding="utf-8" ) return result.returncode + lifecycle_path = output_dir / LIFECYCLE_ARTIFACT + lifecycle_command = [ + sys.executable, + str(ROOT / "scripts/qualification_stability.py"), + "--binary", + str(binary), + "--phase", + "faults", + "--phase", + "restart", + "--output", + str(lifecycle_path), + ] + lifecycle_result = subprocess.run(lifecycle_command, cwd=ROOT, check=False) + if lifecycle_result.returncode != 0: + manifest["status"] = "incomplete" + manifest["stopped_at"] = { + "artifact": LIFECYCLE_ARTIFACT, + "returncode": lifecycle_result.returncode, + } + manifest_path.write_text( + json.dumps(manifest, indent=2) + "\n", encoding="utf-8" + ) + return lifecycle_result.returncode + manifest["lifecycle_artifact"] = LIFECYCLE_ARTIFACT manifest["status"] = "complete" manifest_path.write_text(json.dumps(manifest, indent=2) + "\n", encoding="utf-8") summary = summarize(manifest_path) diff --git a/tests/tooling/test_qualification_ordinary_server.py b/tests/tooling/test_qualification_ordinary_server.py index df796a77..6cef27e1 100644 --- a/tests/tooling/test_qualification_ordinary_server.py +++ b/tests/tooling/test_qualification_ordinary_server.py @@ -97,10 +97,27 @@ def test_m002_raw_to_summary_requires_three_durable_p99_runs(tmp_path: Path) -> "status": "complete", "plan": "on-prem-concurrency-performance M002", "candidate": {"sha256": "a" * 64}, + "lifecycle_artifact": "lifecycle.json", "target": {"architecture": "x86_64"}, "workload": {"requests_per_condition": REQUESTS_PER_CELL}, "conditions": [{"principals": 5, "concurrency": 4, "run_artifacts": names}], } + (tmp_path / "lifecycle.json").write_text( + json.dumps( + { + "schema_version": "runtime-q009.v1", + "status": "pass", + "candidate": {"sha256": "a" * 64}, + "phases": [ + {"phase": "faults", "status": "pass"}, + {"phase": "restart", "status": "pass"}, + ], + "provider": {"request_count": 5}, + "ownership_convergence": {"pending_requests": 0}, + } + ), + encoding="utf-8", + ) manifest_path = tmp_path / "matrix.json" manifest_path.write_text(json.dumps(manifest), encoding="utf-8") @@ -112,6 +129,7 @@ def test_m002_raw_to_summary_requires_three_durable_p99_runs(tmp_path: Path) -> assert cell["median_p95_ms"] == 22 assert cell["median_p99_ms"] == 23 assert cell["settlement"].startswith("zero pending") + assert summary["lifecycle"]["phases"] == ["faults", "restart"] def test_m002_raw_to_summary_rejects_incomplete_matrix(tmp_path: Path) -> None: @@ -122,3 +140,47 @@ def test_m002_raw_to_summary_rejects_incomplete_matrix(tmp_path: Path) -> None: ) with pytest.raises(ValueError, match="incomplete"): summarize(path) + + +def test_m002_raw_to_summary_requires_passing_same_candidate_lifecycle( + tmp_path: Path, +) -> None: + names = [f"run-{index}.json" for index in range(1, REPETITIONS + 1)] + for name in names: + (tmp_path / name).write_text(json.dumps(_run(5, 4, 20)), encoding="utf-8") + (tmp_path / "lifecycle.json").write_text( + json.dumps( + { + "schema_version": "runtime-q009.v1", + "status": "pass", + "candidate": {"sha256": "b" * 64}, + "phases": [ + {"phase": "faults", "status": "pass"}, + {"phase": "restart", "status": "pass"}, + ], + } + ), + encoding="utf-8", + ) + manifest_path = tmp_path / "matrix.json" + manifest_path.write_text( + json.dumps( + { + "schema_version": SCHEMA_VERSION, + "status": "complete", + "candidate": {"sha256": "a" * 64}, + "lifecycle_artifact": "lifecycle.json", + "conditions": [ + { + "principals": 5, + "concurrency": 4, + "run_artifacts": names, + } + ], + } + ), + encoding="utf-8", + ) + + with pytest.raises(ValueError, match="lifecycle qualification did not pass"): + summarize(manifest_path) From 7b5eae3af949471a5b93f31b985a432b69c47eaf Mon Sep 17 00:00:00 2001 From: wr3n-ai Date: Fri, 16 Oct 2026 00:56:53 +0000 Subject: [PATCH 49/62] Fix direct ordinary-server runner invocation --- scripts/qualification_ordinary_server.py | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/scripts/qualification_ordinary_server.py b/scripts/qualification_ordinary_server.py index 47d235f7..e72e700a 100644 --- a/scripts/qualification_ordinary_server.py +++ b/scripts/qualification_ordinary_server.py @@ -16,12 +16,20 @@ from pathlib import Path from typing import Any, cast -from scripts.qualification_sbc import ( - ORDINARY_SERVER_MAX_INITIAL_IO_PRESSURE_PERCENT, - host_io_full_avg10, - host_io_snapshot, - ordinary_server_metadata, -) +try: + from scripts.qualification_sbc import ( + ORDINARY_SERVER_MAX_INITIAL_IO_PRESSURE_PERCENT, + host_io_full_avg10, + host_io_snapshot, + ordinary_server_metadata, + ) +except ModuleNotFoundError: # Running this file directly from ``scripts/``. + from qualification_sbc import ( # type: ignore[no-redef] + ORDINARY_SERVER_MAX_INITIAL_IO_PRESSURE_PERCENT, + host_io_full_avg10, + host_io_snapshot, + ordinary_server_metadata, + ) ROOT = Path(__file__).resolve().parents[1] DEFAULT_OUTPUT = ( From 1c3d15b8911f1f0f3e003c9e918282c0363736e3 Mon Sep 17 00:00:00 2001 From: wr3n-ai Date: Fri, 16 Oct 2026 00:58:12 +0000 Subject: [PATCH 50/62] Keep qualification budget above verification requests --- scripts/qualification_sbc.py | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/scripts/qualification_sbc.py b/scripts/qualification_sbc.py index ebe98951..020d7c17 100644 --- a/scripts/qualification_sbc.py +++ b/scripts/qualification_sbc.py @@ -2877,10 +2877,11 @@ def _configure_benchmark_principals( ) from error if budget_mode in {"user", "user-global"}: - # Keep the hard ceiling above the measured workload and Q008's bounded - # post-batch restart/recovery checks so those checks do not invalidate - # an otherwise complete budget-mode run. - max_requests = str(request_count + 100) + # Q008 runs functional smoke, sequential surface samples, the concurrent + # batch, and post-recovery requests. Keep the hard ceiling above those + # bounded checks so a successful benchmark does not accidentally turn + # its settlement verification into a budget-denial test. + max_requests = str(request_count * 3 + 100) for username in ["sbc-admin"] + [ f"sbc-member-{index:02d}" for index in range(1, principal_count) ]: @@ -2929,7 +2930,7 @@ def _configure_benchmark_principals( "--window", "day", "--max-requests", - str(request_count + 100), + str(request_count * 3 + 100), "--enforcement", "hard", "--actor", From 2ce1bc4d5377df1b9cf30b9256f3f48eb6c4175b Mon Sep 17 00:00:00 2001 From: wr3n-ai Date: Fri, 16 Oct 2026 01:15:05 +0000 Subject: [PATCH 51/62] Tie M002 recovery checks to managed identity --- scripts/qualification_ordinary_server.py | 28 +++ scripts/qualification_stability.py | 161 +++++++++++++++--- .../test_qualification_ordinary_server.py | 10 +- tests/tooling/test_qualification_stability.py | 2 +- 4 files changed, 174 insertions(+), 27 deletions(-) diff --git a/scripts/qualification_ordinary_server.py b/scripts/qualification_ordinary_server.py index e72e700a..ad922571 100644 --- a/scripts/qualification_ordinary_server.py +++ b/scripts/qualification_ordinary_server.py @@ -235,6 +235,10 @@ def summarize(manifest_path: Path) -> dict[str, Any]: run_candidate = cast("dict[str, Any]", run_candidate) if run_candidate.get("sha256") != candidate.get("sha256"): raise ValueError(f"candidate hash changed: {name}") + if run_candidate.get("repository_commit_sha") != candidate.get( + "repository_commit_sha" + ): + raise ValueError(f"candidate source commit changed: {name}") environment = document.get("environment", {}) if environment.get("target_class") != "ordinary-server": raise ValueError(f"ordinary-server attestation missing: {name}") @@ -354,6 +358,30 @@ def run(binary: Path, output_dir: Path) -> int: for condition in manifest["conditions"]: for run_number, name in enumerate(condition["run_artifacts"], start=1): output = output_dir / name + if output.is_file(): + try: + prior = _load(output) + prior_candidate = prior.get("candidate", {}) + prior_benchmark = prior.get("benchmark", {}) + prior_load = prior_benchmark.get("runs", {}).get( + "concurrent_native_responses_finite", {} + ) + reusable = ( + prior.get("status") == "pass" + and prior.get("schema_version") == "runtime-q008.v2" + and prior_candidate.get("sha256") + == manifest["candidate"]["sha256"] + and prior_candidate.get("repository_commit_sha") + == manifest["candidate"]["repository_commit_sha"] + and prior_benchmark.get("principal_count") + == condition["principals"] + and prior_load.get("client_concurrency") + == condition["concurrency"] + ) + except (OSError, ValueError, AttributeError): + reusable = False + if reusable: + continue command = [ sys.executable, str(ROOT / "scripts/qualification_sbc.py"), diff --git a/scripts/qualification_stability.py b/scripts/qualification_stability.py index 4e8aca94..d103a2ba 100644 --- a/scripts/qualification_stability.py +++ b/scripts/qualification_stability.py @@ -21,6 +21,7 @@ import json import os import platform +import re import signal import socket import subprocess @@ -391,6 +392,7 @@ def _post( *, streaming: bool = False, route_session: str | None = None, + api_token: str, ) -> tuple[int, bytes, int, int | None]: paths = { "chat_completions": "/v1/chat/completions", @@ -410,7 +412,7 @@ def _post( if streaming: payload["stream"] = True headers = { - "Authorization": "Bearer q009-server-key", + "Authorization": f"Bearer {api_token}", "Content-Type": "application/json", } if route_session: @@ -424,7 +426,7 @@ def _post( ) -def _cancel_request(port: int) -> None: +def _cancel_request(port: int, api_token: str) -> None: payload = json.dumps( { "model": "q009-cancel", @@ -434,20 +436,22 @@ def _cancel_request(port: int) -> None: ).encode() request = ( "POST /v1/chat/completions HTTP/1.1\r\nHost: localhost\r\n" - "Authorization: Bearer q009-server-key\r\nContent-Type: application/json\r\n" + f"Authorization: Bearer {api_token}\r\nContent-Type: application/json\r\n" f"Content-Length: {len(payload)}\r\nConnection: close\r\n\r\n" ).encode() with socket.create_connection(("127.0.0.1", port), timeout=2) as connection: connection.sendall(request + payload) -def _write_abort_request(port: int) -> None: +def _write_abort_request(port: int, api_token: str) -> None: """Close while writing a bounded request body to exercise write cleanup.""" payload = b'{"model":"q009-cancel","messages":[' request = ( b"POST /v1/chat/completions HTTP/1.1\r\nHost: localhost\r\n" - b"Authorization: Bearer q009-server-key\r\nContent-Type: application/json\r\n" - b"Content-Length: 100000\r\nConnection: close\r\n\r\n" + + ( + f"Authorization: Bearer {api_token}\r\nContent-Type: application/json\r\n" + ).encode() + + b"Content-Length: 100000\r\nConnection: close\r\n\r\n" ) with socket.create_connection(("127.0.0.1", port), timeout=2) as connection: connection.sendall(request + payload) @@ -625,6 +629,48 @@ def run_qualification( commands.append(outcome) if outcome["status"] != "pass": raise QualificationError(f"preflight {args[0]} failed") + try: + bootstrap = subprocess.run( + [ + str(binary), + "--config", + str(config), + "users", + "bootstrap", + "q009-admin", + "--password-stdin", + "--show-secret", + ], + cwd=ROOT, + env=dict(env), + input="Q009 synthetic admin password 123\n", + capture_output=True, + text=True, + timeout=timeout, + check=False, + ) + except (OSError, subprocess.TimeoutExpired) as error: + raise QualificationError( + "synthetic managed administrator bootstrap failed" + ) from error + token_match = ( + re.search(r"epk1_[A-Za-z0-9_-]+", bootstrap.stdout) + if bootstrap.returncode == 0 + else None + ) + if token_match is None: + raise QualificationError( + "synthetic managed administrator token was unavailable" + ) + agent_token = token_match.group(0) + commands.append( + { + "command": ["users", "bootstrap", "q009-admin"], + "status": "pass", + "duration_ms": round((time.monotonic() - started) * 1000), + "reason": "synthetic managed principal created", + } + ) process = _start(binary, config, env) _wait_ready(process, f"http://127.0.0.1:{port}/v1/healthz", timeout) report["environment"] = { @@ -643,7 +689,11 @@ def run_qualification( ("responses", "q009-responses", True), ): status, body, _, _ = _post( - port, surface, model, streaming=streaming + port, + surface, + model, + streaming=streaming, + api_token=agent_token, ) if status != 200 or not body: raise QualificationError( @@ -666,6 +716,7 @@ def run_qualification( surface, model, streaming=streaming, + api_token=agent_token, ) if status != 200 or not body: raise QualificationError( @@ -677,6 +728,7 @@ def run_qualification( "chat_completions", "stability", route_session=f"q009-session-{cycle}", + api_token=agent_token, ) if status != 200 or not body: raise QualificationError( @@ -698,20 +750,54 @@ def run_qualification( "q009-fault-5xx", "q009-timeout", ): - status, body, _, _ = _post(port, "chat_completions", model) + provider_before = provider.count(model) + status, body, _, _ = _post( + port, + "chat_completions", + model, + api_token=agent_token, + ) + provider_after = provider.count(model) + if provider_after <= provider_before: + raise QualificationError( + "provider fault fixture was not observed for " + f"{model}" + ) + if status not in {200, 408, 429, 502, 503, 504}: + raise QualificationError( + f"unexpected provider fault status for {model}: " + f"HTTP {status}" + ) row = { "model": model, - "http_status": status, - "body_bytes": len(body), + "category": "provider", + "expected_upstream_status": { + "q009-fault-408": 408, + "q009-fault-429": 429, + "q009-fault-5xx": 503, + "q009-timeout": "timeout", + }[model], + "client_http_status": status, + "provider_requests": provider_after - provider_before, + "response_bytes": len(body), } - if status != 200 or not body: - raise QualificationError( - f"fault recovery failed for {model}: HTTP {status}" - ) fault_results.append(row) + if status != 200: + _close_started(process, min(timeout, 8)) + process = _start(binary, config, env) + _wait_ready( + process, + f"http://127.0.0.1:{port}/v1/healthz", + timeout, + ) + row["post_fault_server_restart"] = "pass" for model in ("q009-partial", "q009-disconnect"): status, body, _, _ = _post( - port, "chat_completions", model, streaming=True + port, + "chat_completions", + model, + streaming=True, + api_token=agent_token, ) if status != 200 or b"[DONE]" in body: raise QualificationError( @@ -724,8 +810,8 @@ def run_qualification( "terminal_evidence": False, } ) - _cancel_request(port) - _write_abort_request(port) + _cancel_request(port, agent_token) + _write_abort_request(port, agent_token) fault_results.append( { "model": "q009-cancel", @@ -733,7 +819,10 @@ def run_qualification( } ) status, body, _, _ = _post( - port, "chat_completions", "q009-wire" + port, + "chat_completions", + "q009-wire", + api_token=agent_token, ) if status != 200 or not body: raise QualificationError( @@ -747,7 +836,11 @@ def run_qualification( } ) status, body, _, _ = _post( - port, "chat_completions", "q009-malformed", streaming=True + port, + "chat_completions", + "q009-malformed", + streaming=True, + api_token=agent_token, ) if status != 200 or b"[DONE]" in body: raise QualificationError( @@ -761,7 +854,10 @@ def run_qualification( } ) status, body, _, _ = _post( - port, "chat_completions", "q009-absent" + port, + "chat_completions", + "q009-absent", + api_token=agent_token, ) if status != 404: raise QualificationError( @@ -776,7 +872,10 @@ def run_qualification( } ) status, body, _, _ = _post( - port, "chat_completions", "q009-connect/unreachable" + port, + "chat_completions", + "q009-connect/unreachable", + api_token=agent_token, ) if status not in {502, 503} or not body: raise QualificationError( @@ -855,7 +954,12 @@ def run_qualification( raise QualificationError( "reload recovery after rejected candidate failed" ) - _post(port, "chat_completions", "q009-chat") + _post( + port, + "chat_completions", + "q009-chat", + api_token=agent_token, + ) _resource_sample("after-reload", process, database, port, samples) report["phases"].append( _phase_result( @@ -871,7 +975,12 @@ def run_qualification( def in_flight() -> None: try: - _post(port, "chat_completions", "q009-abrupt") + _post( + port, + "chat_completions", + "q009-abrupt", + api_token=agent_token, + ) except OSError as error: request_error.append(type(error).__name__) @@ -919,7 +1028,11 @@ def in_flight() -> None: ("messages", "q009-messages", False), ): status, body, _, _ = _post( - port, surface, model, streaming=streaming + port, + surface, + model, + streaming=streaming, + api_token=agent_token, ) if status != 200 or not body: raise QualificationError( diff --git a/tests/tooling/test_qualification_ordinary_server.py b/tests/tooling/test_qualification_ordinary_server.py index 6cef27e1..7c812a6f 100644 --- a/tests/tooling/test_qualification_ordinary_server.py +++ b/tests/tooling/test_qualification_ordinary_server.py @@ -26,7 +26,10 @@ def _run(principals: int, concurrency: int, p95: int) -> dict[str, Any]: return { "status": "pass", "schema_version": "runtime-q008.v2", - "candidate": {"sha256": "a" * 64}, + "candidate": { + "sha256": "a" * 64, + "repository_commit_sha": "c" * 40, + }, "environment": { "target_class": "ordinary-server", "qualification_database_storage": { @@ -96,7 +99,10 @@ def test_m002_raw_to_summary_requires_three_durable_p99_runs(tmp_path: Path) -> "schema_version": SCHEMA_VERSION, "status": "complete", "plan": "on-prem-concurrency-performance M002", - "candidate": {"sha256": "a" * 64}, + "candidate": { + "sha256": "a" * 64, + "repository_commit_sha": "c" * 40, + }, "lifecycle_artifact": "lifecycle.json", "target": {"architecture": "x86_64"}, "workload": {"requests_per_condition": REQUESTS_PER_CELL}, diff --git a/tests/tooling/test_qualification_stability.py b/tests/tooling/test_qualification_stability.py index fdb947c6..96207868 100644 --- a/tests/tooling/test_qualification_stability.py +++ b/tests/tooling/test_qualification_stability.py @@ -67,5 +67,5 @@ def test_q009_fake_candidate_failure_is_reported_without_workload( assert main(["--binary", str(candidate), "--output", str(output)]) == 1 report = json.loads(output.read_text(encoding="utf-8")) assert report["status"] == "fail" - assert "candidate exited" in report["reason"] + assert report["reason"] == "synthetic managed administrator token was unavailable" assert MAX_SAMPLES > 0 From 44166c2a8417418627a67f02864a8e965411cb40 Mon Sep 17 00:00:00 2001 From: wr3n-ai Date: Fri, 16 Oct 2026 01:15:40 +0000 Subject: [PATCH 52/62] Resume incomplete ordinary-server matrices safely --- scripts/qualification_ordinary_server.py | 28 +++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/scripts/qualification_ordinary_server.py b/scripts/qualification_ordinary_server.py index ad922571..d9388d62 100644 --- a/scripts/qualification_ordinary_server.py +++ b/scripts/qualification_ordinary_server.py @@ -112,6 +112,7 @@ def _manifest(binary: Path, output_dir: Path) -> dict[str, Any]: "build_profile": "release", "build_mode": "locked release candidate", }, + "qualification_tooling_commit_sha": _commit(), "target": target, "workload": { "fresh_root_per_run": True, @@ -351,7 +352,32 @@ def run(binary: Path, output_dir: Path) -> int: output_dir.mkdir(parents=True, exist_ok=True) manifest_path = output_dir / "matrix.json" summary_path = output_dir / "summary.json" - manifest = _manifest(binary, output_dir) + fresh_manifest = _manifest(binary, output_dir) + manifest = fresh_manifest + if manifest_path.is_file(): + try: + previous = _load(manifest_path) + resumable = ( + previous.get("schema_version") == SCHEMA_VERSION + and previous.get("status") == "incomplete" + and previous.get("candidate", {}).get("sha256") + == fresh_manifest["candidate"]["sha256"] + and previous.get("target") == fresh_manifest["target"] + and previous.get("workload") == fresh_manifest["workload"] + and previous.get("conditions") == fresh_manifest["conditions"] + and isinstance( + previous.get("candidate", {}).get("repository_commit_sha"), str + ) + ) + if resumable: + manifest = previous + manifest["qualification_tooling_commit_sha"] = _commit() + manifest["preflight_io"] = fresh_manifest["preflight_io"] + manifest["status"] = fresh_manifest["status"] + manifest["stop_reason"] = fresh_manifest["stop_reason"] + manifest.pop("stopped_at", None) + except (OSError, ValueError, AttributeError): + manifest = fresh_manifest manifest_path.write_text(json.dumps(manifest, indent=2) + "\n", encoding="utf-8") if manifest["status"] == "blocked": return 77 From 63268e1e3d1efe8a524f679d16ef259c91a880ae Mon Sep 17 00:00:00 2001 From: wr3n-ai Date: Fri, 16 Oct 2026 01:17:37 +0000 Subject: [PATCH 53/62] Summarize ordinary-server latency and storage facts --- scripts/qualification_ordinary_server.py | 87 ++++++++++++++++++- .../test_qualification_ordinary_server.py | 41 ++++++++- 2 files changed, 125 insertions(+), 3 deletions(-) diff --git a/scripts/qualification_ordinary_server.py b/scripts/qualification_ordinary_server.py index d9388d62..70c487c0 100644 --- a/scripts/qualification_ordinary_server.py +++ b/scripts/qualification_ordinary_server.py @@ -300,6 +300,74 @@ def timing_median( return None return statistics.median(values) + sequential_keys = ( + "native_responses_finite", + "native_responses_streaming", + "translated_responses_to_messages_streaming", + ) + sequential_surfaces: dict[str, dict[str, int | float | None]] = {} + for surface_key in sequential_keys: + surface_runs = [ + run["document"] + .get("benchmark", {}) + .get("runs", {}) + .get(surface_key, {}) + for run in runs + ] + surface_summary: dict[str, int | float | None] = { + "sample_count": min( + ( + surface.get("sample_count", 0) + for surface in surface_runs + if isinstance(surface.get("sample_count"), int) + ), + default=0, + ) + } + for field in ( + "p50_elapsed_ms", + "p95_elapsed_ms", + "p50_ttft_ms", + "p95_ttft_ms", + ): + values = [surface.get(field) for surface in surface_runs] + surface_summary[field] = ( + statistics.median(values) + if len(values) == REPETITIONS + and all(isinstance(value, (int, float)) for value in values) + else None + ) + sequential_surfaces[surface_key] = surface_summary + + io_pressure_values = [ + run["load"] + .get("host_io", {}) + .get("io_pressure_after", {}) + .get("full", {}) + .get("avg10") + for run in runs + ] + write_delta_values = [ + run["load"].get("host_io", {}).get("root_device_write_delta_bytes") + for run in runs + ] + + wal_bytes: list[int] = [] + for run in runs: + raw_samples = run["document"].get("resource_samples", []) + if not isinstance(raw_samples, list): + continue + for raw_sample in cast("list[Any]", raw_samples): + if not isinstance(raw_sample, dict): + continue + sample = cast("dict[str, Any]", raw_sample) + value = sample.get("wal_bytes") + if isinstance(value, int): + wal_bytes.append(value) + durable_attempts = [ + run["document"].get("durable", {}).get("attempts") for run in runs + ] + summaries.append( { "principals": principals, @@ -321,6 +389,23 @@ def timing_median( run["document"]["benchmark"]["resource_summary"]["peak_rss_bytes"] for run in runs ), + "sequential_surfaces": sequential_surfaces, + "median_io_pressure_full_avg10_after_percent": ( + statistics.median(io_pressure_values) + if all( + isinstance(value, (int, float)) for value in io_pressure_values + ) + else None + ), + "median_root_device_write_delta_bytes": ( + statistics.median(write_delta_values) + if all(isinstance(value, int) for value in write_delta_values) + else None + ), + "maximum_observed_wal_bytes": max(wal_bytes, default=None), + "median_durable_attempts": statistics.median(durable_attempts) + if all(isinstance(value, int) for value in durable_attempts) + else None, "settlement": ( "zero pending requests and active reservations in every run" ), @@ -359,7 +444,7 @@ def run(binary: Path, output_dir: Path) -> int: previous = _load(manifest_path) resumable = ( previous.get("schema_version") == SCHEMA_VERSION - and previous.get("status") == "incomplete" + and previous.get("status") in {"incomplete", "complete"} and previous.get("candidate", {}).get("sha256") == fresh_manifest["candidate"]["sha256"] and previous.get("target") == fresh_manifest["target"] diff --git a/tests/tooling/test_qualification_ordinary_server.py b/tests/tooling/test_qualification_ordinary_server.py index 7c812a6f..812fc535 100644 --- a/tests/tooling/test_qualification_ordinary_server.py +++ b/tests/tooling/test_qualification_ordinary_server.py @@ -51,6 +51,10 @@ def _run(principals: int, concurrency: int, p95: int) -> dict[str, Any]: "failed_count": 0, "requests_per_second": 100.0, "cpu": {"cpu_percent": 50.0}, + "host_io": { + "io_pressure_after": {"full": {"avg10": 1.0}}, + "root_device_write_delta_bytes": 2048, + }, "timings": { "sample_count": REQUESTS_PER_CELL, "p50_elapsed_ms": 10, @@ -58,10 +62,36 @@ def _run(principals: int, concurrency: int, p95: int) -> dict[str, Any]: "p99_elapsed_ms": p95 + 1, "maximum_elapsed_ms": p95 + 2, }, - } + }, + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 10, + "p95_elapsed_ms": p95, + "p50_ttft_ms": 9, + "p95_ttft_ms": p95 - 1, + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 11, + "p95_elapsed_ms": p95 + 1, + "p50_ttft_ms": 8, + "p95_ttft_ms": p95, + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": p95 + 2, + "p50_ttft_ms": 9, + "p95_ttft_ms": p95 + 1, + }, }, }, - "durable": {"pending_requests": 0, "active_reservations": 0}, + "durable": { + "pending_requests": 0, + "active_reservations": 0, + "attempts": 1113, + }, + "resource_samples": [{"wal_bytes": 42}], } @@ -135,6 +165,13 @@ def test_m002_raw_to_summary_requires_three_durable_p99_runs(tmp_path: Path) -> assert cell["median_p95_ms"] == 22 assert cell["median_p99_ms"] == 23 assert cell["settlement"].startswith("zero pending") + assert ( + cell["sequential_surfaces"]["native_responses_streaming"]["p95_ttft_ms"] == 22 + ) + assert cell["median_io_pressure_full_avg10_after_percent"] == 1.0 + assert cell["median_root_device_write_delta_bytes"] == 2048 + assert cell["maximum_observed_wal_bytes"] == 42 + assert cell["median_durable_attempts"] == 1113 assert summary["lifecycle"]["phases"] == ["faults", "restart"] From f7343eb295ef3bb9e72b43211980934fe2ca2b45 Mon Sep 17 00:00:00 2001 From: wr3n-ai Date: Fri, 16 Oct 2026 01:41:52 +0000 Subject: [PATCH 54/62] Complete ordinary server qualification evidence tooling --- architecture/deep-dive-observability.md | 5 +- docs/deployment.md | 34 ++- docs/raspberry-pi.md | 6 +- scripts/qualification_ordinary_server.py | 63 ++++ scripts/qualification_sbc.py | 93 +++++- scripts/qualification_stability.py | 276 +++++++++++++++++- .../test_qualification_ordinary_server.py | 44 ++- 7 files changed, 482 insertions(+), 39 deletions(-) diff --git a/architecture/deep-dive-observability.md b/architecture/deep-dive-observability.md index caf8d726..15ff89ca 100644 --- a/architecture/deep-dive-observability.md +++ b/architecture/deep-dive-observability.md @@ -24,8 +24,9 @@ The offline M002 host-capacity corpus is owned by sanitized host/build identity, storage class, host I/O pressure, response percentiles, and durable settlement for synthetic loopback workloads. These artifacts are qualification evidence, not runtime telemetry or a production -capacity promise; the SQLite gate timing diagnostic remains opt-in and is not -inferred from host I/O counters. +capacity promise. One opt-in 60-request SQLite publication diagnostic records +gate wait/hold phases for this target; those samples are supplementary and are +not implied by, or substituted for, the per-cell host I/O counters. See [Metrics](deep-dive-metrics.md) for the coalescer contract (including the M002 ownership paragraph, which is authoritative) and diff --git a/docs/deployment.md b/docs/deployment.md index 388125e9..eafe8f9d 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -494,23 +494,31 @@ connect, header, TTFT, and total response latency. ### Ordinary-server team sizing evidence -The M002 synthetic reference uses an ordinary Linux host with durable -SSD/NVMe storage, WAL/NORMAL, and one database worker. It records CPU, RAM, -filesystem, device class/model, virtualization/container status, process RSS, -request throughput and percentiles, I/O pressure, and durable reservation +The M002 synthetic reference was measured on one ordinary Linux host with +durable SSD/NVMe storage, WAL/NORMAL, and one database worker. It records CPU, +RAM, filesystem, device class/model, virtualization/container status, process +RSS, request throughput and percentiles, I/O pressure, and durable reservation settlement. Named account rows (1/5/10/25/49) are measured separately from simultaneous inference requests (1/2/4/8/16); neither count is a service guarantee. The synthetic upstream cycles a fixed 0/10/50 ms service-time distribution to show how provider latency affects observed response time. +On the measured Intel i9-9900K host (16 logical CPUs, 15.5 GiB RAM, Samsung +980 NVMe), the median of three 1,000-request runs ranged from 36 to 43 +requests/s at concurrency 1, about 194 requests/s at concurrency 4, and +335–341 requests/s at concurrency 8–16. At concurrency 16, median p95/p99 +were 88/106 ms; maximum observed latency across the three runs was 1.119 s. +Peak process RSS across cells was 25–46 MiB. These values describe this +loopback synthetic workload; upstream latency, request shape, system load, +provider limits, and team policy change the result. No universal throughput, +concurrency, or latency SLA follows from one host and synthetic workload. + The driver writes the manifest, raw runs, and generated summary under `artifacts/qualification/on-prem-concurrency-performance/ordinary-server/`. -Until a complete run is recorded, there is no M002 capacity result. Once -available, use it only for relative sizing; qualify the actual provider mix, -model payloads, login rate, and policy/budget profile before deployment. No -universal throughput, concurrency, or latency SLA follows from one host and -synthetic workload. The Pi microSD envelope remains a separate solo-device -reference in [`raspberry-pi.md`](raspberry-pi.md). See the +Use the result only for relative sizing; qualify the actual provider mix, +model payloads, login rate, and policy/budget profile before deployment. The +Pi microSD envelope remains a separate solo-device reference in +[`raspberry-pi.md`](raspberry-pi.md). See the [M002 closure record](../plans/closure/on-prem-concurrency-performance/002-status.md) for its measured/unmeasured status and required follow-up evidence. @@ -519,9 +527,9 @@ To repeat the frozen corpus from a release binary, run The driver writes its manifest before collection, creates a fresh qualification root for each run, requires three repetitions per condition, validates p99 sample counts and settlement, derives `summary.json` from the raw Q008 -reports, and runs deterministic Q009 provider-fault and abrupt-restart phases -against the same binary. The summary requires the matching `lifecycle.json` -artifact. It refuses non-Linux hosts and non-durable or rotational qualification +reports, and runs deterministic Q009 provider-fault, hard-budget, user-disable, +and abrupt-restart phases against the same binary. The summary requires the +matching `lifecycle.json` artifact. It refuses non-Linux hosts and non-durable or rotational qualification storage, and stops before load when initial full I/O pressure exceeds 25% over the 10-second window. Qualification does not send requests to a real provider or charge a provider account. diff --git a/docs/raspberry-pi.md b/docs/raspberry-pi.md index e48a9a58..df67b204 100644 --- a/docs/raspberry-pi.md +++ b/docs/raspberry-pi.md @@ -274,8 +274,10 @@ saturates. The ordinary-server team baseline is maintained separately from this Pi microSD solo envelope; do not use the Pi limits as the team-host sizing source. -The current M002 result is conditional and contains no server capacity claim; -see its [closure record](../plans/closure/on-prem-concurrency-performance/002-status.md). +M002 now has a measured synthetic reference on one durable-NVMe ordinary Linux +host. Its host-specific observations and limits are in the +[closure record](../plans/closure/on-prem-concurrency-performance/002-status.md) +and [deployment guidance](deployment.md#ordinary-server-team-sizing-evidence). ## Verify from LAN diff --git a/scripts/qualification_ordinary_server.py b/scripts/qualification_ordinary_server.py index 70c487c0..8b4b14d2 100644 --- a/scripts/qualification_ordinary_server.py +++ b/scripts/qualification_ordinary_server.py @@ -201,6 +201,63 @@ def summarize(manifest_path: Path) -> dict[str, Any]: observed_phases.add(name) if not required_phases <= observed_phases: raise ValueError("ordinary-server lifecycle phases are incomplete") + phase_by_name: dict[str, dict[str, Any]] = {} + for raw_phase in lifecycle_phases: + if not isinstance(raw_phase, dict): + continue + phase = cast("dict[str, Any]", raw_phase) + name = phase.get("phase") + if isinstance(name, str): + phase_by_name[name] = phase + restart_phase = phase_by_name.get("restart", {}) + workload_policy = lifecycle.get("workload_policy") + if not isinstance(workload_policy, dict): + raise ValueError("ordinary-server lifecycle workload policy is missing") + workload_policy = cast("dict[str, Any]", workload_policy) + if ( + workload_policy.get("budget_mode") != "hard user and global request ceilings" + or workload_policy.get("principal_count") != 2 + or restart_phase.get("hard_budgets_active") is not True + or restart_phase.get("user_disabled_during_load") is not True + or restart_phase.get("credential_revocation_during_load") is not True + or restart_phase.get("identity_denial_http_status") not in {401, 403} + ): + raise ValueError("ordinary-server managed recovery evidence is incomplete") + fault_phase = phase_by_name.get("faults", {}) + raw_observations = fault_phase.get("observations") + if not isinstance(raw_observations, list): + raise ValueError("ordinary-server fault observations are missing") + raw_observations = cast("list[Any]", raw_observations) + observations: list[dict[str, Any]] = [ + cast("dict[str, Any]", observation) + for observation in raw_observations + if isinstance(observation, dict) + ] + denial_counts = { + "provider_429": sum( + observation.get("category") == "provider" + and observation.get("expected_upstream_status") == 429 + and observation.get("client_http_status") == 429 + for observation in observations + ), + "user_budget_429": sum( + observation.get("category") == "budget" + and observation.get("denial_kind") == "user_budget" + and observation.get("http_status") == 429 + for observation in observations + ), + "global_budget_429": sum( + observation.get("category") == "budget" + and observation.get("denial_kind") == "global_budget" + and observation.get("http_status") == 429 + for observation in observations + ), + "identity_denial": 1, + } + if any( + denial_counts[name] != 1 for name in denial_counts if name != "identity_denial" + ): + raise ValueError("ordinary-server provider/budget denial counts are incomplete") output_dir = manifest_path.parent conditions = manifest.get("conditions") if not isinstance(conditions, list): @@ -424,6 +481,11 @@ def timing_median( "phases": sorted(required_phases), "provider": lifecycle.get("provider"), "ownership_convergence": lifecycle.get("ownership_convergence"), + "workload_policy": workload_policy, + "denial_counts": denial_counts, + "identity_denial_http_status": restart_phase.get( + "identity_denial_http_status" + ), }, "interpretation": ( "Descriptive loopback capacity characterization for this host and " @@ -541,6 +603,7 @@ def run(binary: Path, output_dir: Path) -> int: "faults", "--phase", "restart", + "--exercise-revocation-during-restart", "--output", str(lifecycle_path), ] diff --git a/scripts/qualification_sbc.py b/scripts/qualification_sbc.py index 020d7c17..819d0057 100644 --- a/scripts/qualification_sbc.py +++ b/scripts/qualification_sbc.py @@ -1203,11 +1203,19 @@ def _runtime_task_snapshot(value: Mapping[str, Any]) -> dict[str, dict[str, Any] return snapshot -def _runtime_json(runtime_url: str, server_api_key: str) -> dict[str, Any] | None: +def _runtime_json( + runtime_url: str, + server_api_key: str = "q008-server-key", + *, + session_cookie: str | None = None, +) -> dict[str, Any] | None: try: - status, body = _http( - runtime_url, headers={"Authorization": f"Bearer {server_api_key}"} + headers = ( + {"Cookie": session_cookie} + if session_cookie is not None + else {"Authorization": f"Bearer {server_api_key}"} ) + status, body = _http(runtime_url, headers=headers) if status != 200: return None value = json.loads(body.decode("utf-8")) @@ -1221,11 +1229,14 @@ def _wait_for_diagnostic_quiescence( *, timeout: float = DIAGNOSTIC_QUIESCENCE_TIMEOUT, server_api_key: str = "q008-server-key", + session_cookie: str | None = None, ) -> dict[str, dict[str, Any]]: """Wait for startup checkpoint/metrics work and a quiescent task window.""" deadline = time.monotonic() + timeout while time.monotonic() < deadline: - value = _runtime_json(runtime_url, server_api_key) + value = _runtime_json( + runtime_url, server_api_key, session_cookie=session_cookie + ) tasks = _runtime_task_snapshot(value or {}) checkpoint = tasks.get("checkpoint") metrics_flush = tasks.get("metrics_flush") @@ -1323,6 +1334,7 @@ def resource_sample( *, duration: float = SAMPLE_SECONDS, server_api_key: str = "q008-server-key", + session_cookie: str | None = None, include_peak: bool = False, ) -> dict[str, Any]: """Capture a bounded procfs/runtime/database snapshot. @@ -1343,7 +1355,12 @@ def resource_sample( runtime_read_error = False try: _, body = _http( - runtime_url, headers={"Authorization": f"Bearer {server_api_key}"} + runtime_url, + headers=( + {"Cookie": session_cookie} + if session_cookie is not None + else {"Authorization": f"Bearer {server_api_key}"} + ), ) runtime_value = json.loads(body.decode("utf-8")) runtime = ( @@ -2243,9 +2260,9 @@ def _finite_diagnostic_payload() -> bytes: ).encode() -def _diagnostic_headers() -> dict[str, str]: +def _diagnostic_headers(api_token: str = "q008-server-key") -> dict[str, str]: return { - "Authorization": "Bearer q008-server-key", + "Authorization": f"Bearer {api_token}", "Content-Type": "application/json", } @@ -2374,10 +2391,10 @@ def _direct_provider_control( } -def _diagnostic_finite_warmup(port: int) -> None: +def _diagnostic_finite_warmup(port: int, api_token: str = "q008-server-key") -> None: for _ in range(DIAGNOSTIC_WARMUPS): status, body, _, _ = _benchmark_request( - port, MODELS["responses"], False, "responses" + port, MODELS["responses"], False, "responses", api_token=api_token ) if status != 200 or not body: raise QualificationError( @@ -2395,6 +2412,7 @@ def _diagnostic_finite_batch( timeout: float = DIAGNOSTIC_TIMEOUT, warmup: bool = True, wal_diagnostics: bool = True, + api_token: str = "q008-server-key", ) -> dict[str, Any]: """Run a sequential native-finite diagnostic batch with phase attribution. @@ -2409,9 +2427,9 @@ def _diagnostic_finite_batch( ) url = f"http://127.0.0.1:{port}/v1/responses" payload = _finite_diagnostic_payload() - headers = _diagnostic_headers() + headers = _diagnostic_headers(api_token) if warmup: - _diagnostic_finite_warmup(port) + _diagnostic_finite_warmup(port, api_token) provider.start_diagnostic(capacity=samples) observations: list[DiagnosticSample] = [] wal_before = ( @@ -2767,6 +2785,36 @@ def _command_token( return match.group(0) +def _synthetic_admin_session_cookie(port: int, password: str, timeout: float) -> str: + """Create an ephemeral browser session for protected runtime diagnostics.""" + jar = http.cookiejar.CookieJar() + opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(jar)) + origin = f"http://127.0.0.1:{port}" + try: + opener.open(f"{origin}/login", timeout=timeout).read() + login = urllib.request.Request( + f"{origin}/login", + data=urllib.parse.urlencode( + {"username": "sbc-admin", "password": password} + ).encode(), + headers={ + "Content-Type": "application/x-www-form-urlencoded", + "Origin": origin, + "Referer": f"{origin}/login", + }, + method="POST", + ) + opener.open(login, timeout=timeout).read() + except (OSError, urllib.error.URLError, TimeoutError) as error: + raise QualificationError( + "synthetic admin browser session could not be created" + ) from error + cookie = next((item for item in jar if item.name == "eggpool_session"), None) + if cookie is None: + raise QualificationError("synthetic admin browser session was not issued") + return f"eggpool_session={cookie.value}" + + def _configure_benchmark_principals( *, binary: Path, @@ -3562,6 +3610,14 @@ def run_qualification( report["startup_to_ready_ms"] = round( (time.monotonic() - started) * 1000 ) + diagnostic_session_cookie = ( + _synthetic_admin_session_cookie( + port, SYNTHETIC_ADMIN_PASSWORD, timeout + ) + if diagnose_publication_phases + or diagnose_publication_storage is not None + else None + ) for step_id, path in ( ("startup-readyz", "/v1/readyz"), ("model-listing", "/v1/models"), @@ -3811,18 +3867,21 @@ def run_qualification( benchmark_process, database, diagnostic_runtime_url, + session_cookie=diagnostic_session_cookie, include_peak=True, ) ) benchmark["runs"]["direct_provider_control"] = ( _direct_provider_control(provider) ) - _diagnostic_finite_warmup(port) + _diagnostic_finite_warmup(port, agent_token) baseline_tasks = _wait_for_diagnostic_quiescence( - diagnostic_runtime_url + diagnostic_runtime_url, + session_cookie=diagnostic_session_cookie, ) baseline_runtime = _runtime_json( - diagnostic_runtime_url, "q008-server-key" + diagnostic_runtime_url, + session_cookie=diagnostic_session_cookie, ) if baseline_runtime is None: raise QualificationError( @@ -3874,6 +3933,7 @@ def run_qualification( wal_diagnostics=( qualification_persist_journal_mode != "candidate" ), + api_token=agent_token, ) if ( phase_run["completed_count"] != PUBLICATION_PHASE_SAMPLES @@ -3888,7 +3948,8 @@ def run_qualification( f"failed={phase_run['failed_count']}" ) final_runtime = _runtime_json( - diagnostic_runtime_url, "q008-server-key" + diagnostic_runtime_url, + session_cookie=diagnostic_session_cookie, ) if final_runtime is None: raise QualificationError( @@ -4341,6 +4402,7 @@ def run_qualification( benchmark_process, database, diagnostic_runtime_url, + session_cookie=diagnostic_session_cookie, include_peak=True, ) ) @@ -4349,6 +4411,7 @@ def run_qualification( benchmark_process, database, diagnostic_runtime_url, + session_cookie=diagnostic_session_cookie, duration=STABILIZATION_SECONDS, include_peak=True, ) diff --git a/scripts/qualification_stability.py b/scripts/qualification_stability.py index d103a2ba..8fb828cb 100644 --- a/scripts/qualification_stability.py +++ b/scripts/qualification_stability.py @@ -368,6 +368,64 @@ def _command( } +def _set_user_request_budget( + binary: Path, + config: Path, + env: Mapping[str, str], + limit: int, + timeout: float, + username: str = "q009-admin", +) -> dict[str, Any]: + return _command( + binary, + config, + env, + ( + "users", + "budget", + "set", + username, + "--window", + "day", + "--max-requests", + str(limit), + "--enforcement", + "hard", + "--actor", + "q009-admin", + ), + timeout, + ) + + +def _set_global_request_budget( + binary: Path, + config: Path, + env: Mapping[str, str], + limit: int, + timeout: float, +) -> dict[str, Any]: + return _command( + binary, + config, + env, + ( + "users", + "global-budget", + "set", + "--window", + "day", + "--max-requests", + str(limit), + "--enforcement", + "hard", + "--actor", + "q009-admin", + ), + timeout, + ) + + def _wait_ready(process: subprocess.Popen[str], url: str, timeout: float) -> None: deadline = time.monotonic() + timeout while time.monotonic() < deadline: @@ -547,6 +605,7 @@ def run_qualification( restart_cycles: int = 1, timeout: float = DEFAULT_TIMEOUT, phases: frozenset[str] | None = None, + exercise_revocation_during_restart: bool = False, ) -> dict[str, Any]: """Execute stability qualification and return a bounded report.""" _validate_cycles( @@ -665,12 +724,136 @@ def run_qualification( agent_token = token_match.group(0) commands.append( { - "command": ["users", "bootstrap", "q009-admin"], + "command": ["users", "bootstrap"], "status": "pass", "duration_ms": round((time.monotonic() - started) * 1000), "reason": "synthetic managed principal created", } ) + policy = _command( + binary, + config, + env, + ( + "users", + "policy", + "create", + "q009-disable-policy", + "Q009 Disable Qualification", + "allow", + "--actor", + "q009-admin", + ), + timeout, + ) + commands.append(policy) + if policy["status"] != "pass": + raise QualificationError("synthetic disable policy setup failed") + member_password = "Q009 synthetic member password 123" + try: + member_add = subprocess.run( + [ + str(binary), + "--config", + str(config), + "users", + "add", + "q009-disablee", + "--password-stdin", + "--profile-id", + "q009-disable-policy", + "--actor", + "q009-admin", + ], + cwd=ROOT, + env=dict(env), + input=f"{member_password}\n", + capture_output=True, + text=True, + timeout=timeout, + check=False, + ) + except (OSError, subprocess.TimeoutExpired) as error: + raise QualificationError( + "synthetic disable principal setup failed" + ) from error + commands.append( + { + "command": ["users", "add"], + "status": "pass" if member_add.returncode == 0 else "fail", + "returncode": member_add.returncode, + "duration_ms": round((time.monotonic() - started) * 1000), + "reason": "synthetic disable principal setup", + } + ) + if member_add.returncode != 0: + raise QualificationError("synthetic disable principal setup failed") + try: + member_token_result = subprocess.run( + [ + str(binary), + "--config", + str(config), + "users", + "token", + "q009-disablee", + "--label", + "q009-disable-test", + "--show-secret", + ], + cwd=ROOT, + env=dict(env), + capture_output=True, + text=True, + timeout=timeout, + check=False, + ) + except (OSError, subprocess.TimeoutExpired) as error: + raise QualificationError( + "synthetic disable credential setup failed" + ) from error + member_token_match = ( + re.search(r"epk1_[A-Za-z0-9_-]+", member_token_result.stdout) + if member_token_result.returncode == 0 + else None + ) + commands.append( + { + "command": ["users", "token"], + "status": "pass" if member_token_match else "fail", + "returncode": member_token_result.returncode, + "duration_ms": round((time.monotonic() - started) * 1000), + "reason": "synthetic disable credential issued", + } + ) + if member_token_match is None: + raise QualificationError( + "synthetic disable credential setup failed" + ) + disable_token = member_token_match.group(0) + for budget_command in ( + _set_user_request_budget(binary, config, env, 100, timeout), + _set_user_request_budget( + binary, + config, + env, + 100, + timeout, + username="q009-disablee", + ), + _set_global_request_budget(binary, config, env, 100, timeout), + ): + commands.append(budget_command) + if budget_command["status"] != "pass": + raise QualificationError( + "synthetic hard request budgets could not be enabled" + ) + report["workload_policy"] = { + "principal_count": 2, + "budget_mode": "hard user and global request ceilings", + "initial_user_request_limit": 100, + "initial_global_request_limit": 100, + } process = _start(binary, config, env) _wait_ready(process, f"http://127.0.0.1:{port}/v1/healthz", timeout) report["environment"] = { @@ -888,6 +1071,49 @@ def run_qualification( "http_status": status, } ) + budget_cases = ( + ( + "user_budget", + _set_user_request_budget, + ), + ( + "global_budget", + _set_global_request_budget, + ), + ) + for denial_kind, set_limit in budget_cases: + configured = set_limit(binary, config, env, 1, timeout) + commands.append(configured) + if configured["status"] != "pass": + raise QualificationError( + f"{denial_kind} denial fixture setup failed" + ) + upstream_before = provider.total() + status, _body, _, _ = _post( + port, + "chat_completions", + "q009-chat", + api_token=agent_token, + ) + if status != 429 or provider.total() != upstream_before: + raise QualificationError( + f"{denial_kind} did not reject before provider: " + f"HTTP {status}" + ) + fault_results.append( + { + "category": "budget", + "denial_kind": denial_kind, + "http_status": status, + "provider_requests": 0, + } + ) + restored = set_limit(binary, config, env, 100, timeout) + commands.append(restored) + if restored["status"] != "pass": + raise QualificationError( + f"{denial_kind} budget restore failed" + ) _wait_converged(database, process, port, samples, timeout) _resource_sample("after-faults", process, database, port, samples) report["phases"].append( @@ -979,7 +1205,7 @@ def in_flight() -> None: port, "chat_completions", "q009-abrupt", - api_token=agent_token, + api_token=disable_token, ) except OSError as error: request_error.append(type(error).__name__) @@ -996,6 +1222,42 @@ def in_flight() -> None: raise QualificationError( "abrupt-restart fixture did not reach upstream" ) + identity_status: int | None = None + if exercise_revocation_during_restart: + disable = _command( + binary, + config, + env, + ( + "users", + "disable", + "q009-disablee", + "--actor", + "q009-admin", + ), + timeout, + ) + disable["command"] = ["users", "disable"] + commands.append(disable) + if disable["status"] != "pass": + raise QualificationError( + "synthetic user disable failed during load" + ) + upstream_before_denied_request = provider.total() + identity_status, _identity_body, _, _ = _post( + port, + "chat_completions", + "q009-chat", + api_token=disable_token, + ) + if ( + identity_status not in {401, 403} + or provider.total() != upstream_before_denied_request + ): + raise QualificationError( + "disabled principal was not denied before provider " + "dispatch" + ) os.killpg(process.pid, signal.SIGKILL) process.wait(timeout=5) process = None @@ -1016,6 +1278,14 @@ def in_flight() -> None: "pass", cycles=restart_cycles, unknown_inflight_replayed=False, + hard_budgets_active=True, + user_disabled_during_load=( + exercise_revocation_during_restart + ), + credential_revocation_during_load=( + exercise_revocation_during_restart + ), + identity_denial_http_status=identity_status, client_outcome="cancelled-or-closed" if request_error else "closed", @@ -1095,6 +1365,7 @@ def _parser() -> argparse.ArgumentParser: parser.add_argument("--fault-cycles", type=int, default=1) parser.add_argument("--reload-cycles", type=int, default=2) parser.add_argument("--restart-cycles", type=int, default=1) + parser.add_argument("--exercise-revocation-during-restart", action="store_true") parser.add_argument( "--phase", dest="phases", @@ -1120,6 +1391,7 @@ def main(argv: Sequence[str] | None = None) -> int: restart_cycles=args.restart_cycles, timeout=args.timeout, phases=frozenset(args.phases) if args.phases else None, + exercise_revocation_during_restart=args.exercise_revocation_during_restart, ) except ( OSError, diff --git a/tests/tooling/test_qualification_ordinary_server.py b/tests/tooling/test_qualification_ordinary_server.py index 812fc535..c377c8d6 100644 --- a/tests/tooling/test_qualification_ordinary_server.py +++ b/tests/tooling/test_qualification_ordinary_server.py @@ -144,12 +144,43 @@ def test_m002_raw_to_summary_requires_three_durable_p99_runs(tmp_path: Path) -> "schema_version": "runtime-q009.v1", "status": "pass", "candidate": {"sha256": "a" * 64}, - "phases": [ - {"phase": "faults", "status": "pass"}, - {"phase": "restart", "status": "pass"}, - ], "provider": {"request_count": 5}, "ownership_convergence": {"pending_requests": 0}, + "workload_policy": { + "principal_count": 2, + "budget_mode": "hard user and global request ceilings", + }, + "phases": [ + { + "phase": "faults", + "status": "pass", + "observations": [ + { + "category": "provider", + "expected_upstream_status": 429, + "client_http_status": 429, + }, + { + "category": "budget", + "denial_kind": "user_budget", + "http_status": 429, + }, + { + "category": "budget", + "denial_kind": "global_budget", + "http_status": 429, + }, + ], + }, + { + "phase": "restart", + "status": "pass", + "hard_budgets_active": True, + "user_disabled_during_load": True, + "credential_revocation_during_load": True, + "identity_denial_http_status": 401, + }, + ], } ), encoding="utf-8", @@ -211,7 +242,10 @@ def test_m002_raw_to_summary_requires_passing_same_candidate_lifecycle( { "schema_version": SCHEMA_VERSION, "status": "complete", - "candidate": {"sha256": "a" * 64}, + "candidate": { + "sha256": "a" * 64, + "repository_commit_sha": "c" * 40, + }, "lifecycle_artifact": "lifecycle.json", "conditions": [ { From deda832de2acb86d40ba69ba02407ffa6150a246 Mon Sep 17 00:00:00 2001 From: wr3n-ai Date: Fri, 16 Oct 2026 01:42:51 +0000 Subject: [PATCH 55/62] Close ordinary server baseline and unblock M023 --- .../ordinary-server/disk-contention.json | 1080 +++++++++++++ .../lifecycle-failed-attempt-1.json | 7 + .../ordinary-server/lifecycle.json | 502 ++++++ .../ordinary-server/matrix.json | 174 +++ ...iagnostic-failed-shared-key-attempt-1.json | 7 + ...e-gate-diagnostic-managed-runtime-key.json | 7 + .../sqlite-gate-diagnostic-managed.json | 7 + .../sqlite-gate-diagnostic-session.json | 1350 +++++++++++++++++ .../sqlite-gate-diagnostic.json | 7 + .../ordinary-server/summary.json | 522 +++++++ .../u1-c1-run-1-failed-attempt-1.json | 7 + .../ordinary-server/u1-c1-run-1.json | 1080 +++++++++++++ .../ordinary-server/u1-c1-run-2.json | 1080 +++++++++++++ .../ordinary-server/u1-c1-run-3.json | 1080 +++++++++++++ .../ordinary-server/u10-c1-run-1.json | 1080 +++++++++++++ .../ordinary-server/u10-c1-run-2.json | 1080 +++++++++++++ .../ordinary-server/u10-c1-run-3.json | 1080 +++++++++++++ .../ordinary-server/u25-c1-run-1.json | 1080 +++++++++++++ .../ordinary-server/u25-c1-run-2.json | 1080 +++++++++++++ .../ordinary-server/u25-c1-run-3.json | 1080 +++++++++++++ .../ordinary-server/u49-c1-run-1.json | 1080 +++++++++++++ .../ordinary-server/u49-c1-run-2.json | 1080 +++++++++++++ .../ordinary-server/u49-c1-run-3.json | 1080 +++++++++++++ .../ordinary-server/u5-c1-run-1.json | 1080 +++++++++++++ .../ordinary-server/u5-c1-run-2.json | 1080 +++++++++++++ .../ordinary-server/u5-c1-run-3.json | 1080 +++++++++++++ .../ordinary-server/u5-c16-run-1.json | 1080 +++++++++++++ .../ordinary-server/u5-c16-run-2.json | 1080 +++++++++++++ .../ordinary-server/u5-c16-run-3.json | 1080 +++++++++++++ .../ordinary-server/u5-c2-run-1.json | 1080 +++++++++++++ .../ordinary-server/u5-c2-run-2.json | 1080 +++++++++++++ .../ordinary-server/u5-c2-run-3.json | 1080 +++++++++++++ .../ordinary-server/u5-c4-run-1.json | 1080 +++++++++++++ .../ordinary-server/u5-c4-run-2.json | 1080 +++++++++++++ .../ordinary-server/u5-c4-run-3.json | 1080 +++++++++++++ .../ordinary-server/u5-c8-run-1.json | 1080 +++++++++++++ .../ordinary-server/u5-c8-run-2.json | 1080 +++++++++++++ .../ordinary-server/u5-c8-run-3.json | 1080 +++++++++++++ docs/deployment.md | 7 +- .../002-status.md | 189 ++- .../023-short-window-fairness-controls.md | 6 +- plans/registry.md | 21 +- ...on-prem-concurrency-performance-roadmap.md | 18 +- .../subsystems/on-prem-user-access-roadmap.md | 18 +- 44 files changed, 33008 insertions(+), 81 deletions(-) create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/disk-contention.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/lifecycle-failed-attempt-1.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/lifecycle.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/matrix.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-failed-shared-key-attempt-1.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-managed-runtime-key.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-managed.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-session.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/summary.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u1-c1-run-1-failed-attempt-1.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u1-c1-run-1.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u1-c1-run-2.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u1-c1-run-3.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u10-c1-run-1.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u10-c1-run-2.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u10-c1-run-3.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u25-c1-run-1.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u25-c1-run-2.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u25-c1-run-3.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u49-c1-run-1.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u49-c1-run-2.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u49-c1-run-3.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c1-run-1.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c1-run-2.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c1-run-3.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c16-run-1.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c16-run-2.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c16-run-3.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c2-run-1.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c2-run-2.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c2-run-3.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c4-run-1.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c4-run-2.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c4-run-3.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c8-run-1.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c8-run-2.json create mode 100644 artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c8-run-3.json diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/disk-contention.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/disk-contention.json new file mode 100644 index 00000000..80af8631 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/disk-contention.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "ed73390647eb89246aeec636e791b016cf5ccb73e0dd96b583126c0d81a2f168", + "archive_size_bytes": 4672983, + "elapsed_ms": 596, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 4, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23797760, + "baseline_rss_bytes": 23797760, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46608384, + "peak_rss_bytes": 46608384 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 5814, + "client_concurrency": 4, + "competing_io": { + "bytes_written": 24379392, + "configured_mib_per_second": 4, + "elapsed_ms": 5815, + "enabled": true, + "maximum_mib": 32, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 48.33, + "cpu_ticks": 281, + "cpu_time_ms": 2810 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 6.06, + "avg300": 7.68, + "avg60": 1.56 + }, + "some": { + "avg10": 6.06, + "avg300": 7.84, + "avg60": 1.56 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.03, + "avg300": 7.57, + "avg60": 0.36 + }, + "some": { + "avg10": 0.03, + "avg300": 7.74, + "avg60": 0.36 + } + }, + "root_device_read_delta_bytes": 360579072, + "root_device_write_delta_bytes": 412164096 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 171.992, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 159, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 85, + "p95_ttft_ms": 85, + "p99_elapsed_ms": 125, + "p99_ttft_ms": 125, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 5, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 74, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 10, + "p50_ttft_ms": 7, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39157144, + "build_mode": "q005-qualified-aarch64-copy", + "build_profile": "release", + "repository_commit_sha": "63268e1e3d1efe8a524f679d16ef259c91a880ae", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "11670f0c0880335020cbb41574d0a368aaebe3bea5329751b4fe321ea1cb3726" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-z_bzfxtk/config.toml Server: 127.0.0.1:49851 Accounts: 1 Database: Content digest: 0771a592877c1032e73bb6e83c907b3fef8f44c727cdd789ec7b80e4a781ba2c", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 85, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2451687, \"ppid\": 2451665, \"uptime_seconds\": 2.91703492, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, ", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 596, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-z_bzfxtk/backups/eggpool-backup-20261016-013238.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 105, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2451687)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 378, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-z_bzfxtk/recovery/config.toml db: /tmp/eggpool-q008-z_bzfxtk/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 204, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop", + "reason": "Stopping server (PID 2452954)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.07, + "avg300": 7.72, + "avg60": 0.39 + }, + "some": { + "avg10": 0.07, + "avg300": 7.9, + "avg60": 0.39 + } + }, + "root_device_read_bytes": 731942481920, + "root_device_write_bytes": 1303070368768 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 11 + }, + { + "body_bytes": 309, + "elapsed_ms": 9, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 213, + "elapsed_ms": 9, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 145, + "elapsed_ms": 19, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 270, + "elapsed_ms": 20, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 480, + "elapsed_ms": 60, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21639168, + 21639168, + 23797760, + 23797760, + 23924736, + 24051712, + 24227840, + 46608384, + 46608384, + 46608384, + 24260608 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 12, + 9, + 9 + ], + "finite_ttft_ms": [ + 11, + 9, + 9 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 19, + 20, + 60 + ], + "streaming_ttft_ms": [ + 17, + 17, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21639168, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21639168, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103714816, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21639168, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21639168, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103714816, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23797760, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23797760, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173080576, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23797760, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23797760, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173080576, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23924736, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23924736, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173080576, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24051712, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24051712, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173080576, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24227840, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24227840, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173219840, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4640768, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46608384, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46608384, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173826048, + "wal_bytes": 4375472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4640768, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46608384, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46608384, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173826048, + "wal_bytes": 4375472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4640768, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46608384, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46608384, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173826048, + "wal_bytes": 4375472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 24260608, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24260608, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103747584, + "wal_bytes": 873472, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/lifecycle-failed-attempt-1.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/lifecycle-failed-attempt-1.json new file mode 100644 index 00000000..883d4f7d --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/lifecycle-failed-attempt-1.json @@ -0,0 +1,7 @@ +{ + "manifest": "runtime-q001.v1", + "plan": "stability qualification", + "reason": "fault recovery failed for q009-fault-408: HTTP 401", + "schema_version": "runtime-q009.v1", + "status": "fail" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/lifecycle.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/lifecycle.json new file mode 100644 index 00000000..79662227 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/lifecycle.json @@ -0,0 +1,502 @@ +{ + "candidate": { + "binary_size_bytes": 38939984, + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "check-config" + ], + "duration_ms": 2, + "reason": "Configuration loaded successfully from /tmp/eggpool-q009-uhp1o7s8/config.toml Server: 127.0.0.1:36243 Accounts: 3 Database: /tmp/eggpool-q009-uhp1o7s8/data-home/usage.sqlite3 Content digest: 512cb68c4dddad9305ce774a58ea749ec3a0e34db8d5a6e765f10ae33c34150c", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "migrate" + ], + "duration_ms": 84, + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "bootstrap" + ], + "duration_ms": 204, + "reason": "synthetic managed principal created", + "status": "pass" + }, + { + "command": [ + "users", + "policy", + "create", + "q009-disable-policy", + "Q009 Disable Qualification", + "allow", + "--actor", + "q009-admin" + ], + "duration_ms": 17, + "reason": "Model policy profile q009-disable-policy created.", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "add" + ], + "duration_ms": 259, + "reason": "synthetic disable principal setup", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "token" + ], + "duration_ms": 276, + "reason": "synthetic disable credential issued", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "budget", + "set", + "q009-admin", + "--window", + "day", + "--max-requests", + "100", + "--enforcement", + "hard", + "--actor", + "q009-admin" + ], + "duration_ms": 17, + "reason": "Budget rule for q009-admin saved (hard).", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "budget", + "set", + "q009-disablee", + "--window", + "day", + "--max-requests", + "100", + "--enforcement", + "hard", + "--actor", + "q009-admin" + ], + "duration_ms": 17, + "reason": "Budget rule for q009-disablee saved (hard).", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "global-budget", + "set", + "--window", + "day", + "--max-requests", + "100", + "--enforcement", + "hard", + "--actor", + "q009-admin" + ], + "duration_ms": 18, + "reason": "Instance budget ceiling saved (hard).", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "budget", + "set", + "q009-admin", + "--window", + "day", + "--max-requests", + "1", + "--enforcement", + "hard", + "--actor", + "q009-admin" + ], + "duration_ms": 13, + "reason": "Budget rule for q009-admin saved (hard).", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "budget", + "set", + "q009-admin", + "--window", + "day", + "--max-requests", + "100", + "--enforcement", + "hard", + "--actor", + "q009-admin" + ], + "duration_ms": 5, + "reason": "Budget rule for q009-admin saved (hard).", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "global-budget", + "set", + "--window", + "day", + "--max-requests", + "1", + "--enforcement", + "hard", + "--actor", + "q009-admin" + ], + "duration_ms": 5, + "reason": "Instance budget ceiling saved (hard).", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "global-budget", + "set", + "--window", + "day", + "--max-requests", + "100", + "--enforcement", + "hard", + "--actor", + "q009-admin" + ], + "duration_ms": 5, + "reason": "Instance budget ceiling saved (hard).", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "disable" + ], + "duration_ms": 10, + "reason": "User q009-disablee disabled; browser sessions and personal tokens were revoked.", + "returncode": 0, + "status": "pass" + } + ], + "configuration": { + "fault_cycles": 1, + "reload_cycles": 2, + "restart_cycles": 1, + "steady_cycles": 2, + "timeout_s": 20.0, + "warmup_cycles": 1 + }, + "durable": { + "active_reservations": 0, + "attempts": 14, + "pending_requests": 0, + "request_statuses": { + "completed": 4, + "error": 6, + "interrupted": 1 + }, + "requests": 11, + "reservations": 14 + }, + "duration_ms": 2209, + "environment": { + "architecture": "x86_64", + "network_policy": "loopback-only", + "platform": "Linux", + "system": "posix" + }, + "findings": [], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "ownership_convergence": { + "active_leases": null, + "active_reservations": 0, + "finalization_jobs": null, + "logical_leaks": false, + "pending_requests": 0, + "retiring_generations": null, + "terminal_references": 0 + }, + "phases": [ + { + "cycles": 1, + "observations": [ + { + "category": "provider", + "client_http_status": 200, + "expected_upstream_status": 408, + "model": "q009-fault-408", + "provider_requests": 2, + "response_bytes": 242 + }, + { + "category": "provider", + "client_http_status": 429, + "expected_upstream_status": 429, + "model": "q009-fault-429", + "post_fault_server_restart": "pass", + "provider_requests": 1, + "response_bytes": 37 + }, + { + "category": "provider", + "client_http_status": 200, + "expected_upstream_status": 503, + "model": "q009-fault-5xx", + "provider_requests": 2, + "response_bytes": 242 + }, + { + "category": "provider", + "client_http_status": 200, + "expected_upstream_status": "timeout", + "model": "q009-timeout", + "provider_requests": 1, + "response_bytes": 242 + }, + { + "http_status": 200, + "model": "q009-partial", + "terminal_evidence": false + }, + { + "http_status": 200, + "model": "q009-disconnect", + "terminal_evidence": false + }, + { + "model": "q009-cancel", + "status": "client-cancelled-and-write-aborted" + }, + { + "http_status": 200, + "model": "q009-wire", + "status": "alternate-wire-success" + }, + { + "http_status": 200, + "model": "q009-malformed", + "status": "malformed-incomplete" + }, + { + "http_status": 404, + "model": "q009-absent", + "status": "model-absent" + }, + { + "http_status": 503, + "model": "q009-connect/unreachable", + "status": "connect-failure" + }, + { + "category": "budget", + "denial_kind": "user_budget", + "http_status": 429, + "provider_requests": 0 + }, + { + "category": "budget", + "denial_kind": "global_budget", + "http_status": 429, + "provider_requests": 0 + } + ], + "phase": "faults", + "status": "pass" + }, + { + "client_outcome": "cancelled-or-closed", + "credential_revocation_during_load": true, + "cycles": 1, + "hard_budgets_active": true, + "identity_denial_http_status": 401, + "phase": "restart", + "status": "pass", + "unknown_inflight_replayed": false, + "user_disabled_during_load": true + } + ], + "plan": "stability qualification", + "provider": { + "request_count": 13, + "requests_by_model": { + "q009-abrupt": 1, + "q009-absent": 1, + "q009-disconnect": 1, + "q009-fault-408": 2, + "q009-fault-429": 1, + "q009-fault-5xx": 2, + "q009-malformed": 1, + "q009-partial": 1, + "q009-timeout": 1, + "q009-wire": 2 + } + }, + "resource_analysis": { + "fd_counts": [ + 14, + 14, + 14 + ], + "interpretation": "bounded characterization; warmup/high-water allocator RSS is not treated as a leak and no SLA is inferred", + "rss_bytes": [ + 21532672, + 23384064, + 21860352 + ], + "sample_count": 3, + "thread_counts": [ + 2, + 2, + 2 + ] + }, + "resource_evidence": { + "runtime_read_errors": 0, + "semantics_affected": false, + "status": "pass" + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21532672, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 50, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103714816, + "wal_bytes": 111272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 13, + "cpu_percent": 0.0, + "db_bytes": 847872, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-faults", + "open_fd_count": 14, + "pending_requests": 0, + "request_statuses": { + "completed": 4, + "error": 6 + }, + "requests": 10, + "reservations": 13, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23384064, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 50, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103833600, + "wal_bytes": 2776912, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 14, + "cpu_percent": 0.0, + "db_bytes": 872448, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-restart", + "open_fd_count": 14, + "pending_requests": 0, + "request_statuses": { + "completed": 4, + "error": 6, + "interrupted": 1 + }, + "requests": 11, + "reservations": 14, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21860352, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 50, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103723008, + "wal_bytes": 3114752, + "wire_flights": null + } + ], + "schema_version": "runtime-q009.v1", + "seed": 9009, + "status": "pass", + "temporary_leftovers": 0, + "workload_policy": { + "budget_mode": "hard user and global request ceilings", + "initial_global_request_limit": 100, + "initial_user_request_limit": 100, + "principal_count": 2 + } +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/matrix.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/matrix.json new file mode 100644 index 00000000..c58ce980 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/matrix.json @@ -0,0 +1,174 @@ +{ + "schema_version": "on-prem-ordinary-server-matrix.v1", + "plan": "on-prem-concurrency-performance M002", + "status": "complete", + "preflight_io": { + "root_device_read_bytes": 734360875008, + "root_device_write_bytes": 1305547351040, + "io_pressure": { + "some": { + "avg10": 0.89, + "avg60": 0.48, + "avg300": 1.62 + }, + "full": { + "avg10": 0.89, + "avg60": 0.47, + "avg300": 1.58 + } + } + }, + "stop_reason": null, + "candidate": { + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "build_profile": "release", + "build_mode": "locked release candidate" + }, + "qualification_tooling_commit_sha": "f7343eb295ef3bb9e72b43211980934fe2ca2b45", + "target": { + "target_class": "ordinary-server", + "architecture": "x86_64", + "kernel": "6.8.0-147-generic", + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "logical_cpu_count": 16, + "ram_bytes": 16622407680, + "system_vendor": "Gigabyte Technology Co., Ltd.", + "product_name": "Z390 AORUS MASTER", + "board_name": "Z390 AORUS MASTER-CF", + "root_filesystem": "ext4", + "root_storage_medium_class": "non-rotational", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_mount_options": "rw,relatime,stripe=32", + "virtualization_type": "none", + "container_type": "none", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt" + }, + "workload": { + "fresh_root_per_run": true, + "named_principal_counts": [ + 1, + 5, + 10, + 25, + 49 + ], + "principal_separation_condition": "concurrency=1", + "concurrent_request_counts": [ + 1, + 2, + 4, + 8, + 16 + ], + "concurrency_separation_condition": "principals=5", + "requests_per_condition": 1000, + "repetitions_per_condition": 3, + "sequential_wire_samples_per_surface": 30, + "provider_latency_profile": "mixed-0-10-50ms", + "budget_mode": "user-global hard request limits", + "surfaces": [ + "native Responses finite", + "native Responses streaming", + "translated Responses-to-Messages streaming" + ] + }, + "gates": { + "all_requests_complete": true, + "zero_http_failures": true, + "zero_pending_requests_after_stabilization": true, + "zero_active_reservations_after_stabilization": true, + "p99_minimum_completed_observations": 1000, + "three_repetitions_required": true, + "maximum_initial_io_pressure_full_avg10_percent": 25.0, + "no_capacity_sla_claim": true + }, + "conditions": [ + { + "principals": 1, + "concurrency": 1, + "run_artifacts": [ + "u1-c1-run-1.json", + "u1-c1-run-2.json", + "u1-c1-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 1, + "run_artifacts": [ + "u5-c1-run-1.json", + "u5-c1-run-2.json", + "u5-c1-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 2, + "run_artifacts": [ + "u5-c2-run-1.json", + "u5-c2-run-2.json", + "u5-c2-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 4, + "run_artifacts": [ + "u5-c4-run-1.json", + "u5-c4-run-2.json", + "u5-c4-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 8, + "run_artifacts": [ + "u5-c8-run-1.json", + "u5-c8-run-2.json", + "u5-c8-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 16, + "run_artifacts": [ + "u5-c16-run-1.json", + "u5-c16-run-2.json", + "u5-c16-run-3.json" + ] + }, + { + "principals": 10, + "concurrency": 1, + "run_artifacts": [ + "u10-c1-run-1.json", + "u10-c1-run-2.json", + "u10-c1-run-3.json" + ] + }, + { + "principals": 25, + "concurrency": 1, + "run_artifacts": [ + "u25-c1-run-1.json", + "u25-c1-run-2.json", + "u25-c1-run-3.json" + ] + }, + { + "principals": 49, + "concurrency": 1, + "run_artifacts": [ + "u49-c1-run-1.json", + "u49-c1-run-2.json", + "u49-c1-run-3.json" + ] + } + ], + "artifacts_directory": "ordinary-server", + "lifecycle_artifact": "lifecycle.json" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-failed-shared-key-attempt-1.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-failed-shared-key-attempt-1.json new file mode 100644 index 00000000..987d933a --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-failed-shared-key-attempt-1.json @@ -0,0 +1,7 @@ +{ + "manifest": "runtime-q001.v1", + "plan": "SBC qualification", + "reason": "diagnostic warm-up returned HTTP 401 without a body", + "schema_version": "runtime-q008.v2", + "status": "fail" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-managed-runtime-key.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-managed-runtime-key.json new file mode 100644 index 00000000..10d149de --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-managed-runtime-key.json @@ -0,0 +1,7 @@ +{ + "manifest": "runtime-q001.v1", + "plan": "SBC qualification", + "reason": "database task quiescence was not reached", + "schema_version": "runtime-q008.v2", + "status": "fail" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-managed.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-managed.json new file mode 100644 index 00000000..10d149de --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-managed.json @@ -0,0 +1,7 @@ +{ + "manifest": "runtime-q001.v1", + "plan": "SBC qualification", + "reason": "database task quiescence was not reached", + "schema_version": "runtime-q008.v2", + "status": "fail" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-session.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-session.json new file mode 100644 index 00000000..f4c123ab --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-session.json @@ -0,0 +1,1350 @@ +{ + "backup": { + "archive_sha256": "1aef1132fe54c7644dde27b3eb1cd79c47ef85269882d9e0845fdc35c84749b6", + "archive_size_bytes": 986583, + "elapsed_ms": 109, + "member_count": 3 + }, + "benchmark": { + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "collector": { + "baseline_record_seq": 51, + "capacity": 256, + "final_record_seq": 172, + "records_after_baseline": 121, + "schema_version": "sqlite-db-phase.v1" + }, + "config_fixture": "sbc-benchmark.toml", + "diagnostic_mode": "publication_commit_checkpoint_phase", + "effective": { + "journal_mode": "wal", + "page_size": 4096, + "synchronous": "NORMAL", + "wal_autocheckpoint_pages": 1000 + }, + "interpretation": "qualification-only transaction phase correlation; no production SQLite policy or performance threshold applied", + "resource_summary": { + "final_active_reservations": 0, + "final_finalization_jobs": 0, + "final_peak_rss_bytes": 44654592, + "final_pending_requests": 0, + "final_rss_bytes": 44654592 + }, + "runs": { + "direct_provider_control": { + "completed_count": 30, + "failed_count": 0, + "fixture_path_suffix": "/responses", + "interpretation": "fixture/host control only; do not compare absolute latency to EggPool as a benchmark score", + "maximum_client_body_ms": 0, + "maximum_post_provider_ttft_ms": 0, + "maximum_pre_provider_ms": 0, + "maximum_provider_service_ms": 0, + "maximum_total_ms": 1, + "p50_client_body_ms": 0, + "p50_post_provider_ttft_ms": 0, + "p50_pre_provider_ms": 0, + "p50_provider_service_ms": 0, + "p50_total_ms": 0, + "p95_client_body_ms": 0, + "p95_post_provider_ttft_ms": 0, + "p95_pre_provider_ms": 0, + "p95_provider_service_ms": 0, + "p95_total_ms": 1, + "request_target": "fixture-provider-direct", + "sample_count": 30, + "slowest_five": [ + { + "client_body_ms": 0, + "http_status": 200, + "last_phase": "completed", + "post_provider_ttft_ms": 0, + "pre_provider_ms": 0, + "provider_service_ms": 0, + "sequence": 11, + "timed_out": false, + "total_ms": 1 + }, + { + "client_body_ms": 0, + "http_status": 200, + "last_phase": "completed", + "post_provider_ttft_ms": 0, + "pre_provider_ms": 0, + "provider_service_ms": 0, + "sequence": 15, + "timed_out": false, + "total_ms": 1 + }, + { + "client_body_ms": 0, + "http_status": 200, + "last_phase": "completed", + "post_provider_ttft_ms": 0, + "pre_provider_ms": 0, + "provider_service_ms": 0, + "sequence": 16, + "timed_out": false, + "total_ms": 1 + }, + { + "client_body_ms": 0, + "http_status": 200, + "last_phase": "completed", + "post_provider_ttft_ms": 0, + "pre_provider_ms": 0, + "provider_service_ms": 0, + "sequence": 17, + "timed_out": false, + "total_ms": 1 + }, + { + "client_body_ms": 0, + "http_status": 200, + "last_phase": "completed", + "post_provider_ttft_ms": 0, + "pre_provider_ms": 0, + "provider_service_ms": 0, + "sequence": 25, + "timed_out": false, + "total_ms": 1 + } + ], + "status": "measured", + "timeout_count": 0, + "warmup_count": 5 + }, + "publication_phase_diagnostic": { + "batch_elapsed_ms": 218, + "client_surface": "responses", + "completed_count": 60, + "cpu": { + "cpu_percent": 50.57, + "cpu_ticks": 11, + "cpu_time_ms": 110 + }, + "failed_count": 0, + "finalization_phase_summary": { + "maximum_begin_us": 43, + "maximum_body_us": 1569, + "maximum_commit_us": 11106, + "maximum_gate_wait_us": 1, + "maximum_total_us": 11455, + "maximum_worker_queue_us": 101, + "maximum_worker_return_us": 91, + "p50_begin_us": 7, + "p50_body_us": 262, + "p50_commit_us": 48, + "p50_gate_wait_us": 0, + "p50_total_us": 384, + "p50_worker_queue_us": 33, + "p50_worker_return_us": 31, + "p95_begin_us": 42, + "p95_body_us": 1544, + "p95_commit_us": 3871, + "p95_gate_wait_us": 1, + "p95_total_us": 4221, + "p95_worker_queue_us": 73, + "p95_worker_return_us": 85, + "sample_count": 60 + }, + "foreground_record_count": 120, + "interpretation": "bounded provider-boundary phase attribution; T0 client start, T1 provider receive, T2 provider finish, T3 first byte, T4 done", + "maximum_client_body_ms": 0, + "maximum_post_provider_ttft_ms": 12, + "maximum_pre_provider_ms": 28, + "maximum_provider_service_ms": 0, + "maximum_total_ms": 33, + "maximum_total_ms_with_checkpoint_sequence_change": 33, + "maximum_total_ms_without_checkpoint_sequence_change": 13, + "model": "q008-responses", + "p50_client_body_ms": 0, + "p50_post_provider_ttft_ms": 1, + "p50_pre_provider_ms": 1, + "p50_provider_service_ms": 0, + "p50_total_ms": 2, + "p95_client_body_ms": 0, + "p95_post_provider_ttft_ms": 4, + "p95_pre_provider_ms": 6, + "p95_provider_service_ms": 0, + "p95_total_ms": 13, + "publication_phase_summary": { + "maximum_begin_us": 34, + "maximum_body_us": 1350, + "maximum_commit_us": 24448, + "maximum_gate_wait_us": 1, + "maximum_total_us": 25765, + "maximum_worker_queue_us": 114, + "maximum_worker_return_us": 103, + "p50_begin_us": 5, + "p50_body_us": 202, + "p50_commit_us": 115, + "p50_gate_wait_us": 0, + "p50_total_us": 388, + "p50_worker_queue_us": 33, + "p50_worker_return_us": 30, + "p95_begin_us": 34, + "p95_body_us": 1171, + "p95_commit_us": 3786, + "p95_gate_wait_us": 1, + "p95_total_us": 4067, + "p95_worker_queue_us": 72, + "p95_worker_return_us": 89, + "sample_count": 60 + }, + "sample_count": 60, + "sequential": true, + "slowest_five": [ + { + "client_body_ms": 0, + "http_status": 200, + "last_phase": "completed", + "post_provider_ttft_ms": 5, + "pre_provider_ms": 28, + "provider_service_ms": 0, + "sequence": 5, + "timed_out": false, + "total_ms": 33, + "wal_bytes_after": 4185952, + "wal_bytes_before": 4004672, + "wal_bytes_delta": 181280, + "wal_checkpoint_sequence_after": 1, + "wal_checkpoint_sequence_before": 0, + "wal_checkpoint_sequence_changed": true + }, + { + "client_body_ms": 0, + "http_status": 200, + "last_phase": "completed", + "post_provider_ttft_ms": 4, + "pre_provider_ms": 13, + "provider_service_ms": 0, + "sequence": 38, + "timed_out": false, + "total_ms": 17, + "wal_bytes_after": 4301312, + "wal_bytes_before": 4185952, + "wal_bytes_delta": 115360, + "wal_checkpoint_sequence_after": 3, + "wal_checkpoint_sequence_before": 2, + "wal_checkpoint_sequence_changed": true + }, + { + "client_body_ms": 0, + "http_status": 200, + "last_phase": "completed", + "post_provider_ttft_ms": 4, + "pre_provider_ms": 12, + "provider_service_ms": 0, + "sequence": 54, + "timed_out": false, + "total_ms": 17, + "wal_bytes_after": 4301312, + "wal_bytes_before": 4301312, + "wal_bytes_delta": 0, + "wal_checkpoint_sequence_after": 4, + "wal_checkpoint_sequence_before": 3, + "wal_checkpoint_sequence_changed": true + }, + { + "client_body_ms": 0, + "http_status": 200, + "last_phase": "completed", + "post_provider_ttft_ms": 12, + "pre_provider_ms": 1, + "provider_service_ms": 0, + "sequence": 21, + "timed_out": false, + "total_ms": 13, + "wal_bytes_after": 4185952, + "wal_bytes_before": 4185952, + "wal_bytes_delta": 0, + "wal_checkpoint_sequence_after": 1, + "wal_checkpoint_sequence_before": 1, + "wal_checkpoint_sequence_changed": false + }, + { + "client_body_ms": 0, + "http_status": 200, + "last_phase": "completed", + "post_provider_ttft_ms": 3, + "pre_provider_ms": 6, + "provider_service_ms": 0, + "sequence": 1, + "timed_out": false, + "total_ms": 10, + "wal_bytes_after": 3242472, + "wal_bytes_before": 2978792, + "wal_bytes_delta": 263680, + "wal_checkpoint_sequence_after": 0, + "wal_checkpoint_sequence_before": 0, + "wal_checkpoint_sequence_changed": false + } + ], + "slowest_five_correlated": [ + { + "finalization": { + "begin_us": 9, + "body_us": 296, + "commit_us": 3994, + "gate_wait_us": 0, + "kind": "finalization", + "record_seq": 61, + "success": true, + "total_us": 4371, + "worker_queue_us": 35, + "worker_return_us": 35 + }, + "post_provider_ttft_ms": 5, + "post_provider_unattributed_us": 629, + "pre_provider_ms": 28, + "pre_provider_unattributed_us": 2235, + "provider_service_ms": 0, + "publication": { + "begin_us": 34, + "body_us": 1172, + "commit_us": 24448, + "gate_wait_us": 1, + "kind": "publication", + "record_seq": 60, + "success": true, + "total_us": 25765, + "worker_queue_us": 72, + "worker_return_us": 34 + }, + "request_sequence": 5, + "total_ms": 33, + "wal_checkpoint_sequence_changed": true + }, + { + "finalization": { + "begin_us": 6, + "body_us": 274, + "commit_us": 3878, + "gate_wait_us": 0, + "kind": "finalization", + "record_seq": 127, + "success": true, + "total_us": 4252, + "worker_queue_us": 56, + "worker_return_us": 35 + }, + "post_provider_ttft_ms": 4, + "post_provider_unattributed_us": 0, + "pre_provider_ms": 13, + "pre_provider_unattributed_us": 901, + "provider_service_ms": 0, + "publication": { + "begin_us": 5, + "body_us": 202, + "commit_us": 11825, + "gate_wait_us": 0, + "kind": "publication", + "record_seq": 126, + "success": true, + "total_us": 12099, + "worker_queue_us": 32, + "worker_return_us": 32 + }, + "request_sequence": 38, + "total_ms": 17, + "wal_checkpoint_sequence_changed": true + }, + { + "finalization": { + "begin_us": 7, + "body_us": 277, + "commit_us": 3871, + "gate_wait_us": 0, + "kind": "finalization", + "record_seq": 159, + "success": true, + "total_us": 4221, + "worker_queue_us": 33, + "worker_return_us": 31 + }, + "post_provider_ttft_ms": 4, + "post_provider_unattributed_us": 0, + "pre_provider_ms": 12, + "pre_provider_unattributed_us": 437, + "provider_service_ms": 0, + "publication": { + "begin_us": 6, + "body_us": 201, + "commit_us": 11271, + "gate_wait_us": 0, + "kind": "publication", + "record_seq": 158, + "success": true, + "total_us": 11563, + "worker_queue_us": 33, + "worker_return_us": 50 + }, + "request_sequence": 54, + "total_ms": 17, + "wal_checkpoint_sequence_changed": true + }, + { + "finalization": { + "begin_us": 7, + "body_us": 263, + "commit_us": 11106, + "gate_wait_us": 0, + "kind": "finalization", + "record_seq": 93, + "success": true, + "total_us": 11455, + "worker_queue_us": 34, + "worker_return_us": 42 + }, + "post_provider_ttft_ms": 12, + "post_provider_unattributed_us": 545, + "pre_provider_ms": 1, + "pre_provider_unattributed_us": 626, + "provider_service_ms": 0, + "publication": { + "begin_us": 5, + "body_us": 193, + "commit_us": 110, + "gate_wait_us": 0, + "kind": "publication", + "record_seq": 92, + "success": true, + "total_us": 374, + "worker_queue_us": 34, + "worker_return_us": 29 + }, + "request_sequence": 21, + "total_ms": 13, + "wal_checkpoint_sequence_changed": false + }, + { + "finalization": { + "begin_us": 43, + "body_us": 1569, + "commit_us": 371, + "gate_wait_us": 1, + "kind": "finalization", + "record_seq": 53, + "success": true, + "total_us": 2151, + "worker_queue_us": 73, + "worker_return_us": 91 + }, + "post_provider_ttft_ms": 3, + "post_provider_unattributed_us": 849, + "pre_provider_ms": 6, + "pre_provider_unattributed_us": 3479, + "provider_service_ms": 0, + "publication": { + "begin_us": 34, + "body_us": 1350, + "commit_us": 929, + "gate_wait_us": 1, + "kind": "publication", + "record_seq": 52, + "success": true, + "total_us": 2521, + "worker_queue_us": 114, + "worker_return_us": 90 + }, + "request_sequence": 1, + "total_ms": 10, + "wal_checkpoint_sequence_changed": false + } + ], + "status": "measured", + "streaming": false, + "timeout_count": 0, + "wal_checkpoint_sequence_change_count": 4, + "wal_checkpoint_sequence_unchanged_count": 56, + "warmup_count": 5 + } + }, + "sample_count": 60, + "task_quiescence": { + "background_db_activity": false, + "baseline": { + "catalog_refresh": { + "in_tick": false, + "tick_count": 0 + }, + "checkpoint": { + "in_tick": false, + "tick_count": 1 + }, + "metrics_flush": { + "in_tick": false, + "tick_count": 1 + }, + "retention_cleanup": { + "in_tick": false, + "tick_count": 0 + } + }, + "deltas": { + "catalog_refresh": { + "baseline_tick_count": 0, + "final_in_tick": false, + "final_tick_count": 0, + "tick_count_delta": 0 + }, + "checkpoint": { + "baseline_tick_count": 1, + "final_in_tick": false, + "final_tick_count": 1, + "tick_count_delta": 0 + }, + "metrics_flush": { + "baseline_tick_count": 1, + "final_in_tick": false, + "final_tick_count": 1, + "tick_count_delta": 0 + }, + "retention_cleanup": { + "baseline_tick_count": 0, + "final_in_tick": false, + "final_tick_count": 0, + "tick_count_delta": 0 + } + }, + "final": { + "catalog_refresh": { + "in_tick": false, + "tick_count": 0 + }, + "checkpoint": { + "in_tick": false, + "tick_count": 1 + }, + "metrics_flush": { + "in_tick": false, + "tick_count": 1 + }, + "retention_cleanup": { + "in_tick": false, + "tick_count": 0 + } + }, + "fixed_task_names": [ + "checkpoint", + "metrics_flush", + "catalog_refresh", + "retention_cleanup", + "automatic_backup" + ], + "wait_timeout_s": 15.0 + }, + "wal_autocheckpoint_override_pages": null + }, + "candidate": { + "binary_size_bytes": 39157144, + "build_mode": "q005-qualified-aarch64-copy", + "build_profile": "release", + "repository_commit_sha": "63268e1e3d1efe8a524f679d16ef259c91a880ae", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "11670f0c0880335020cbb41574d0a368aaebe3bea5329751b4fe321ea1cb3726" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-zqhtau87/config.toml Server: 127.0.0.1:50597 Accounts: 1 Database: Content digest: 354d0d0204c01d6dd38deba05b44ae03365d20a23b844f94afb6c318d21e2abc", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 89, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 38, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 2, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 5, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 5, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2454487, \"ppid\": 2454464, \"uptime_seconds\": 2.887225586, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 109, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-zqhtau87/backups/eggpool-backup-20261016-013535.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2454487)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 47, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-zqhtau87/recovery/config.toml db: /tmp/eggpool-q008-zqhtau87/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 48, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2454665)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 73, + "pending_requests": 0, + "request_statuses": { + "completed": 73 + }, + "requests": 73, + "reservations": 73 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 2, + "/responses": 103 + }, + "loopback_provider_requests": 108, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.0, + "avg300": 4.3, + "avg60": 0.12 + }, + "some": { + "avg10": 0.0, + "avg300": 4.39, + "avg60": 0.12 + } + }, + "root_device_read_bytes": 732401647616, + "root_device_write_bytes": 1303667213312 + }, + "product_name": "Z390 AORUS MASTER", + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 3, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 3 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 1 + }, + { + "body_bytes": 270, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 1 + }, + { + "body_bytes": 480, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 1 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 42160128, + 42160128, + 44199936, + 44265472, + 44646400, + 44654592, + 44654592, + 23171072 + ], + "samples": 8, + "status": "pass", + "thread_counts": [ + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 3, + 2, + 2 + ], + "finite_ttft_ms": [ + 3, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 6, + 5 + ], + "second_workload_ttft_ms": [ + 5, + 5 + ], + "streaming_elapsed_ms": [ + 2, + 2, + 2 + ], + "streaming_ttft_ms": [ + 1, + 1, + 1 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 42160128, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 42160128, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172937216, + "wal_bytes": 123632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 42160128, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 42160128, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172937216, + "wal_bytes": 123632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 44199936, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 44199936, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173305856, + "wal_bytes": 1693352, + "wire_flights": null + }, + { + "active_generation_id": 1, + "active_leases": 0, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": 256, + "finalization_jobs": 0, + "generation_count": 1, + "label": "publication-phases-before-quiescence", + "open_fd_count": 14, + "peak_rss_bytes": 44265472, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": 0, + "retiring_leases": 0, + "rss_bytes": 44265472, + "running_tasks": 4, + "runtime_read_error": false, + "runtime_tasks": 4, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173305856, + "wal_bytes": 1693352, + "wire_flights": null + }, + { + "active_generation_id": 1, + "active_leases": 0, + "active_reservations": 0, + "attempts": 71, + "cpu_percent": 0.0, + "db_bytes": 962560, + "finalization_capacity": 256, + "finalization_jobs": 0, + "generation_count": 1, + "label": "after-publication-phases-diagnostic", + "open_fd_count": 14, + "peak_rss_bytes": 44646400, + "pending_requests": 0, + "request_statuses": { + "completed": 71 + }, + "requests": 71, + "reservations": 71, + "retiring_generations": 0, + "retiring_leases": 0, + "rss_bytes": 44646400, + "running_tasks": 4, + "runtime_read_error": false, + "runtime_tasks": 4, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173441024, + "wal_bytes": 4301312, + "wire_flights": null + }, + { + "active_generation_id": 1, + "active_leases": 0, + "active_reservations": 0, + "attempts": 71, + "cpu_percent": 0.0, + "db_bytes": 962560, + "finalization_capacity": 256, + "finalization_jobs": 0, + "generation_count": 1, + "label": "after-publication-phases-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 44654592, + "pending_requests": 0, + "request_statuses": { + "completed": 71 + }, + "requests": 71, + "reservations": 71, + "retiring_generations": 0, + "retiring_leases": 0, + "rss_bytes": 44654592, + "running_tasks": 4, + "runtime_read_error": false, + "runtime_tasks": 4, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173441024, + "wal_bytes": 4301312, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 71, + "cpu_percent": 0.0, + "db_bytes": 962560, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 44654592, + "pending_requests": 0, + "request_statuses": { + "completed": 71 + }, + "requests": 71, + "reservations": 71, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 44654592, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173441024, + "wal_bytes": 4301312, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 73, + "cpu_percent": 0.0, + "db_bytes": 970752, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23171072, + "pending_requests": 0, + "request_statuses": { + "completed": 73 + }, + "requests": 73, + "reservations": 73, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23171072, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103747584, + "wal_bytes": 585072, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic.json new file mode 100644 index 00000000..987d933a --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic.json @@ -0,0 +1,7 @@ +{ + "manifest": "runtime-q001.v1", + "plan": "SBC qualification", + "reason": "diagnostic warm-up returned HTTP 401 without a body", + "schema_version": "runtime-q008.v2", + "status": "fail" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/summary.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/summary.json new file mode 100644 index 00000000..b8bb830a --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/summary.json @@ -0,0 +1,522 @@ +{ + "schema_version": "on-prem-ordinary-server-summary.v1", + "plan": "on-prem-concurrency-performance M002", + "candidate": { + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "build_profile": "release", + "build_mode": "locked release candidate" + }, + "target": { + "target_class": "ordinary-server", + "architecture": "x86_64", + "kernel": "6.8.0-147-generic", + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "logical_cpu_count": 16, + "ram_bytes": 16622407680, + "system_vendor": "Gigabyte Technology Co., Ltd.", + "product_name": "Z390 AORUS MASTER", + "board_name": "Z390 AORUS MASTER-CF", + "root_filesystem": "ext4", + "root_storage_medium_class": "non-rotational", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_mount_options": "rw,relatime,stripe=32", + "virtualization_type": "none", + "container_type": "none", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt" + }, + "workload": { + "fresh_root_per_run": true, + "named_principal_counts": [ + 1, + 5, + 10, + 25, + 49 + ], + "principal_separation_condition": "concurrency=1", + "concurrent_request_counts": [ + 1, + 2, + 4, + 8, + 16 + ], + "concurrency_separation_condition": "principals=5", + "requests_per_condition": 1000, + "repetitions_per_condition": 3, + "sequential_wire_samples_per_surface": 30, + "provider_latency_profile": "mixed-0-10-50ms", + "budget_mode": "user-global hard request limits", + "surfaces": [ + "native Responses finite", + "native Responses streaming", + "translated Responses-to-Messages streaming" + ] + }, + "lifecycle": { + "artifact": "lifecycle.json", + "status": "pass", + "phases": [ + "faults", + "restart" + ], + "provider": { + "request_count": 13, + "requests_by_model": { + "q009-abrupt": 1, + "q009-absent": 1, + "q009-disconnect": 1, + "q009-fault-408": 2, + "q009-fault-429": 1, + "q009-fault-5xx": 2, + "q009-malformed": 1, + "q009-partial": 1, + "q009-timeout": 1, + "q009-wire": 2 + } + }, + "ownership_convergence": { + "active_leases": null, + "active_reservations": 0, + "finalization_jobs": null, + "logical_leaks": false, + "pending_requests": 0, + "retiring_generations": null, + "terminal_references": 0 + }, + "workload_policy": { + "budget_mode": "hard user and global request ceilings", + "initial_global_request_limit": 100, + "initial_user_request_limit": 100, + "principal_count": 2 + }, + "denial_counts": { + "provider_429": 1, + "user_budget_429": 1, + "global_budget_429": 1, + "identity_denial": 1 + }, + "identity_denial_http_status": 401 + }, + "interpretation": "Descriptive loopback capacity characterization for this host and synthetic workload; not an SLA or provider-throughput promise.", + "conditions": [ + { + "principals": 1, + "concurrency": 1, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 36.495, + "median_cpu_percent": 28.9, + "median_p50_ms": 15, + "median_p95_ms": 82, + "median_p99_ms": 148, + "maximum_ms": 232, + "maximum_peak_rss_bytes": 26152960, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 15, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 15, + "p95_ttft_ms": 59 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 10, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 9, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 13, + "p95_elapsed_ms": 60, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 4.3, + "median_root_device_write_delta_bytes": 391094272, + "maximum_observed_wal_bytes": 4408432, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u1-c1-run-1.json", + "u1-c1-run-2.json", + "u1-c1-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 1, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 42.179, + "median_cpu_percent": 33.03, + "median_p50_ms": 15, + "median_p95_ms": 61, + "median_p99_ms": 89, + "maximum_ms": 243, + "maximum_peak_rss_bytes": 46403584, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 12, + "p95_ttft_ms": 59 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 15, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 1.68, + "median_root_device_write_delta_bytes": 389419008, + "maximum_observed_wal_bytes": 4424912, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u5-c1-run-1.json", + "u5-c1-run-2.json", + "u5-c1-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 2, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 93.976, + "median_cpu_percent": 54.79, + "median_p50_ms": 14, + "median_p95_ms": 61, + "median_p99_ms": 79, + "maximum_ms": 96, + "maximum_peak_rss_bytes": 46477312, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 13, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 13, + "p95_ttft_ms": 59 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 13, + "p95_elapsed_ms": 60, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 1.33, + "median_root_device_write_delta_bytes": 385441792, + "maximum_observed_wal_bytes": 4461992, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u5-c2-run-1.json", + "u5-c2-run-2.json", + "u5-c2-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 4, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 193.683, + "median_cpu_percent": 65.27, + "median_p50_ms": 14, + "median_p95_ms": 59, + "median_p99_ms": 75, + "maximum_ms": 100, + "maximum_peak_rss_bytes": 46784512, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 13, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 13, + "p95_ttft_ms": 59 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 13, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 19, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 1.79, + "median_root_device_write_delta_bytes": 386105344, + "maximum_observed_wal_bytes": 4466112, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u5-c4-run-1.json", + "u5-c4-run-2.json", + "u5-c4-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 8, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 335.268, + "median_cpu_percent": 66.56, + "median_p50_ms": 19, + "median_p95_ms": 69, + "median_p99_ms": 76, + "maximum_ms": 1088, + "maximum_peak_rss_bytes": 47173632, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 19, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 19, + "p95_ttft_ms": 59 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 13, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 5.7, + "median_root_device_write_delta_bytes": 390234112, + "maximum_observed_wal_bytes": 4387832, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u5-c8-run-1.json", + "u5-c8-run-2.json", + "u5-c8-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 16, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 341.347, + "median_cpu_percent": 71.09, + "median_p50_ms": 35, + "median_p95_ms": 88, + "median_p99_ms": 106, + "maximum_ms": 1119, + "maximum_peak_rss_bytes": 48001024, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 15, + "p95_elapsed_ms": 83, + "p50_ttft_ms": 15, + "p95_ttft_ms": 83 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 112, + "p50_ttft_ms": 12, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 15, + "p95_elapsed_ms": 91, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 3.19, + "median_root_device_write_delta_bytes": 391352320, + "maximum_observed_wal_bytes": 4404312, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u5-c16-run-1.json", + "u5-c16-run-2.json", + "u5-c16-run-3.json" + ] + }, + { + "principals": 10, + "concurrency": 1, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 42.858, + "median_cpu_percent": 34.11, + "median_p50_ms": 14, + "median_p95_ms": 61, + "median_p99_ms": 77, + "maximum_ms": 160, + "maximum_peak_rss_bytes": 46424064, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 15, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 15, + "p95_ttft_ms": 59 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 13, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 0.57, + "median_root_device_write_delta_bytes": 387596288, + "maximum_observed_wal_bytes": 4437272, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u10-c1-run-1.json", + "u10-c1-run-2.json", + "u10-c1-run-3.json" + ] + }, + { + "principals": 25, + "concurrency": 1, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 42.495, + "median_cpu_percent": 34.82, + "median_p50_ms": 16, + "median_p95_ms": 61, + "median_p99_ms": 81, + "maximum_ms": 228, + "maximum_peak_rss_bytes": 46788608, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 12, + "p95_ttft_ms": 59 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 19, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 0.43, + "median_root_device_write_delta_bytes": 385429504, + "maximum_observed_wal_bytes": 4424912, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u25-c1-run-1.json", + "u25-c1-run-2.json", + "u25-c1-run-3.json" + ] + }, + { + "principals": 49, + "concurrency": 1, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 42.322, + "median_cpu_percent": 35.14, + "median_p50_ms": 15, + "median_p95_ms": 61, + "median_p99_ms": 81, + "maximum_ms": 195, + "maximum_peak_rss_bytes": 47308800, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 12, + "p95_ttft_ms": 59 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 13, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 19, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 0.52, + "median_root_device_write_delta_bytes": 383197184, + "maximum_observed_wal_bytes": 4424912, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u49-c1-run-1.json", + "u49-c1-run-2.json", + "u49-c1-run-3.json" + ] + } + ] +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u1-c1-run-1-failed-attempt-1.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u1-c1-run-1-failed-attempt-1.json new file mode 100644 index 00000000..66d24bc3 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u1-c1-run-1-failed-attempt-1.json @@ -0,0 +1,7 @@ +{ + "manifest": "runtime-q001.v1", + "plan": "SBC qualification", + "reason": "second workload chat_completions failed with HTTP 429 (budget-exceeded; user_rules=1,instance_rules=1)", + "schema_version": "runtime-q008.v2", + "status": "fail" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u1-c1-run-1.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u1-c1-run-1.json new file mode 100644 index 00000000..f6be42d3 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u1-c1-run-1.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "56a5a55cab0bd07ef58970898db30df44b991a9b40345df51d5dd1be2eb83179", + "archive_size_bytes": 4648407, + "elapsed_ms": 460, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 1, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23801856, + "baseline_rss_bytes": 23801856, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 26152960, + "peak_rss_bytes": 26152960 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 27637, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 28.62, + "cpu_ticks": 791, + "cpu_time_ms": 7910 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 4.3, + "avg300": 0.55, + "avg60": 1.99 + }, + "some": { + "avg10": 4.3, + "avg300": 0.55, + "avg60": 1.99 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.54, + "avg300": 0.09, + "avg60": 0.2 + }, + "some": { + "avg10": 0.54, + "avg300": 0.1, + "avg60": 0.2 + } + }, + "root_device_read_delta_bytes": 344064, + "root_device_write_delta_bytes": 391094272 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 36.184, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 188, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 96, + "p95_ttft_ms": 96, + "p99_elapsed_ms": 148, + "p99_ttft_ms": 148, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 147, + "minimum_elapsed_ms": 6, + "model": "q008-responses", + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 99, + "p95_ttft_ms": 99, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 175, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 14, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 100, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 131, + "minimum_elapsed_ms": 9, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 93, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-8s5olt8y/config.toml Server: 127.0.0.1:57187 Accounts: 1 Database: Content digest: 069b288a841bd912bad63dd93c336e08f780281e229e5ffa6f7aa537320bcfef", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 89, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 38, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2399016, \"ppid\": 2398994, \"uptime_seconds\": 2.897474637, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 8, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 460, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-8s5olt8y/backups/eggpool-backup-20261016-005854.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2399016)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 94, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-8s5olt8y/recovery/config.toml db: /tmp/eggpool-q008-8s5olt8y/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 47, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2400344)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.0, + "avg300": 0.08, + "avg60": 0.11 + }, + "some": { + "avg10": 0.0, + "avg300": 0.08, + "avg60": 0.11 + } + }, + "root_device_read_bytes": 723924996096, + "root_device_write_bytes": 1274574586880 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 11, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 11 + }, + { + "body_bytes": 309, + "elapsed_ms": 9, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 213, + "elapsed_ms": 10, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 145, + "elapsed_ms": 14, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 14 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 54, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 52 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21966848, + 21970944, + 23801856, + 23801856, + 23924736, + 24076288, + 24289280, + 26152960, + 26152960, + 26152960, + 23756800 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 11, + 9, + 10 + ], + "finite_ttft_ms": [ + 11, + 9, + 9 + ], + "second_workload_elapsed_ms": [ + 11, + 2 + ], + "second_workload_ttft_ms": [ + 11, + 2 + ], + "streaming_elapsed_ms": [ + 14, + 12, + 54 + ], + "streaming_ttft_ms": [ + 14, + 12, + 52 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21966848, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21966848, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21970944, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21970944, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23801856, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23801856, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172904448, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23801856, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23801856, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172904448, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23924736, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23924736, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172904448, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24076288, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24076288, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173039616, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24289280, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24289280, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173039616, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 26152960, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 26152960, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173162496, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 26152960, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 26152960, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173162496, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 26152960, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 26152960, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173162496, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23756800, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23756800, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103591936, + "wal_bytes": 885832, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 102, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u1-c1-run-2.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u1-c1-run-2.json new file mode 100644 index 00000000..38c87213 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u1-c1-run-2.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "f6fb44d9b19f67c0dd4e7ab5effcbf494cc04d7d34a2c027e13b86bbe3b629fc", + "archive_size_bytes": 4648407, + "elapsed_ms": 481, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 1, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23773184, + "baseline_rss_bytes": 23773184, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 26099712, + "peak_rss_bytes": 26099712 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 26349, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 30.67, + "cpu_ticks": 808, + "cpu_time_ms": 8080 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 2.51, + "avg300": 0.78, + "avg60": 2.2 + }, + "some": { + "avg10": 2.55, + "avg300": 0.78, + "avg60": 2.23 + } + }, + "io_pressure_before": { + "full": { + "avg10": 1.39, + "avg300": 0.54, + "avg60": 1.66 + }, + "some": { + "avg10": 1.39, + "avg300": 0.54, + "avg60": 1.66 + } + }, + "root_device_read_delta_bytes": 1249280, + "root_device_write_delta_bytes": 387997696 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 37.953, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 219, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 65, + "p95_ttft_ms": 65, + "p99_elapsed_ms": 148, + "p99_ttft_ms": 147, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 19, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 64, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 9, + "p50_ttft_ms": 7, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 9, + "model": "q008-messages", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-ih4fxwvt/config.toml Server: 127.0.0.1:42165 Accounts: 1 Database: Content digest: c06197cdfcfe5cf8c713f5b52df3daa2a74e9af34290e3c12166dfcc71338844", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 93, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 5, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2400373, \"ppid\": 2400351, \"uptime_seconds\": 2.9196932479999997, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\"", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 481, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-ih4fxwvt/backups/eggpool-backup-20261016-005932.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 56, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2400373)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 212, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-ih4fxwvt/recovery/config.toml db: /tmp/eggpool-q008-ih4fxwvt/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 115, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2401671)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 2.88, + "avg300": 0.54, + "avg60": 1.86 + }, + "some": { + "avg10": 2.88, + "avg300": 0.55, + "avg60": 1.86 + } + }, + "root_device_read_bytes": 723926884352, + "root_device_write_bytes": 1275040637952 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 11, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 11 + }, + { + "body_bytes": 309, + "elapsed_ms": 9, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 213, + "elapsed_ms": 9, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 145, + "elapsed_ms": 20, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 270, + "elapsed_ms": 19, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 480, + "elapsed_ms": 60, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21872640, + 21876736, + 23773184, + 23773184, + 23891968, + 24014848, + 24264704, + 26099712, + 26099712, + 26099712, + 23719936 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 11, + 9, + 9 + ], + "finite_ttft_ms": [ + 11, + 9, + 9 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 20, + 19, + 60 + ], + "streaming_ttft_ms": [ + 17, + 17, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21872640, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21872640, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103559168, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21876736, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21876736, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103559168, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23773184, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23773184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23773184, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23773184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23891968, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23891968, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24014848, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24014848, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24264704, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24264704, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173047808, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 26099712, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 26099712, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173170688, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 26099712, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 26099712, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173170688, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 26099712, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 26099712, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173170688, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23719936, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23719936, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103596032, + "wal_bytes": 885832, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u1-c1-run-3.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u1-c1-run-3.json new file mode 100644 index 00000000..9dbff4d1 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u1-c1-run-3.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "ad46742b53a9c30ffe0920dbb51cf572195a4762cf291d5b8d7ea23c25148701", + "archive_size_bytes": 4648407, + "elapsed_ms": 608, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 1, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23703552, + "baseline_rss_bytes": 23703552, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 25960448, + "peak_rss_bytes": 25960448 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 27401, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 28.9, + "cpu_ticks": 792, + "cpu_time_ms": 7920 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 5.44, + "avg300": 1.08, + "avg60": 2.83 + }, + "some": { + "avg10": 5.59, + "avg300": 1.09, + "avg60": 2.88 + } + }, + "io_pressure_before": { + "full": { + "avg10": 1.08, + "avg300": 0.77, + "avg60": 1.92 + }, + "some": { + "avg10": 1.1, + "avg300": 0.78, + "avg60": 1.94 + } + }, + "root_device_read_delta_bytes": 823296, + "root_device_write_delta_bytes": 395808768 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 36.495, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 232, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 82, + "p95_ttft_ms": 82, + "p99_elapsed_ms": 142, + "p99_ttft_ms": 142, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 3, + "model": "q008-responses", + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 58, + "p95_ttft_ms": 58, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 83, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 10, + "p50_ttft_ms": 9, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 3, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-jwv4g20s/config.toml Server: 127.0.0.1:49029 Accounts: 1 Database: Content digest: d3674c51a89e91dc91e2cada2b384d3445e4f8d5a03ffb17179c3602e8e5b7d5", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 99, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2401721, \"ppid\": 2401678, \"uptime_seconds\": 2.871493338, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 7, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 608, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-jwv4g20s/backups/eggpool-backup-20261016-010011.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 58, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2401721)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 567, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-jwv4g20s/recovery/config.toml db: /tmp/eggpool-q008-jwv4g20s/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 266, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2403052)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 1.97, + "avg300": 0.78, + "avg60": 2.12 + }, + "some": { + "avg10": 2.0, + "avg300": 0.79, + "avg60": 2.15 + } + }, + "root_device_read_bytes": 723928264704, + "root_device_write_bytes": 1275503830016 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 55, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21741568, + 21745664, + 23703552, + 23703552, + 23830528, + 23953408, + 24133632, + 25960448, + 25960448, + 26025984, + 23728128 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 3, + 2 + ], + "second_workload_ttft_ms": [ + 3, + 2 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 55 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21741568, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21741568, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21745664, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21745664, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23703552, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23703552, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172916736, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23703552, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23703552, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172916736, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23830528, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23830528, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172916736, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23953408, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23953408, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172916736, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24133632, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24133632, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173051904, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 25960448, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 25960448, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173174784, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 25960448, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 25960448, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173174784, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 26025984, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 26025984, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173174784, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23728128, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23728128, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103596032, + "wal_bytes": 885832, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u10-c1-run-1.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u10-c1-run-1.json new file mode 100644 index 00000000..15b5466e --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u10-c1-run-1.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "b84e28d8c0d812dd429d891ddbccaaaaaf03fc1d18670685aff7afcb85da9dc0", + "archive_size_bytes": 4599255, + "elapsed_ms": 439, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 10, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23781376, + "baseline_rss_bytes": 23781376, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46424064, + "peak_rss_bytes": 46424064 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 24771, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 31.69, + "cpu_ticks": 785, + "cpu_time_ms": 7850 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 2.58, + "avg300": 2.13, + "avg60": 2.66 + }, + "some": { + "avg10": 2.58, + "avg300": 2.14, + "avg60": 2.66 + } + }, + "io_pressure_before": { + "full": { + "avg10": 1.42, + "avg300": 2.16, + "avg60": 3.02 + }, + "some": { + "avg10": 1.42, + "avg300": 2.18, + "avg60": 3.04 + } + }, + "root_device_read_delta_bytes": 237568, + "root_device_write_delta_bytes": 387596288 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 40.37, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 160, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 112, + "p99_ttft_ms": 112, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 3, + "model": "q008-responses", + "p50_elapsed_ms": 17, + "p50_ttft_ms": 17, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 81, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 57, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-tvots72h/config.toml Server: 127.0.0.1:48009 Accounts: 1 Database: Content digest: 70384cb3c4f7bb46a3bb8300d4953114d621bf8886d79299bbfa4a3e53c82fe9", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 87, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 40, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2421735, \"ppid\": 2421713, \"uptime_seconds\": 2.872418947, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 439, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-tvots72h/backups/eggpool-backup-20261016-010615.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 55, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2421735)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 127, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-tvots72h/recovery/config.toml db: /tmp/eggpool-q008-tvots72h/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 48, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2423003)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 2.6, + "avg300": 2.21, + "avg60": 3.34 + }, + "some": { + "avg10": 2.6, + "avg300": 2.22, + "avg60": 3.36 + } + }, + "root_device_read_bytes": 723934371840, + "root_device_write_bytes": 1282975065088 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 11 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21946368, + 21946368, + 23781376, + 23781376, + 23904256, + 24035328, + 24276992, + 46424064, + 46424064, + 46424064, + 24129536 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 56 + ], + "streaming_ttft_ms": [ + 11, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21946368, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21946368, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21946368, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21946368, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23781376, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23781376, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23781376, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23781376, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23904256, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23904256, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24035328, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24035328, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173047808, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24276992, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24276992, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173047808, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4571136, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46424064, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46424064, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173174784, + "wal_bytes": 4375472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4571136, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46424064, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46424064, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173174784, + "wal_bytes": 4375472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4571136, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46424064, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46424064, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173174784, + "wal_bytes": 4375472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 24129536, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24129536, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103596032, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 103, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u10-c1-run-2.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u10-c1-run-2.json new file mode 100644 index 00000000..1d68b8c0 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u10-c1-run-2.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "7f42e70a49207f322bd3045897a465d07903d4c0af5218c8f53abaafa5eea685", + "archive_size_bytes": 4603351, + "elapsed_ms": 465, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 10, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23633920, + "baseline_rss_bytes": 23633920, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46280704, + "peak_rss_bytes": 46280704 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 22952, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 34.11, + "cpu_ticks": 783, + "cpu_time_ms": 7830 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 0.57, + "avg300": 1.93, + "avg60": 1.65 + }, + "some": { + "avg10": 0.57, + "avg300": 1.94, + "avg60": 1.66 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.97, + "avg300": 2.06, + "avg60": 2.26 + }, + "some": { + "avg10": 0.97, + "avg300": 2.08, + "avg60": 2.26 + } + }, + "root_device_read_delta_bytes": 8192, + "root_device_write_delta_bytes": 391172096 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 43.568, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 90, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 77, + "p99_ttft_ms": 76, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 81, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 8, + "model": "q008-messages", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-kd0tqlu0/config.toml Server: 127.0.0.1:46527 Accounts: 1 Database: Content digest: 29ee51d3acf14a3943b89a5f5f0ff5c2c5c286257db44ca19c96189daa952615", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 90, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 43, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2423032, \"ppid\": 2423010, \"uptime_seconds\": 2.880023998, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 5, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 465, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-kd0tqlu0/backups/eggpool-backup-20261016-010649.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 62, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2423032)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 97, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-kd0tqlu0/recovery/config.toml db: /tmp/eggpool-q008-kd0tqlu0/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 49, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 55, + "id": "stop", + "reason": "Stopping server (PID 2424298)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 1.78, + "avg300": 2.11, + "avg60": 2.49 + }, + "some": { + "avg10": 1.78, + "avg300": 2.12, + "avg60": 2.5 + } + }, + "root_device_read_bytes": 723934629888, + "root_device_write_bytes": 1283440776192 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 4, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 4 + }, + { + "body_bytes": 309, + "elapsed_ms": 4, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 4 + }, + { + "body_bytes": 213, + "elapsed_ms": 4, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 4 + }, + { + "body_bytes": 145, + "elapsed_ms": 16, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 14 + }, + { + "body_bytes": 270, + "elapsed_ms": 15, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 14 + }, + { + "body_bytes": 480, + "elapsed_ms": 54, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 52 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21737472, + 21741568, + 23633920, + 23633920, + 23756800, + 23887872, + 24059904, + 46280704, + 46280704, + 46280704, + 23789568 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 4, + 4, + 4 + ], + "finite_ttft_ms": [ + 4, + 4, + 4 + ], + "second_workload_elapsed_ms": [ + 11, + 2 + ], + "second_workload_ttft_ms": [ + 11, + 2 + ], + "streaming_elapsed_ms": [ + 16, + 15, + 54 + ], + "streaming_ttft_ms": [ + 14, + 14, + 52 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21737472, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21737472, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21741568, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21741568, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23633920, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23633920, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172904448, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23633920, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23633920, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172904448, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23756800, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23756800, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172904448, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23887872, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23887872, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173039616, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24059904, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24059904, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173039616, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4599808, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46280704, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46280704, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173174784, + "wal_bytes": 4437272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4599808, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46280704, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46280704, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173174784, + "wal_bytes": 4437272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4599808, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46280704, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46280704, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173174784, + "wal_bytes": 4437272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23789568, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23789568, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103596032, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 108, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u10-c1-run-3.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u10-c1-run-3.json new file mode 100644 index 00000000..724a5cf3 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u10-c1-run-3.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "40dbefb31739029ea58e49e314c4022177ffd85ba8c80539d0a80b79ed9c24b4", + "archive_size_bytes": 4607447, + "elapsed_ms": 451, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 10, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23683072, + "baseline_rss_bytes": 23683072, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46333952, + "peak_rss_bytes": 46333952 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 23333, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 34.84, + "cpu_ticks": 813, + "cpu_time_ms": 8130 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 0.24, + "avg300": 1.74, + "avg60": 1.04 + }, + "some": { + "avg10": 0.26, + "avg300": 1.76, + "avg60": 1.06 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.33, + "avg300": 1.87, + "avg60": 1.43 + }, + "some": { + "avg10": 0.33, + "avg300": 1.88, + "avg60": 1.43 + } + }, + "root_device_read_delta_bytes": 6541312, + "root_device_write_delta_bytes": 384139264 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 42.858, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 89, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 16, + "p50_ttft_ms": 16, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 65, + "p99_ttft_ms": 65, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 60, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 92, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 9, + "model": "q008-messages", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-vqf_94ch/config.toml Server: 127.0.0.1:37271 Accounts: 1 Database: Content digest: 59e583a18342392bf6c9ed40b7bc7a2836140ed5d2826a3cfd07077072837c92", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 99, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2424329, \"ppid\": 2424306, \"uptime_seconds\": 2.874455588, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 8, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 451, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-vqf_94ch/backups/eggpool-backup-20261016-010724.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 55, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2424329)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 128, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-vqf_94ch/recovery/config.toml db: /tmp/eggpool-q008-vqf_94ch/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 47, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2425577)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.38, + "avg300": 1.9, + "avg60": 1.55 + }, + "some": { + "avg10": 0.38, + "avg300": 1.92, + "avg60": 1.55 + } + }, + "root_device_read_bytes": 723934646272, + "root_device_write_bytes": 1283907466240 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21905408, + 21909504, + 23683072, + 23683072, + 23793664, + 23924736, + 24109056, + 46333952, + 46333952, + 46333952, + 23793664 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 10, + 2 + ], + "second_workload_ttft_ms": [ + 10, + 2 + ], + "streaming_elapsed_ms": [ + 12, + 13, + 56 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21905408, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21905408, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103559168, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21909504, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21909504, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103559168, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23683072, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23683072, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23683072, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23683072, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23793664, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23793664, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23924736, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23924736, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24109056, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24109056, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173047808, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4579328, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46333952, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46333952, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173174784, + "wal_bytes": 4338392, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4579328, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46333952, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46333952, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173174784, + "wal_bytes": 4338392, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4579328, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46333952, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46333952, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173174784, + "wal_bytes": 4338392, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23793664, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23793664, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103596032, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u25-c1-run-1.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u25-c1-run-1.json new file mode 100644 index 00000000..3b90fbbd --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u25-c1-run-1.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "55f7cf2b8bdf60a01d6c1e790c13b798a04c149c49dd4e01897e115d0682116f", + "archive_size_bytes": 4627927, + "elapsed_ms": 443, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 25, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23769088, + "baseline_rss_bytes": 23769088, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46555136, + "peak_rss_bytes": 46555136 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 23532, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 35.82, + "cpu_ticks": 843, + "cpu_time_ms": 8430 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 0.43, + "avg300": 1.57, + "avg60": 0.73 + }, + "some": { + "avg10": 0.43, + "avg300": 1.59, + "avg60": 0.74 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.07, + "avg300": 1.67, + "avg60": 0.85 + }, + "some": { + "avg10": 0.08, + "avg300": 1.68, + "avg60": 0.87 + } + }, + "root_device_read_delta_bytes": 114688, + "root_device_write_delta_bytes": 384835584 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 42.495, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 91, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 16, + "p50_ttft_ms": 16, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 77, + "p99_ttft_ms": 76, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 3, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 77, + "minimum_elapsed_ms": 3, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 9, + "model": "q008-messages", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-b2idl2d4/config.toml Server: 127.0.0.1:59547 Accounts: 1 Database: Content digest: fdc86b8c175090d7a7dc54c7dcb2126873a747a13b1f2a49aab58335374567d7", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 100, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 5, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2425606, \"ppid\": 2425584, \"uptime_seconds\": 2.92280493, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, ", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 443, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-b2idl2d4/backups/eggpool-backup-20261016-010800.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 57, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2425606)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 129, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-b2idl2d4/recovery/config.toml db: /tmp/eggpool-q008-b2idl2d4/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 72, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2426919)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.13, + "avg300": 1.7, + "avg60": 0.94 + }, + "some": { + "avg10": 0.14, + "avg300": 1.72, + "avg60": 0.96 + } + }, + "root_device_read_bytes": 723941212160, + "root_device_write_bytes": 1284371088384 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 11, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 11 + }, + { + "body_bytes": 309, + "elapsed_ms": 9, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 213, + "elapsed_ms": 9, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 145, + "elapsed_ms": 20, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 270, + "elapsed_ms": 19, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 480, + "elapsed_ms": 60, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21880832, + 21884928, + 23769088, + 23769088, + 23900160, + 24039424, + 24268800, + 46555136, + 46555136, + 46555136, + 23875584 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 11, + 9, + 9 + ], + "finite_ttft_ms": [ + 11, + 9, + 9 + ], + "second_workload_elapsed_ms": [ + 11, + 2 + ], + "second_workload_ttft_ms": [ + 11, + 2 + ], + "streaming_elapsed_ms": [ + 20, + 19, + 60 + ], + "streaming_ttft_ms": [ + 17, + 17, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21880832, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21880832, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21884928, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21884928, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23769088, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23769088, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172900352, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23769088, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23769088, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172900352, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23900160, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23900160, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172900352, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24039424, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24039424, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173035520, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24268800, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24268800, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173035520, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4599808, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46555136, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46555136, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173461504, + "wal_bytes": 4371352, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4599808, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46555136, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46555136, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173461504, + "wal_bytes": 4371352, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4599808, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46555136, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46555136, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173461504, + "wal_bytes": 4371352, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4497408, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23875584, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23875584, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103600128, + "wal_bytes": 885832, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u25-c1-run-2.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u25-c1-run-2.json new file mode 100644 index 00000000..c1e3b8f8 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u25-c1-run-2.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "29c23718454ceeac9ac460ff1daf48931b848dbec8bd52e795cd888398a69e32", + "archive_size_bytes": 4627927, + "elapsed_ms": 453, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 25, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23773184, + "baseline_rss_bytes": 23773184, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46624768, + "peak_rss_bytes": 46624768 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 23174, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 34.82, + "cpu_ticks": 807, + "cpu_time_ms": 8070 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 0.31, + "avg300": 1.42, + "avg60": 0.55 + }, + "some": { + "avg10": 0.54, + "avg300": 1.45, + "avg60": 0.62 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.22, + "avg300": 1.51, + "avg60": 0.62 + }, + "some": { + "avg10": 0.22, + "avg300": 1.53, + "avg60": 0.64 + } + }, + "root_device_read_delta_bytes": 348160, + "root_device_write_delta_bytes": 388775936 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 43.152, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 92, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 81, + "p99_ttft_ms": 81, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 70, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-8wkz4g5z/config.toml Server: 127.0.0.1:36941 Accounts: 1 Database: Content digest: 86599a79da5a5ad860dd0a3dc927c345ac2b0e9f26fb85f89e286bd19ef826d7", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 90, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 11, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 9, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 5, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2426948, \"ppid\": 2426926, \"uptime_seconds\": 2.886664202, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 453, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-8wkz4g5z/backups/eggpool-backup-20261016-010835.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 55, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2426948)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 91, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-8wkz4g5z/recovery/config.toml db: /tmp/eggpool-q008-8wkz4g5z/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 55, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop", + "reason": "Stopping server (PID 2428287)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.28, + "avg300": 1.55, + "avg60": 0.68 + }, + "some": { + "avg10": 0.28, + "avg300": 1.56, + "avg60": 0.69 + } + }, + "root_device_read_bytes": 723941453824, + "root_device_write_bytes": 1284838523904 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 11 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 55, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 22077440, + 22081536, + 23773184, + 23773184, + 23896064, + 24027136, + 24203264, + 46624768, + 46624768, + 46624768, + 23887872 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 12, + 13, + 55 + ], + "streaming_ttft_ms": [ + 11, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 22077440, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 22077440, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 22081536, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 22081536, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23773184, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23773184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23773184, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23773184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23896064, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23896064, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24027136, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24027136, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24203264, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24203264, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173047808, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4624384, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46624768, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46624768, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173461504, + "wal_bytes": 4375472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4624384, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46624768, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46624768, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173461504, + "wal_bytes": 4375472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4624384, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46624768, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46624768, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173461504, + "wal_bytes": 4375472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4497408, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23887872, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23887872, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103591936, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u25-c1-run-3.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u25-c1-run-3.json new file mode 100644 index 00000000..44d28139 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u25-c1-run-3.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "1dbc74f13410f176a40ac9360358520ff6a4dcd5665e68266205b324234c1c9f", + "archive_size_bytes": 4627927, + "elapsed_ms": 464, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 25, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23879680, + "baseline_rss_bytes": 23879680, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46788608, + "peak_rss_bytes": 46788608 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 26014, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 32.71, + "cpu_ticks": 851, + "cpu_time_ms": 8510 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 2.25, + "avg300": 1.48, + "avg60": 1.24 + }, + "some": { + "avg10": 2.41, + "avg300": 1.51, + "avg60": 1.31 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.19, + "avg300": 1.37, + "avg60": 0.48 + }, + "some": { + "avg10": 0.26, + "avg300": 1.4, + "avg60": 0.54 + } + }, + "root_device_read_delta_bytes": 135168, + "root_device_write_delta_bytes": 385429504 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 38.441, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 228, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 17, + "p50_ttft_ms": 17, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 115, + "p99_ttft_ms": 115, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 3, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 81, + "minimum_elapsed_ms": 6, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 9, + "model": "q008-messages", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-31r3wbmm/config.toml Server: 127.0.0.1:42967 Accounts: 1 Database: Content digest: fe3cd8817f96392da3d12ab8eba66f7f16ffae45586533577c6dcb1608f652f1", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 92, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 40, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 5, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 6, \"native_count\": 6, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2428316, \"ppid\": 2428294, \"uptime_seconds\": 2.872301926, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 464, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-31r3wbmm/backups/eggpool-backup-20261016-010913.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2428316)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 92, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-31r3wbmm/recovery/config.toml db: /tmp/eggpool-q008-31r3wbmm/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 46, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 55, + "id": "stop", + "reason": "Stopping server (PID 2429623)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.2, + "avg300": 1.4, + "avg60": 0.51 + }, + "some": { + "avg10": 0.36, + "avg300": 1.43, + "avg60": 0.58 + } + }, + "root_device_read_bytes": 723941834752, + "root_device_write_bytes": 1285302752256 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 11 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 55, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 22110208, + 22114304, + 23879680, + 23879680, + 23994368, + 24113152, + 24371200, + 46788608, + 46788608, + 46788608, + 23855104 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 17, + 8 + ], + "second_workload_ttft_ms": [ + 17, + 8 + ], + "streaming_elapsed_ms": [ + 12, + 13, + 55 + ], + "streaming_ttft_ms": [ + 11, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 22110208, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 22110208, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 22114304, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 22114304, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23879680, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23879680, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23879680, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23879680, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23994368, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23994368, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24113152, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24113152, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24371200, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24371200, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173051904, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4603904, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46788608, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46788608, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173457408, + "wal_bytes": 4424912, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4603904, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46788608, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46788608, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173457408, + "wal_bytes": 4424912, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4603904, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46788608, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46788608, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173457408, + "wal_bytes": 4424912, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4497408, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23855104, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23855104, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103596032, + "wal_bytes": 885832, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u49-c1-run-1.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u49-c1-run-1.json new file mode 100644 index 00000000..80784457 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u49-c1-run-1.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "792ee31f1f0504ddb3e0461d28e4840292bc3e807d3e500ae2b6b0ff130a63b5", + "archive_size_bytes": 4697559, + "elapsed_ms": 450, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 49, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23535616, + "baseline_rss_bytes": 23535616, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46989312, + "peak_rss_bytes": 46989312 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 23363, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 35.14, + "cpu_ticks": 821, + "cpu_time_ms": 8210 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 0.08, + "avg300": 1.31, + "avg60": 0.7 + }, + "some": { + "avg10": 0.08, + "avg300": 1.33, + "avg60": 0.74 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.67, + "avg300": 1.42, + "avg60": 1.01 + }, + "some": { + "avg10": 0.72, + "avg300": 1.44, + "avg60": 1.07 + } + }, + "root_device_read_delta_bytes": 12288, + "root_device_write_delta_bytes": 383197184 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 42.803, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 97, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 81, + "p99_ttft_ms": 81, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 75, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 9, + "model": "q008-messages", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-q7yjcsxi/config.toml Server: 127.0.0.1:47773 Accounts: 1 Database: Content digest: 7ecbca9543c9dd51cd376816023099057f5083a30ddba1bcb7531109acf1e2e1", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 83, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 3, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 5, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 6, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 7, + "id": "transcoding-stats", + "reason": "{ \"total\": 6, \"native_count\": 6, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 4, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2429654, \"ppid\": 2429631, \"uptime_seconds\": 2.876774059, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 450, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-q7yjcsxi/backups/eggpool-backup-20261016-010950.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 63, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2429654)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 99, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-q7yjcsxi/recovery/config.toml db: /tmp/eggpool-q008-q7yjcsxi/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 52, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 56, + "id": "stop", + "reason": "Stopping server (PID 2431084)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 1.51, + "avg300": 1.46, + "avg60": 1.16 + }, + "some": { + "avg10": 1.61, + "avg300": 1.49, + "avg60": 1.22 + } + }, + "root_device_read_bytes": 723942068224, + "root_device_write_bytes": 1285767267328 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 53, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 52 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21573632, + 21573632, + 23535616, + 23535616, + 23646208, + 23773184, + 23957504, + 46989312, + 46989312, + 46989312, + 23904256 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 53 + ], + "streaming_ttft_ms": [ + 12, + 12, + 52 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21573632, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21573632, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21573632, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21573632, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23535616, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23535616, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23535616, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23535616, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23646208, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23646208, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23773184, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23773184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23957504, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23957504, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173047808, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4694016, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46989312, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46989312, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173953024, + "wal_bytes": 4350752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4694016, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46989312, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46989312, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173953024, + "wal_bytes": 4350752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4694016, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46989312, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46989312, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173953024, + "wal_bytes": 4350752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4567040, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23904256, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23904256, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103600128, + "wal_bytes": 898192, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u49-c1-run-2.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u49-c1-run-2.json new file mode 100644 index 00000000..5ff6af41 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u49-c1-run-2.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "4bf8f8763c656f05eba9826b99785b4319cb5b855bd30e21e233436923a27887", + "archive_size_bytes": 4705751, + "elapsed_ms": 452, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 49, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23875584, + "baseline_rss_bytes": 23875584, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 47308800, + "peak_rss_bytes": 47308800 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 25996, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 31.39, + "cpu_ticks": 816, + "cpu_time_ms": 8160 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 1.89, + "avg300": 1.43, + "avg60": 1.47 + }, + "some": { + "avg10": 1.89, + "avg300": 1.45, + "avg60": 1.49 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.12, + "avg300": 1.26, + "avg60": 0.6 + }, + "some": { + "avg10": 0.12, + "avg300": 1.28, + "avg60": 0.63 + } + }, + "root_device_read_delta_bytes": 704512, + "root_device_write_delta_bytes": 387809280 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 38.467, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 195, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 131, + "p99_ttft_ms": 131, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 74, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 6, + "model": "q008-messages", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-lwtsb9aq/config.toml Server: 127.0.0.1:50077 Accounts: 1 Database: Content digest: 8d41f75ee7197732e39e17a95453fe80e04b4e1457a41aca8849303e202fa968", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 87, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 38, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 2, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 5, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 5, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2431113, \"ppid\": 2431091, \"uptime_seconds\": 2.870891068, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 14, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 452, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-lwtsb9aq/backups/eggpool-backup-20261016-011028.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2431113)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 130, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-lwtsb9aq/recovery/config.toml db: /tmp/eggpool-q008-lwtsb9aq/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 56, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2432532)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.05, + "avg300": 1.29, + "avg60": 0.65 + }, + "some": { + "avg10": 0.05, + "avg300": 1.31, + "avg60": 0.69 + } + }, + "root_device_read_bytes": 723942166528, + "root_device_write_bytes": 1286230963200 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 53, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 52 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 22110208, + 22114304, + 23875584, + 23875584, + 24002560, + 24125440, + 24383488, + 47308800, + 47308800, + 47308800, + 23977984 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 11 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 53 + ], + "streaming_ttft_ms": [ + 12, + 12, + 52 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 22110208, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 22110208, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 22114304, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 22114304, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23875584, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23875584, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23875584, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23875584, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 24002560, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24002560, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 4.99, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24125440, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24125440, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173043712, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24383488, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24383488, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173043712, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4694016, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 47308800, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47308800, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173977600, + "wal_bytes": 4416672, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4694016, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 47308800, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47308800, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173977600, + "wal_bytes": 4416672, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4694016, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 47308800, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47308800, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173977600, + "wal_bytes": 4416672, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4567040, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23977984, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23977984, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103600128, + "wal_bytes": 894072, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u49-c1-run-3.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u49-c1-run-3.json new file mode 100644 index 00000000..4259fc6f --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u49-c1-run-3.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "fc423b471df4ce2bd5dfe9a3fb42765343bdebeef7e425dd8f1dcb866af6b0bb", + "archive_size_bytes": 4697559, + "elapsed_ms": 444, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 49, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23691264, + "baseline_rss_bytes": 23691264, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46985216, + "peak_rss_bytes": 46985216 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 23628, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 35.42, + "cpu_ticks": 837, + "cpu_time_ms": 8370 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 0.52, + "avg300": 1.29, + "avg60": 0.95 + }, + "some": { + "avg10": 0.52, + "avg300": 1.31, + "avg60": 0.96 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.46, + "avg300": 1.36, + "avg60": 1.16 + }, + "some": { + "avg10": 0.46, + "avg300": 1.38, + "avg60": 1.18 + } + }, + "root_device_read_delta_bytes": 65536, + "root_device_write_delta_bytes": 382627840 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 42.322, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 102, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 17, + "p50_ttft_ms": 17, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 80, + "p99_ttft_ms": 80, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 79, + "minimum_elapsed_ms": 3, + "model": "q008-responses", + "p50_elapsed_ms": 14, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 9, + "model": "q008-messages", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-rxh_90la/config.toml Server: 127.0.0.1:43721 Accounts: 1 Database: Content digest: 9621da4f60d4532e5257f06193793086091f8530a8e1e3118aa8882c889e0a8c", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 95, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 4, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 5, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2432561, \"ppid\": 2432539, \"uptime_seconds\": 2.871375416, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 444, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-rxh_90la/backups/eggpool-backup-20261016-011105.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 62, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2432561)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 122, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-rxh_90la/recovery/config.toml db: /tmp/eggpool-q008-rxh_90la/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 49, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2433967)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 1.03, + "avg300": 1.4, + "avg60": 1.33 + }, + "some": { + "avg10": 1.03, + "avg300": 1.42, + "avg60": 1.35 + } + }, + "root_device_read_bytes": 723942912000, + "root_device_write_bytes": 1286700110848 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 11 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21733376, + 21737472, + 23691264, + 23691264, + 23822336, + 23949312, + 24125440, + 46985216, + 46985216, + 46985216, + 23855104 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 56 + ], + "streaming_ttft_ms": [ + 11, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21733376, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21733376, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21737472, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21737472, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23691264, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23691264, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172916736, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23691264, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23691264, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172916736, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23822336, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23822336, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172916736, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23949312, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23949312, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172916736, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24125440, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24125440, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173060096, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4685824, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46985216, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46985216, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173961216, + "wal_bytes": 4424912, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4685824, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46985216, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46985216, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173961216, + "wal_bytes": 4424912, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4685824, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46985216, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46985216, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173961216, + "wal_bytes": 4424912, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4567040, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23855104, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23855104, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103600128, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c1-run-1.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c1-run-1.json new file mode 100644 index 00000000..134a3f6e --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c1-run-1.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "1de7247e2c8fe429bd9d15b8a0d683d9ba033a80b273b8577dced8cdb8675243", + "archive_size_bytes": 4611543, + "elapsed_ms": 483, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23695360, + "baseline_rss_bytes": 23695360, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46284800, + "peak_rss_bytes": 46284800 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 23709, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 33.03, + "cpu_ticks": 783, + "cpu_time_ms": 7830 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 1.2, + "avg300": 1.19, + "avg60": 2.38 + }, + "some": { + "avg10": 1.21, + "avg300": 1.2, + "avg60": 2.4 + } + }, + "io_pressure_before": { + "full": { + "avg10": 3.51, + "avg300": 1.16, + "avg60": 2.86 + }, + "some": { + "avg10": 3.54, + "avg300": 1.18, + "avg60": 2.9 + } + }, + "root_device_read_delta_bytes": 1531904, + "root_device_write_delta_bytes": 391294976 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 42.179, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 152, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 89, + "p99_ttft_ms": 89, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 127, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 97, + "p95_ttft_ms": 97, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 161, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 123, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 121, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 99, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-hexyh2tc/config.toml Server: 127.0.0.1:50003 Accounts: 1 Database: Content digest: 87b8e43d694f9916a4ce00a9765ec3f9197d5aaa8a9888f046d7da35a2c0794d", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 158, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 57, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 3, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 5, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 6, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 6, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 4, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2403081, \"ppid\": 2403059, \"uptime_seconds\": 2.851965981, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 483, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-hexyh2tc/backups/eggpool-backup-20261016-010047.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2403081)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 94, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-hexyh2tc/recovery/config.toml db: /tmp/eggpool-q008-hexyh2tc/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 46, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 56, + "id": "stop", + "reason": "Stopping server (PID 2404348)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 4.32, + "avg300": 1.11, + "avg60": 2.85 + }, + "some": { + "avg10": 4.4, + "avg300": 1.13, + "avg60": 2.89 + } + }, + "root_device_read_bytes": 723929182208, + "root_device_write_bytes": 1275975922688 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 53, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 52 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21737472, + 21741568, + 23695360, + 23695360, + 23822336, + 23977984, + 24133632, + 46284800, + 46284800, + 46284800, + 23875584 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 12, + 13, + 53 + ], + "streaming_ttft_ms": [ + 12, + 12, + 52 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21737472, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21737472, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21741568, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21741568, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23695360, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23695360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172916736, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23695360, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23695360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172916736, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23822336, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23822336, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172916736, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23977984, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23977984, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172916736, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24133632, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24133632, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173056000, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4608000, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46284800, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46284800, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173182976, + "wal_bytes": 4424912, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4608000, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46284800, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46284800, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173182976, + "wal_bytes": 4424912, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4608000, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46284800, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46284800, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173182976, + "wal_bytes": 4424912, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23875584, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23875584, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103596032, + "wal_bytes": 885832, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 103, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c1-run-2.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c1-run-2.json new file mode 100644 index 00000000..bf29c407 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c1-run-2.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "0aa0b5d0e14f8c819e1fe7e8b104aaf7ad86679e2ac551cd802ab2524f64f587", + "archive_size_bytes": 4607447, + "elapsed_ms": 450, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23695360, + "baseline_rss_bytes": 23695360, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46301184, + "peak_rss_bytes": 46301184 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 26494, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 30.99, + "cpu_ticks": 821, + "cpu_time_ms": 8210 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 3.15, + "avg300": 1.39, + "avg60": 2.67 + }, + "some": { + "avg10": 3.15, + "avg300": 1.4, + "avg60": 2.69 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.56, + "avg300": 1.15, + "avg60": 2.04 + }, + "some": { + "avg10": 0.56, + "avg300": 1.16, + "avg60": 2.06 + } + }, + "root_device_read_delta_bytes": 1421312, + "root_device_write_delta_bytes": 389419008 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 37.744, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 243, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 62, + "p95_ttft_ms": 62, + "p99_elapsed_ms": 141, + "p99_ttft_ms": 141, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 19, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 81, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 9, + "model": "q008-messages", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-znco7qvs/config.toml Server: 127.0.0.1:51993 Accounts: 1 Database: Content digest: dd7464240ea490da99151fecd6304089a6bd1dd845684356e08f2b18c1942dcf", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 93, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2404379, \"ppid\": 2404355, \"uptime_seconds\": 2.918807319, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 450, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-znco7qvs/backups/eggpool-backup-20261016-010125.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2404379)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 139, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-znco7qvs/recovery/config.toml db: /tmp/eggpool-q008-znco7qvs/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 46, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2405617)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.81, + "avg300": 1.17, + "avg60": 2.22 + }, + "some": { + "avg10": 0.81, + "avg300": 1.18, + "avg60": 2.25 + } + }, + "root_device_read_bytes": 723931471872, + "root_device_write_bytes": 1276442641408 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 11, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 11 + }, + { + "body_bytes": 309, + "elapsed_ms": 9, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 213, + "elapsed_ms": 9, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 145, + "elapsed_ms": 19, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 270, + "elapsed_ms": 19, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 480, + "elapsed_ms": 59, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21741568, + 21745664, + 23695360, + 23695360, + 23838720, + 23977984, + 24137728, + 46301184, + 46301184, + 46301184, + 23851008 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 11, + 9, + 9 + ], + "finite_ttft_ms": [ + 11, + 9, + 9 + ], + "second_workload_elapsed_ms": [ + 10, + 2 + ], + "second_workload_ttft_ms": [ + 10, + 2 + ], + "streaming_elapsed_ms": [ + 19, + 19, + 59 + ], + "streaming_ttft_ms": [ + 17, + 17, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21741568, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21741568, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21745664, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21745664, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23695360, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23695360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23695360, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23695360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23838720, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23838720, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23977984, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23977984, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173047808, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24137728, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24137728, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173047808, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4603904, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46301184, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46301184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173170688, + "wal_bytes": 4391952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4603904, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46301184, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46301184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173170688, + "wal_bytes": 4391952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4603904, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46301184, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46301184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173170688, + "wal_bytes": 4391952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23851008, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23851008, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103600128, + "wal_bytes": 885832, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c1-run-3.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c1-run-3.json new file mode 100644 index 00000000..7df9d6f3 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c1-run-3.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "f992278c116098c1507666a85d2f7df4eba7829cabe077aad656b54ed8bd854f", + "archive_size_bytes": 4607447, + "elapsed_ms": 654, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23707648, + "baseline_rss_bytes": 23707648, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46403584, + "peak_rss_bytes": 46403584 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 23328, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 33.48, + "cpu_ticks": 781, + "cpu_time_ms": 7810 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 1.68, + "avg300": 1.35, + "avg60": 2.0 + }, + "some": { + "avg10": 1.68, + "avg300": 1.35, + "avg60": 2.01 + } + }, + "io_pressure_before": { + "full": { + "avg10": 1.28, + "avg300": 1.35, + "avg60": 2.29 + }, + "some": { + "avg10": 1.28, + "avg300": 1.36, + "avg60": 2.3 + } + }, + "root_device_read_delta_bytes": 40960, + "root_device_write_delta_bytes": 388472832 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 42.868, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 120, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 84, + "p99_ttft_ms": 84, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 3, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 82, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 9, + "model": "q008-messages", + "p50_elapsed_ms": 15, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-7kgb938_/config.toml Server: 127.0.0.1:33525 Accounts: 1 Database: Content digest: 7d51d33789b123f419d87a9cdf768473cab72db5d1436c817b6336a19adcad21", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 84, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2405646, \"ppid\": 2405624, \"uptime_seconds\": 2.857263451, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 5, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 654, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-7kgb938_/backups/eggpool-backup-20261016-010200.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 62, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2405646)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 556, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-7kgb938_/recovery/config.toml db: /tmp/eggpool-q008-7kgb938_/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 240, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop", + "reason": "Stopping server (PID 2406892)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 2.11, + "avg300": 1.37, + "avg60": 2.5 + }, + "some": { + "avg10": 2.11, + "avg300": 1.38, + "avg60": 2.51 + } + }, + "root_device_read_bytes": 723932942336, + "root_device_write_bytes": 1276906857472 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 1 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 11 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21950464, + 21954560, + 23707648, + 23707648, + 23838720, + 23961600, + 24145920, + 46403584, + 46403584, + 46403584, + 23826432 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 1, + 2 + ], + "second_workload_elapsed_ms": [ + 10, + 2 + ], + "second_workload_ttft_ms": [ + 10, + 2 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 56 + ], + "streaming_ttft_ms": [ + 11, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21950464, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21950464, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21954560, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21954560, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23707648, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23707648, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23707648, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23707648, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23838720, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23838720, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23961600, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23961600, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24145920, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24145920, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173043712, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4595712, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46403584, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46403584, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173178880, + "wal_bytes": 4346632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4595712, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46403584, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46403584, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173178880, + "wal_bytes": 4346632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4595712, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46403584, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46403584, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173178880, + "wal_bytes": 4346632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23826432, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23826432, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103600128, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c16-run-1.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c16-run-1.json new file mode 100644 index 00000000..0f280d74 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c16-run-1.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "7b9fa91add8f848c2b241b04bbe1d62af56222984b298c554966cdd0d7ecbccf", + "archive_size_bytes": 4705751, + "elapsed_ms": 460, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 16, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23695360, + "baseline_rss_bytes": 23695360, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 47927296, + "peak_rss_bytes": 47927296 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 3123, + "client_concurrency": 16, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 71.09, + "cpu_ticks": 222, + "cpu_time_ms": 2220 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 3.94, + "avg300": 2.19, + "avg60": 4.1 + }, + "some": { + "avg10": 3.94, + "avg300": 2.21, + "avg60": 4.12 + } + }, + "io_pressure_before": { + "full": { + "avg10": 3.48, + "avg300": 2.16, + "avg60": 4.04 + }, + "some": { + "avg10": 3.48, + "avg300": 2.18, + "avg60": 4.06 + } + }, + "root_device_read_delta_bytes": 8192, + "root_device_write_delta_bytes": 392232960 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 320.239, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 1084, + "minimum_elapsed_ms": 5, + "p50_elapsed_ms": 35, + "p50_ttft_ms": 35, + "p95_elapsed_ms": 95, + "p95_ttft_ms": 95, + "p99_elapsed_ms": 151, + "p99_ttft_ms": 151, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 93, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 83, + "p95_ttft_ms": 83, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 145, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 143, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 113, + "minimum_elapsed_ms": 6, + "model": "q008-messages", + "p50_elapsed_ms": 17, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 96, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-xw_0dr0w/config.toml Server: 127.0.0.1:48497 Accounts: 1 Database: Content digest: f33b62a3fc9ce3fc8598bf021c1497462dc72b3257b1da79a0cf734f63351e0a", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 137, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 61, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 11, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 16, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2418008, \"ppid\": 2417984, \"uptime_seconds\": 2.8869378, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 460, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-xw_0dr0w/backups/eggpool-backup-20261016-010509.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2418008)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 97, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-xw_0dr0w/recovery/config.toml db: /tmp/eggpool-q008-xw_0dr0w/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 51, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop", + "reason": "Stopping server (PID 2419206)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 4.04, + "avg300": 2.13, + "avg60": 4.16 + }, + "some": { + "avg10": 4.04, + "avg300": 2.15, + "avg60": 4.18 + } + }, + "root_device_read_bytes": 723933765632, + "root_device_write_bytes": 1281566082048 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21737472, + 21741568, + 23695360, + 23695360, + 23822336, + 23977984, + 24137728, + 47927296, + 47927296, + 47927296, + 23584768 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 12, + 13, + 56 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21737472, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21737472, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21741568, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21741568, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23695360, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23695360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172900352, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23695360, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23695360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172900352, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23822336, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23822336, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172900352, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23977984, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23977984, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173035520, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24137728, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24137728, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173035520, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4702208, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 47927296, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47927296, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174964736, + "wal_bytes": 4404312, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4702208, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 47927296, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47927296, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174964736, + "wal_bytes": 4404312, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4702208, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 47927296, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47927296, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174964736, + "wal_bytes": 4404312, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23584768, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23584768, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103596032, + "wal_bytes": 894072, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c16-run-2.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c16-run-2.json new file mode 100644 index 00000000..e2f0b6b2 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c16-run-2.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "2bf28a912183aec602af144fd862cdc7e6be5b171cd8efda0b4723b8e07a8233", + "archive_size_bytes": 4713943, + "elapsed_ms": 628, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 16, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23818240, + "baseline_rss_bytes": 23818240, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 47943680, + "peak_rss_bytes": 47943680 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 2930, + "client_concurrency": 16, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 72.37, + "cpu_ticks": 212, + "cpu_time_ms": 2120 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 2.73, + "avg300": 2.17, + "avg60": 3.6 + }, + "some": { + "avg10": 2.73, + "avg300": 2.19, + "avg60": 3.61 + } + }, + "io_pressure_before": { + "full": { + "avg10": 1.52, + "avg300": 2.13, + "avg60": 3.5 + }, + "some": { + "avg10": 1.52, + "avg300": 2.15, + "avg60": 3.51 + } + }, + "root_device_read_delta_bytes": 8192, + "root_device_write_delta_bytes": 391352320 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 341.347, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 1091, + "minimum_elapsed_ms": 9, + "p50_elapsed_ms": 35, + "p50_ttft_ms": 35, + "p95_elapsed_ms": 88, + "p95_ttft_ms": 88, + "p99_elapsed_ms": 106, + "p99_ttft_ms": 106, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 77, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 7, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 56, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-7xfvl9_7/config.toml Server: 127.0.0.1:39357 Accounts: 1 Database: Content digest: 5dd56ce5438fc49fd7865c9ca62df0f298e4ee4d390cc3c35de0a5cd08d6ed14", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 86, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 38, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 5, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2419235, \"ppid\": 2419213, \"uptime_seconds\": 2.874520744, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 628, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-7xfvl9_7/backups/eggpool-backup-20261016-010523.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 111, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2419235)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 534, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-7xfvl9_7/recovery/config.toml db: /tmp/eggpool-q008-7xfvl9_7/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 186, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2420481)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 2.77, + "avg300": 2.18, + "avg60": 3.87 + }, + "some": { + "avg10": 2.77, + "avg300": 2.2, + "avg60": 3.88 + } + }, + "root_device_read_bytes": 723934302208, + "root_device_write_bytes": 1282034078720 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 55, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21991424, + 21995520, + 23818240, + 23818240, + 23945216, + 24064000, + 24313856, + 47943680, + 47943680, + 47943680, + 23613440 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 11, + 2 + ], + "second_workload_ttft_ms": [ + 11, + 2 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 55 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21991424, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21991424, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21995520, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21995520, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23818240, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23818240, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172900352, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23818240, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23818240, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172900352, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23945216, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23945216, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172900352, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24064000, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24064000, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173039616, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24313856, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24313856, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173039616, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4706304, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 47943680, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47943680, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174837760, + "wal_bytes": 4400192, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4706304, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 47943680, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47943680, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174837760, + "wal_bytes": 4400192, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4706304, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 47943680, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47943680, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174837760, + "wal_bytes": 4400192, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23613440, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23613440, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103596032, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c16-run-3.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c16-run-3.json new file mode 100644 index 00000000..fe66003e --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c16-run-3.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "9b67a5e439b48f0133e8854134d59b5608c83f3d6e88139d1152f67ce4821b07", + "archive_size_bytes": 4701655, + "elapsed_ms": 458, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 16, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23805952, + "baseline_rss_bytes": 23805952, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 48001024, + "peak_rss_bytes": 48001024 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 2701, + "client_concurrency": 16, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 68.11, + "cpu_ticks": 184, + "cpu_time_ms": 1840 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 3.19, + "avg300": 2.2, + "avg60": 3.46 + }, + "some": { + "avg10": 3.19, + "avg300": 2.22, + "avg60": 3.47 + } + }, + "io_pressure_before": { + "full": { + "avg10": 2.57, + "avg300": 2.18, + "avg60": 3.37 + }, + "some": { + "avg10": 2.57, + "avg300": 2.19, + "avg60": 3.39 + } + }, + "root_device_read_delta_bytes": 0, + "root_device_write_delta_bytes": 390918144 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 370.175, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 1119, + "minimum_elapsed_ms": 7, + "p50_elapsed_ms": 32, + "p50_ttft_ms": 31, + "p95_elapsed_ms": 81, + "p95_ttft_ms": 81, + "p99_elapsed_ms": 94, + "p99_ttft_ms": 94, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 116, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 19, + "p95_elapsed_ms": 99, + "p95_ttft_ms": 99, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 144, + "minimum_elapsed_ms": 4, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 112, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 158, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 15, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 91, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-yzyilv24/config.toml Server: 127.0.0.1:40327 Accounts: 1 Database: Content digest: 379f76baade5b8baf44b658a29347512b2bff5d14d04be3e5bd25e1ab08d9adf", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 127, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 57, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 5, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2420510, \"ppid\": 2420488, \"uptime_seconds\": 2.914681292, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 458, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-yzyilv24/backups/eggpool-backup-20261016-010539.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 55, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2420510)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 130, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-yzyilv24/recovery/config.toml db: /tmp/eggpool-q008-yzyilv24/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 46, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2421706)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 2.01, + "avg300": 2.15, + "avg60": 3.4 + }, + "some": { + "avg10": 2.01, + "avg300": 2.17, + "avg60": 3.41 + } + }, + "root_device_read_bytes": 723934326784, + "root_device_write_bytes": 1282505425920 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 11, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 11 + }, + { + "body_bytes": 309, + "elapsed_ms": 9, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 213, + "elapsed_ms": 9, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 145, + "elapsed_ms": 20, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 270, + "elapsed_ms": 19, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 480, + "elapsed_ms": 60, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21909504, + 21913600, + 23805952, + 23805952, + 23941120, + 24080384, + 24309760, + 48001024, + 48001024, + 48001024, + 23900160 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 11, + 9, + 9 + ], + "finite_ttft_ms": [ + 11, + 9, + 9 + ], + "second_workload_elapsed_ms": [ + 10, + 2 + ], + "second_workload_ttft_ms": [ + 10, + 2 + ], + "streaming_elapsed_ms": [ + 20, + 19, + 60 + ], + "streaming_ttft_ms": [ + 17, + 17, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21909504, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21909504, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21913600, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21913600, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23805952, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23805952, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23805952, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23805952, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23941120, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23941120, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24080384, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24080384, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173043712, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24309760, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24309760, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173043712, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4694016, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 48001024, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 48001024, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 175001600, + "wal_bytes": 4334272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4694016, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 48001024, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 48001024, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 175001600, + "wal_bytes": 4334272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4694016, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 48001024, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 48001024, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 175001600, + "wal_bytes": 4334272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23900160, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23900160, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103596032, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c2-run-1.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c2-run-1.json new file mode 100644 index 00000000..769b993e --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c2-run-1.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "5766b4a282ddd307ed2b1a24cec47dccb70152c240302cf14d1cf99a259f342e", + "archive_size_bytes": 4636119, + "elapsed_ms": 443, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 2, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23666688, + "baseline_rss_bytes": 23666688, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46329856, + "peak_rss_bytes": 46329856 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 11004, + "client_concurrency": 2, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 57.07, + "cpu_ticks": 628, + "cpu_time_ms": 6280 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 1.33, + "avg300": 1.34, + "avg60": 1.77 + }, + "some": { + "avg10": 1.33, + "avg300": 1.35, + "avg60": 1.78 + } + }, + "io_pressure_before": { + "full": { + "avg10": 1.29, + "avg300": 1.34, + "avg60": 1.87 + }, + "some": { + "avg10": 1.29, + "avg300": 1.35, + "avg60": 1.88 + } + }, + "root_device_read_delta_bytes": 24576, + "root_device_write_delta_bytes": 385441792 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 90.874, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 95, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 62, + "p95_ttft_ms": 62, + "p99_elapsed_ms": 83, + "p99_ttft_ms": 83, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 82, + "minimum_elapsed_ms": 3, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 9, + "model": "q008-messages", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-tmm9wgh7/config.toml Server: 127.0.0.1:46959 Accounts: 1 Database: Content digest: 15540ca02bf84d5d0000da62aa11f298abda0f8cfabd1dce27c44c548e8a74ab", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 126, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 56, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2406923, \"ppid\": 2406900, \"uptime_seconds\": 2.87283895, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, ", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 443, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-tmm9wgh7/backups/eggpool-backup-20261016-010223.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 57, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2406923)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 98, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-tmm9wgh7/recovery/config.toml db: /tmp/eggpool-q008-tmm9wgh7/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 57, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 58, + "id": "stop", + "reason": "Stopping server (PID 2408144)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 2.13, + "avg300": 1.37, + "avg60": 2.03 + }, + "some": { + "avg10": 2.13, + "avg300": 1.37, + "avg60": 2.04 + } + }, + "root_device_read_bytes": 723933110272, + "root_device_write_bytes": 1277372298240 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 55, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21700608, + 21704704, + 23666688, + 23666688, + 23789568, + 23912448, + 24092672, + 46329856, + 46329856, + 46329856, + 23781376 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 10, + 2 + ], + "second_workload_ttft_ms": [ + 10, + 2 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 55 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21700608, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21700608, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21704704, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21704704, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23666688, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23666688, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172916736, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23666688, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23666688, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172916736, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23789568, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23789568, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172916736, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23912448, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23912448, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172916736, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24092672, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24092672, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173051904, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4620288, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46329856, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46329856, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173318144, + "wal_bytes": 4461992, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4620288, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46329856, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46329856, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173318144, + "wal_bytes": 4461992, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4620288, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46329856, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46329856, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173318144, + "wal_bytes": 4461992, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23781376, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23781376, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103596032, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c2-run-2.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c2-run-2.json new file mode 100644 index 00000000..4dd6dc24 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c2-run-2.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "291cc09ee01ae1a815fffc7de9d51150b407fd4d5f14cb1bf1a47eaadfadb47a", + "archive_size_bytes": 4640215, + "elapsed_ms": 602, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 2, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23625728, + "baseline_rss_bytes": 23625728, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46415872, + "peak_rss_bytes": 46415872 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 10405, + "client_concurrency": 2, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 54.3, + "cpu_ticks": 565, + "cpu_time_ms": 5650 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 1.02, + "avg300": 1.3, + "avg60": 1.48 + }, + "some": { + "avg10": 1.02, + "avg300": 1.31, + "avg60": 1.48 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.61, + "avg300": 1.31, + "avg60": 1.53 + }, + "some": { + "avg10": 0.61, + "avg300": 1.31, + "avg60": 1.53 + } + }, + "root_device_read_delta_bytes": 36864, + "root_device_write_delta_bytes": 386818048 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 96.109, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 96, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 77, + "p99_ttft_ms": 77, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 76, + "minimum_elapsed_ms": 4, + "model": "q008-responses", + "p50_elapsed_ms": 14, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 57, + "p95_ttft_ms": 56, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-d5wayb4y/config.toml Server: 127.0.0.1:50685 Accounts: 1 Database: Content digest: 722390fa1af80c235e28da3de0c678a11bbeaf8878bb144d8e77dc717b6dfd68", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 92, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2408173, \"ppid\": 2408151, \"uptime_seconds\": 2.867988475, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 602, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-d5wayb4y/backups/eggpool-backup-20261016-010245.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2408173)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 560, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-d5wayb4y/recovery/config.toml db: /tmp/eggpool-q008-d5wayb4y/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 221, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2409360)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.89, + "avg300": 1.33, + "avg60": 1.66 + }, + "some": { + "avg10": 0.89, + "avg300": 1.33, + "avg60": 1.66 + } + }, + "root_device_read_bytes": 723933175808, + "root_device_write_bytes": 1277835502592 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 3, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 3 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 11 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 55, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21729280, + 21733376, + 23625728, + 23625728, + 23760896, + 23871488, + 24064000, + 46415872, + 46415872, + 46415872, + 23744512 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 3, + 2, + 2 + ], + "finite_ttft_ms": [ + 3, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 55 + ], + "streaming_ttft_ms": [ + 11, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21729280, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21729280, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103559168, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21733376, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21733376, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103559168, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23625728, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23625728, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172904448, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23625728, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23625728, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172904448, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23760896, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23760896, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172904448, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23871488, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23871488, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173039616, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24064000, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24064000, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173039616, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4628480, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46415872, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46415872, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173297664, + "wal_bytes": 4449632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4628480, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46415872, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46415872, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173297664, + "wal_bytes": 4449632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4628480, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46415872, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46415872, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173297664, + "wal_bytes": 4449632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23744512, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23744512, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103596032, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 103, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c2-run-3.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c2-run-3.json new file mode 100644 index 00000000..55ace46b --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c2-run-3.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "273913b26e9d4158ada57bc9be3a9dbc5ca2360e726e0dbccfa73f80af0810dd", + "archive_size_bytes": 4640215, + "elapsed_ms": 486, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 2, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23781376, + "baseline_rss_bytes": 23781376, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46477312, + "peak_rss_bytes": 46477312 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 10641, + "client_concurrency": 2, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 54.79, + "cpu_ticks": 583, + "cpu_time_ms": 5830 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 1.68, + "avg300": 1.35, + "avg60": 1.62 + }, + "some": { + "avg10": 1.68, + "avg300": 1.35, + "avg60": 1.62 + } + }, + "io_pressure_before": { + "full": { + "avg10": 2.13, + "avg300": 1.35, + "avg60": 1.66 + }, + "some": { + "avg10": 2.13, + "avg300": 1.35, + "avg60": 1.66 + } + }, + "root_device_read_delta_bytes": 32768, + "root_device_write_delta_bytes": 383922176 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 93.976, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 91, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 79, + "p99_ttft_ms": 79, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 127, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 19, + "p95_elapsed_ms": 107, + "p95_ttft_ms": 107, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 135, + "minimum_elapsed_ms": 3, + "model": "q008-responses", + "p50_elapsed_ms": 9, + "p50_ttft_ms": 7, + "p95_elapsed_ms": 123, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 121, + "minimum_elapsed_ms": 4, + "model": "q008-messages", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-jv66w9nc/config.toml Server: 127.0.0.1:43231 Accounts: 1 Database: Content digest: 4d4799b8985a801d6b3f941a7cb70e591df7f6a39b5a19b5f7d36c5cd48222f2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 137, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 46, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 3, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 6, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 7, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 8, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 6, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2409402, \"ppid\": 2409367, \"uptime_seconds\": 2.908564229, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 8, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 486, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-jv66w9nc/backups/eggpool-backup-20261016-010308.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 62, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2409402)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 101, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-jv66w9nc/recovery/config.toml db: /tmp/eggpool-q008-jv66w9nc/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 55, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2410605)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 1.04, + "avg300": 1.29, + "avg60": 1.44 + }, + "some": { + "avg10": 1.04, + "avg300": 1.3, + "avg60": 1.44 + } + }, + "root_device_read_bytes": 723933229056, + "root_device_write_bytes": 1278299997184 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 6, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 6 + }, + { + "body_bytes": 309, + "elapsed_ms": 7, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 7 + }, + { + "body_bytes": 213, + "elapsed_ms": 9, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 145, + "elapsed_ms": 20, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 270, + "elapsed_ms": 19, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 480, + "elapsed_ms": 57, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 56 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21889024, + 21893120, + 23781376, + 23781376, + 23920640, + 24064000, + 24276992, + 46477312, + 46477312, + 46477312, + 23744512 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 6, + 7, + 9 + ], + "finite_ttft_ms": [ + 6, + 7, + 9 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 11 + ], + "streaming_elapsed_ms": [ + 20, + 19, + 57 + ], + "streaming_ttft_ms": [ + 17, + 17, + 56 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21889024, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21889024, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21893120, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21893120, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23781376, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23781376, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172900352, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23781376, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23781376, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172900352, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23920640, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23920640, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172900352, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24064000, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24064000, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173035520, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24276992, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24276992, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173035520, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4620288, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46477312, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46477312, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173297664, + "wal_bytes": 4330152, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4620288, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46477312, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46477312, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173297664, + "wal_bytes": 4330152, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4620288, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46477312, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46477312, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173297664, + "wal_bytes": 4330152, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23744512, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23744512, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103596032, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c4-run-1.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c4-run-1.json new file mode 100644 index 00000000..42851976 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c4-run-1.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "22e4499033056685f0a8d9736d73810c578f408b49492294d9112510879137e4", + "archive_size_bytes": 4668887, + "elapsed_ms": 597, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 4, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23863296, + "baseline_rss_bytes": 23863296, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46784512, + "peak_rss_bytes": 46784512 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 5163, + "client_concurrency": 4, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 65.27, + "cpu_ticks": 337, + "cpu_time_ms": 3370 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 0.85, + "avg300": 1.3, + "avg60": 1.33 + }, + "some": { + "avg10": 1.03, + "avg300": 1.31, + "avg60": 1.37 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.5, + "avg300": 1.29, + "avg60": 1.32 + }, + "some": { + "avg10": 0.5, + "avg300": 1.3, + "avg60": 1.32 + } + }, + "root_device_read_delta_bytes": 0, + "root_device_write_delta_bytes": 389509120 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 193.683, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 91, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "p99_elapsed_ms": 75, + "p99_ttft_ms": 75, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 7, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 81, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 57, + "p95_ttft_ms": 56, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-62yz1xuv/config.toml Server: 127.0.0.1:48929 Accounts: 1 Database: Content digest: 22610a45786b1ecaa083919ae237e18f2ceb48015197aaf1212798cedbbfd518", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 108, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2410635, \"ppid\": 2410612, \"uptime_seconds\": 2.869951552, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 597, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-62yz1xuv/backups/eggpool-backup-20261016-010325.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 62, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2410635)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 499, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-62yz1xuv/recovery/config.toml db: /tmp/eggpool-q008-62yz1xuv/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 214, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 57, + "id": "stop", + "reason": "Stopping server (PID 2411827)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.92, + "avg300": 1.32, + "avg60": 1.46 + }, + "some": { + "avg10": 0.92, + "avg300": 1.33, + "avg60": 1.46 + } + }, + "root_device_read_bytes": 723933261824, + "root_device_write_bytes": 1278762427392 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 11 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 55, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 22102016, + 22106112, + 23863296, + 23863296, + 23990272, + 24100864, + 24354816, + 46784512, + 46784512, + 46784512, + 24113152 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 55 + ], + "streaming_ttft_ms": [ + 11, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 22102016, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 22102016, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 22106112, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 22106112, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23863296, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23863296, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172900352, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23863296, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23863296, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172900352, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23990272, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23990272, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172900352, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24100864, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24100864, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172900352, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24354816, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24354816, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173035520, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4665344, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46784512, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46784512, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173592576, + "wal_bytes": 4330152, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4665344, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46784512, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46784512, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173592576, + "wal_bytes": 4330152, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4665344, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46784512, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46784512, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173592576, + "wal_bytes": 4330152, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 24113152, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24113152, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103600128, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c4-run-2.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c4-run-2.json new file mode 100644 index 00000000..5374f76e --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c4-run-2.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "b148eca271cdf156ac02a15970758337e9932344d80ee9eebde713b7e6e5f7b1", + "archive_size_bytes": 4668887, + "elapsed_ms": 472, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 4, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23711744, + "baseline_rss_bytes": 23711744, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46481408, + "peak_rss_bytes": 46481408 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 4695, + "client_concurrency": 4, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 56.23, + "cpu_ticks": 264, + "cpu_time_ms": 2640 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 2.8, + "avg300": 1.39, + "avg60": 1.75 + }, + "some": { + "avg10": 3.02, + "avg300": 1.42, + "avg60": 1.83 + } + }, + "io_pressure_before": { + "full": { + "avg10": 2.38, + "avg300": 1.36, + "avg60": 1.63 + }, + "some": { + "avg10": 2.71, + "avg300": 1.39, + "avg60": 1.72 + } + }, + "root_device_read_delta_bytes": 0, + "root_device_write_delta_bytes": 385634304 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 212.977, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 86, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 55, + "p95_ttft_ms": 55, + "p99_elapsed_ms": 63, + "p99_ttft_ms": 63, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 232, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 117, + "p95_ttft_ms": 117, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 140, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 14, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 105, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 156, + "minimum_elapsed_ms": 9, + "model": "q008-messages", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 7, + "p95_elapsed_ms": 87, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-x59i9sxv/config.toml Server: 127.0.0.1:37095 Accounts: 1 Database: Content digest: fc2df0066d36e54853d7185f5c8a6c952a3a443929e7a3448241b813e888ee68", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 119, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 75, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2411856, \"ppid\": 2411834, \"uptime_seconds\": 2.899831611, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 14, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 472, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-x59i9sxv/backups/eggpool-backup-20261016-010343.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2411856)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 141, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-x59i9sxv/recovery/config.toml db: /tmp/eggpool-q008-x59i9sxv/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 48, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 55, + "id": "stop", + "reason": "Stopping server (PID 2413052)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 1.01, + "avg300": 1.3, + "avg60": 1.33 + }, + "some": { + "avg10": 1.11, + "avg300": 1.31, + "avg60": 1.36 + } + }, + "root_device_read_bytes": 723933310976, + "root_device_write_bytes": 1279228191744 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 8, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 8 + }, + { + "body_bytes": 309, + "elapsed_ms": 10, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 213, + "elapsed_ms": 9, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 145, + "elapsed_ms": 19, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 270, + "elapsed_ms": 20, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 480, + "elapsed_ms": 60, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21688320, + 21692416, + 23711744, + 23711744, + 23830528, + 23977984, + 24137728, + 46481408, + 46481408, + 46481408, + 23793664 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 8, + 10, + 9 + ], + "finite_ttft_ms": [ + 8, + 9, + 9 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 19, + 20, + 60 + ], + "streaming_ttft_ms": [ + 17, + 17, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21688320, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21688320, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21692416, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21692416, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23711744, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23711744, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172904448, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23711744, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23711744, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172904448, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23830528, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23830528, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172904448, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23977984, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23977984, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172904448, + "wal_bytes": 4206552, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24137728, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24137728, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173047808, + "wal_bytes": 4297192, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4648960, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46481408, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46481408, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173613056, + "wal_bytes": 4429032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4648960, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46481408, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46481408, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173613056, + "wal_bytes": 4429032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4648960, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46481408, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46481408, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173613056, + "wal_bytes": 4429032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23793664, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23793664, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103600128, + "wal_bytes": 885832, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 103, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c4-run-3.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c4-run-3.json new file mode 100644 index 00000000..2b2b0bc7 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c4-run-3.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "2ebd80a3ff1784e93849ca28063d00cb5b2e2301b239de6e402204b2adf56f02", + "archive_size_bytes": 4664791, + "elapsed_ms": 454, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 4, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23629824, + "baseline_rss_bytes": 23629824, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46600192, + "peak_rss_bytes": 46600192 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 5259, + "client_concurrency": 4, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 67.13, + "cpu_ticks": 353, + "cpu_time_ms": 3530 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 1.79, + "avg300": 1.37, + "avg60": 1.6 + }, + "some": { + "avg10": 1.83, + "avg300": 1.4, + "avg60": 1.66 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.97, + "avg300": 1.35, + "avg60": 1.48 + }, + "some": { + "avg10": 1.04, + "avg300": 1.37, + "avg60": 1.55 + } + }, + "root_device_read_delta_bytes": 16384, + "root_device_write_delta_bytes": 386105344 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 190.163, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 100, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 63, + "p95_ttft_ms": 63, + "p99_elapsed_ms": 76, + "p99_ttft_ms": 76, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 19, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 82, + "minimum_elapsed_ms": 3, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 4, + "model": "q008-messages", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-xpji5hl3/config.toml Server: 127.0.0.1:39583 Accounts: 1 Database: Content digest: 252d3a25760458d8eb905b5747c89d0390abe77bbd2ea5a63193f6ea53b5bfb9", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 91, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 40, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2413081, \"ppid\": 2413059, \"uptime_seconds\": 2.883297894, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 454, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-xpji5hl3/backups/eggpool-backup-20261016-010359.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 60, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2413081)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 143, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-xpji5hl3/recovery/config.toml db: /tmp/eggpool-q008-xpji5hl3/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 51, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2414267)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 1.78, + "avg300": 1.38, + "avg60": 1.64 + }, + "some": { + "avg10": 1.9, + "avg300": 1.4, + "avg60": 1.72 + } + }, + "root_device_read_bytes": 723933364224, + "root_device_write_bytes": 1279695565824 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 4, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 4 + }, + { + "body_bytes": 309, + "elapsed_ms": 4, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 4 + }, + { + "body_bytes": 213, + "elapsed_ms": 4, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 4 + }, + { + "body_bytes": 145, + "elapsed_ms": 17, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 14 + }, + { + "body_bytes": 270, + "elapsed_ms": 19, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 480, + "elapsed_ms": 60, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21860352, + 21864448, + 23629824, + 23629824, + 23748608, + 23887872, + 24051712, + 46600192, + 46600192, + 46600192, + 23744512 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 4, + 4, + 4 + ], + "finite_ttft_ms": [ + 4, + 4, + 4 + ], + "second_workload_elapsed_ms": [ + 10, + 2 + ], + "second_workload_ttft_ms": [ + 10, + 2 + ], + "streaming_elapsed_ms": [ + 17, + 19, + 60 + ], + "streaming_ttft_ms": [ + 14, + 17, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21860352, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21860352, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21864448, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21864448, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103567360, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23629824, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23629824, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172904448, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23629824, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23629824, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172904448, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23748608, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23748608, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172904448, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23887872, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23887872, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173039616, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24051712, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24051712, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173039616, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4640768, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46600192, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46600192, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173625344, + "wal_bytes": 4466112, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4640768, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46600192, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46600192, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173625344, + "wal_bytes": 4466112, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4640768, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46600192, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46600192, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173625344, + "wal_bytes": 4466112, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23744512, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23744512, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103600128, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c8-run-1.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c8-run-1.json new file mode 100644 index 00000000..85753f80 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c8-run-1.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "27d5d704658f0715df17712392e0a1a1c99491cf2e759e472b6f56b127f23b9f", + "archive_size_bytes": 4709847, + "elapsed_ms": 445, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 8, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23748608, + "baseline_rss_bytes": 23748608, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46989312, + "peak_rss_bytes": 46989312 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 2960, + "client_concurrency": 8, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 66.56, + "cpu_ticks": 197, + "cpu_time_ms": 1970 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 1.94, + "avg300": 1.37, + "avg60": 1.55 + }, + "some": { + "avg10": 1.94, + "avg300": 1.39, + "avg60": 1.6 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.78, + "avg300": 1.33, + "avg60": 1.38 + }, + "some": { + "avg10": 0.79, + "avg300": 1.36, + "avg60": 1.44 + } + }, + "root_device_read_delta_bytes": 4096, + "root_device_write_delta_bytes": 389353472 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 337.843, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 102, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 17, + "p50_ttft_ms": 17, + "p95_elapsed_ms": 69, + "p95_ttft_ms": 69, + "p99_elapsed_ms": 76, + "p99_ttft_ms": 76, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 3, + "model": "q008-responses", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 19, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 80, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 9, + "p50_ttft_ms": 7, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 54, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 5, + "model": "q008-messages", + "p50_elapsed_ms": 14, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-qin89aoi/config.toml Server: 127.0.0.1:52015 Accounts: 1 Database: Content digest: 8ddc9847c9e5187966e53e353613417ba0a1e822cd41839adab21b2a9b98d76b", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 95, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 38, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 3, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 7, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 9, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 11, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 9, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2414297, \"ppid\": 2414275, \"uptime_seconds\": 2.927291012, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 445, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-qin89aoi/backups/eggpool-backup-20261016-010414.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 64, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2414297)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 98, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-qin89aoi/recovery/config.toml db: /tmp/eggpool-q008-qin89aoi/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 49, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop", + "reason": "Stopping server (PID 2415494)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 1.2, + "avg300": 1.36, + "avg60": 1.5 + }, + "some": { + "avg10": 1.22, + "avg300": 1.38, + "avg60": 1.55 + } + }, + "root_device_read_bytes": 723933409280, + "root_device_write_bytes": 1280160850944 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 11, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 11 + }, + { + "body_bytes": 309, + "elapsed_ms": 9, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 213, + "elapsed_ms": 9, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 145, + "elapsed_ms": 19, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 270, + "elapsed_ms": 20, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 480, + "elapsed_ms": 57, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 56 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21852160, + 21856256, + 23748608, + 23748608, + 23867392, + 23990272, + 24170496, + 46989312, + 46989312, + 47054848, + 23785472 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 11, + 9, + 9 + ], + "finite_ttft_ms": [ + 11, + 9, + 9 + ], + "second_workload_elapsed_ms": [ + 3, + 2 + ], + "second_workload_ttft_ms": [ + 3, + 2 + ], + "streaming_elapsed_ms": [ + 19, + 20, + 57 + ], + "streaming_ttft_ms": [ + 17, + 17, + 56 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21852160, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21852160, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21856256, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21856256, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23748608, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23748608, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23748608, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23748608, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23867392, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23867392, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23990272, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23990272, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172908544, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24170496, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24170496, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173047808, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4689920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46989312, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46989312, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174026752, + "wal_bytes": 4334272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4689920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46989312, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46989312, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174026752, + "wal_bytes": 4334272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4689920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 47054848, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47054848, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174026752, + "wal_bytes": 4334272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23785472, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23785472, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103591936, + "wal_bytes": 885832, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c8-run-2.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c8-run-2.json new file mode 100644 index 00000000..de7f39ab --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c8-run-2.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "bb29ca594d57d30d343db70d013b84b9923b325dfae2d9599388491a42f6582d", + "archive_size_bytes": 4685271, + "elapsed_ms": 625, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 8, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23793664, + "baseline_rss_bytes": 23793664, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 47173632, + "peak_rss_bytes": 47173632 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 12378, + "client_concurrency": 8, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 43.3, + "cpu_ticks": 536, + "cpu_time_ms": 5360 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 13.96, + "avg300": 2.08, + "avg60": 4.69 + }, + "some": { + "avg10": 13.96, + "avg300": 2.1, + "avg60": 4.72 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.95, + "avg300": 1.35, + "avg60": 1.41 + }, + "some": { + "avg10": 0.95, + "avg300": 1.37, + "avg60": 1.45 + } + }, + "root_device_read_delta_bytes": 53248, + "root_device_write_delta_bytes": 390234112 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 80.791, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 1088, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 115, + "p50_ttft_ms": 115, + "p95_elapsed_ms": 214, + "p95_ttft_ms": 214, + "p99_elapsed_ms": 281, + "p99_ttft_ms": 281, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 19, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 84, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 8, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-fcvw8s6g/config.toml Server: 127.0.0.1:33869 Accounts: 1 Database: Content digest: b3518997b55ce483b9d3025f9a869a97d4533c93d00c98094ca7e25ccd0be3e8", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 93, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 11, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 9, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2415525, \"ppid\": 2415503, \"uptime_seconds\": 2.887722885, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 625, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-fcvw8s6g/backups/eggpool-backup-20261016-010438.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2415525)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 501, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-fcvw8s6g/recovery/config.toml db: /tmp/eggpool-q008-fcvw8s6g/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 200, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 57, + "id": "stop", + "reason": "Stopping server (PID 2416725)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 1.89, + "avg300": 1.38, + "avg60": 1.58 + }, + "some": { + "avg10": 1.9, + "avg300": 1.4, + "avg60": 1.63 + } + }, + "root_device_read_bytes": 723933560832, + "root_device_write_bytes": 1280629126144 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21966848, + 21970944, + 23793664, + 23793664, + 23920640, + 24059904, + 24289280, + 47173632, + 47173632, + 47173632, + 23834624 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 9, + 10 + ], + "second_workload_ttft_ms": [ + 9, + 10 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 56 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21966848, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21966848, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21970944, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21970944, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23793664, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23793664, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23793664, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23793664, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23920640, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23920640, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24059904, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24059904, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173051904, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24289280, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24289280, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173051904, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4681728, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 47173632, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47173632, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174022656, + "wal_bytes": 4387832, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4681728, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 47173632, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47173632, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174022656, + "wal_bytes": 4387832, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4681728, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 47173632, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47173632, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174022656, + "wal_bytes": 4387832, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23834624, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23834624, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103600128, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 104, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c8-run-3.json b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c8-run-3.json new file mode 100644 index 00000000..a8b94e91 --- /dev/null +++ b/artifacts/qualification/on-prem-concurrency-performance/ordinary-server/u5-c8-run-3.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "4ad4826de49f35d093ffe33de9be8351d475f96921b7a3c0b7a5d50f13935275", + "archive_size_bytes": 4705751, + "elapsed_ms": 627, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 8, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23674880, + "baseline_rss_bytes": 23674880, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46985216, + "peak_rss_bytes": 46985216 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 2983, + "client_concurrency": 8, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 67.39, + "cpu_ticks": 201, + "cpu_time_ms": 2010 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 5.7, + "avg300": 2.13, + "avg60": 4.36 + }, + "some": { + "avg10": 5.7, + "avg300": 2.15, + "avg60": 4.38 + } + }, + "io_pressure_before": { + "full": { + "avg10": 5.86, + "avg300": 2.11, + "avg60": 4.33 + }, + "some": { + "avg10": 5.86, + "avg300": 2.13, + "avg60": 4.36 + } + }, + "root_device_read_delta_bytes": 24576, + "root_device_write_delta_bytes": 391208960 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 335.268, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 87, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 19, + "p50_ttft_ms": 19, + "p95_elapsed_ms": 68, + "p95_ttft_ms": 68, + "p99_elapsed_ms": 73, + "p99_ttft_ms": 73, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 144, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 19, + "p95_elapsed_ms": 90, + "p95_ttft_ms": 90, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 141, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 105, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 9, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 38939984, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-wk3mue3w/config.toml Server: 127.0.0.1:33075 Accounts: 1 Database: Content digest: 769cdabec6e5e9e0df0a11d80998d0271e2b4875ac48c3093ec8149f1e65924b", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 123, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 62, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2416756, \"ppid\": 2416734, \"uptime_seconds\": 2.871153753, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 627, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-wk3mue3w/backups/eggpool-backup-20261016-010453.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2416756)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 502, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-wk3mue3w/recovery/config.toml db: /tmp/eggpool-q008-wk3mue3w/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 227, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 107, + "id": "stop", + "reason": "Stopping server (PID 2417977)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 10.02, + "avg300": 2.14, + "avg60": 4.72 + }, + "some": { + "avg10": 10.02, + "avg300": 2.16, + "avg60": 4.75 + } + }, + "root_device_read_bytes": 723933659136, + "root_device_write_bytes": 1281095898112 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 11 + }, + { + "body_bytes": 270, + "elapsed_ms": 15, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 13 + }, + { + "body_bytes": 480, + "elapsed_ms": 59, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21721088, + 21721088, + 23674880, + 23674880, + 23805952, + 23953408, + 24109056, + 46985216, + 46985216, + 46985216, + 23740416 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 11 + ], + "streaming_elapsed_ms": [ + 12, + 15, + 59 + ], + "streaming_ttft_ms": [ + 11, + 13, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21721088, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21721088, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21721088, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21721088, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103563264, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23674880, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23674880, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23674880, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23674880, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23805952, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23805952, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23953408, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23953408, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172912640, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24109056, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24109056, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173051904, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4702208, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46985216, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46985216, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174039040, + "wal_bytes": 4326032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4702208, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46985216, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46985216, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174039040, + "wal_bytes": 4326032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4702208, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46985216, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46985216, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174039040, + "wal_bytes": 4326032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23740416, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23740416, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103600128, + "wal_bytes": 894072, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 103, + "status": "pass" +} diff --git a/docs/deployment.md b/docs/deployment.md index eafe8f9d..832b4f14 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -520,7 +520,7 @@ model payloads, login rate, and policy/budget profile before deployment. The Pi microSD envelope remains a separate solo-device reference in [`raspberry-pi.md`](raspberry-pi.md). See the [M002 closure record](../plans/closure/on-prem-concurrency-performance/002-status.md) -for its measured/unmeasured status and required follow-up evidence. +for its evidence, scope and interpretation limits. To repeat the frozen corpus from a release binary, run `uv run python scripts/qualification_ordinary_server.py --binary `. @@ -529,8 +529,9 @@ root for each run, requires three repetitions per condition, validates p99 sample counts and settlement, derives `summary.json` from the raw Q008 reports, and runs deterministic Q009 provider-fault, hard-budget, user-disable, and abrupt-restart phases against the same binary. The summary requires the -matching `lifecycle.json` artifact. It refuses non-Linux hosts and non-durable or rotational qualification -storage, and stops before load when initial full I/O pressure exceeds 25% over +matching `lifecycle.json` artifact. It refuses non-Linux hosts and +non-durable or rotational qualification storage, and stops before load when +initial full I/O pressure exceeds 25% over the 10-second window. Qualification does not send requests to a real provider or charge a provider account. diff --git a/plans/closure/on-prem-concurrency-performance/002-status.md b/plans/closure/on-prem-concurrency-performance/002-status.md index ef0eff5a..9297c2d0 100644 --- a/plans/closure/on-prem-concurrency-performance/002-status.md +++ b/plans/closure/on-prem-concurrency-performance/002-status.md @@ -1,6 +1,6 @@ # On-Prem Concurrency Performance M002 — Closure Status -Status: conditionally closed — ordinary-server tooling and workload design are implemented; the isolated host run and capacity envelope remain unmeasured +Status: closed Source implementation plan: @@ -14,51 +14,91 @@ Repository baseline reviewed: `cdfb6fa8d4aaa1d9d5b43af8c87eaac68f83c402` Implementation commits or pull requests: -- `cdfb6fa8d4aaa1d9d5b43af8c87eaac68f83c402` — ordinary Linux host attestation, safe SSD/NVMe gates, deterministic upstream latency profile, bounded host I/O observations, 16-client support, frozen 27-run driver, summary validation, and deployment/observability guidance. No runtime change. +- `cdfb6fa8d4aaa1d9d5b43af8c87eaac68f83c402` — ordinary-host gates, frozen matrix driver, summary validation, and first deployment guidance. +- `f7343eb295ef3bb9e72b43211980934fe2ca2b45` — committed managed lifecycle and protected SQLite-diagnostic tooling; measured deployment guidance and observability updates. +- Closure/status pass — completed 27-run host matrix, bounded competing-I/O and SQLite publication diagnostics, and formal plan/registry updates. ## 1. Executive finding -The tooling/setup slice is complete. `scripts/qualification_ordinary_server.py` freezes the manifest before collection, schedules nine principal/concurrency conditions with three fresh-root runs each, and derives an aggregate report only after all raw runs pass the same build, storage, p99-sample, and settlement gates. `scripts/qualification_sbc.py` retains the SBC default and adds ordinary-host metadata, persistent database-storage checks, the 16-client tier, a deterministic 0/10/50 ms upstream profile, and host I/O observations. +M002 is complete. The locked release candidate was measured on a bare-metal +Ubuntu 24.04.5 host with ext4 on Samsung SSD 980 NVMe. All 27 fresh-root +Q008 runs across nine cells passed; the same candidate passed managed Q009 +fault, hard-budget, user-disable, and abrupt-restart checks. A bounded +same-filesystem competing-writer run and an opt-in SQLite gate diagnostic add +contention and storage-phase evidence. The deployment guide now reports a +measured host-specific reference and explicitly rejects an SLA interpretation. -No capacity run was started and no release candidate was built. The available host is a bare-metal Ubuntu 24.04.5 x86_64 workstation with durable ext4/NVMe, but it is shared with long-running unrelated CPU work and is not an explicitly isolated server qualification target. During the required Rust suite build, Linux I/O pressure exceeded the predeclared ordinary-runner stop threshold of 25% full `avg10` by a wide margin. Both suite attempts were stopped. No timing or throughput result is inferred from the host or from M001's separate exploratory NVMe cell. The team capacity envelope, SQLite gate tails, active-load cancellation/restart behavior, and policy-denial rates remain unmeasured. - -Disposition is conditional under the plan's stop condition. Do not claim an ordinary-server throughput, concurrency, or latency envelope from this closure. +The result describes this synthetic loopback workload on one host. It does not +promise throughput for other hardware, providers, models, request shapes, or +policy settings. No production runtime, schema, database topology, or policy +behavior changed. ## 2. Requirement-to-evidence matrix | Requirement | Evidence | Result | Notes | |---|---|---|---| -| Freeze named-account and concurrent-request levels separately | `scripts/qualification_ordinary_server.py`, `tests/tooling/test_qualification_ordinary_server.py` | pass | Five account levels (1/5/10/25/49 at concurrency 1) and five concurrency levels (1/2/4/8/16 at five principals) form nine unique conditions and 27 planned runs. | -| Freeze workload, candidate identity, and gates before collection | M002 driver and summary contract tests | pass | The driver writes its matrix before invoking any condition; each condition is 1,000 finite requests, three repetitions, hard user/global budgets, and the fixed synthetic 0/10/50 ms upstream profile. | -| Record host/build/storage facts and require durable SSD/NVMe | `ordinary_server_metadata`, Q008 report schema, durable-path guard | partial | Host facts are implemented. The host was attested as Ubuntu 24.04.5, Linux 6.8.0-147-generic, Intel Core i9-9900K, 16 logical CPUs, 15 GiB RAM, Gigabyte Z390 AORUS MASTER, ext4 on Samsung SSD 980 1TB NVMe, bare metal (`systemd-detect-virt: none`, container: none). No release candidate was built and no raw run was produced. | -| Run three fresh-root durable-server repetitions per condition and collect latency/resource metrics | Frozen driver, full artifacts absent | not run | No ordinary-server experiment ran. CPU, RSS, throughput, TTFT, I/O, WAL, gate, and per-condition latency values are not reported. | -| Require adequate p99 observations and converged durable state | Q008 runner plus summary guard tests | pass — tooling contract only | The driver requires 1,000 completed observations, a p99 value, zero request failures, zero pending requests, and zero active reservations in all three raw runs before summarizing. No observed run passed or failed these gates because no run started. | -| Exercise failure, cancellation, active-user restart, mixed policy denies, and token revocation | Existing Q008 functional lifecycle coverage; no M002 raw run | not run | The new matrix measures successful budgeted inference. It does not yet drive cancellation/restart during active load or count provider, identity, and budget 429s separately. No behavior is claimed. | -| Publish an honest ordinary-server deployment envelope and compare with Pi solo evidence | `docs/deployment.md`, `docs/raspberry-pi.md`, this record | partial | The docs separate named users from simultaneous requests and state that there is no M002 capacity result. The immutable Pi M001/M012 evidence is unchanged. No sizing range is published. | -| Keep runtime, persistence, and user-policy invariants unchanged | Changed-file review | pass | No Rust, schema, admission, quota, auth, provider transport, SQLite journal, worker, or checkpoint policy changes. | +| Separate named accounts from simultaneous requests | `matrix.json`, 27 Q008 reports | pass | Principal cells 1/5/10/25/49 at concurrency 1; concurrency cells 1/2/4/8/16 at five principals; 1,000 requests and three fresh roots per cell. | +| Freeze candidate, workload and gates before collection | `matrix.json`; summary guards in `tests/tooling/test_qualification_ordinary_server.py` | pass | Candidate SHA-256 `43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3`, source commit `1c3d15b8911f1f0f3e003c9e918282c0363736e3`; maximum full I/O PSI avg10 before/after any of the 27 raw runs was 13.96%, below the 25% stop gate. | +| Record durable ordinary-host measurements | `summary.json`, `u*-c*-run-*.json` | pass | Ubuntu 24.04.5, kernel 6.8.0-147-generic, Intel i9-9900K, 16 logical CPUs, 15.5 GiB RAM, bare metal, ext4, Samsung SSD 980 1TB NVMe; WAL/NORMAL and one worker. | +| Require p99 sample size and settled durable state | 27 raw Q008 reports and generated `summary.json` | pass | Each run completed 1,000 concurrent requests with p99; no HTTP failures, pending requests, or active reservations. Three repetitions per each of nine cells. | +| Exercise failure, cancellation, retry/partial response, budgets, disable and restart | `lifecycle.json`; raw-to-summary guards | pass | Provider 408/429/503/timeout, partial/disconnect streams, client cancel, alternate wire, malformed response, absent model and connect failure. Provider, user-budget and global-budget 429s are counted independently (one each). User disable revoked a second synthetic member's credential during an in-flight request; the next request received 401 before provider dispatch. Abrupt restart did not replay unknown work; recovery converged to zero pending requests/reservations. | +| Exercise disk contention and record SQLite gate phases | `disk-contention.json`; `sqlite-gate-diagnostic-session.json` | pass | The 1,000-request four-way run used a bounded same-filesystem writer capped at 4 MiB/s and 32 MiB; it wrote 23.2 MiB, completed all requests, and settled all reservations. Its p95/p99 were 85/125 ms, and full I/O PSI avg10 after was 6.06%. The separate 60-request feature-build publication diagnostic measured gate wait p50/p95/max at 0/1/1 μs for both publication and finalization. These diagnostic samples are supplementary, not per-cell or universal claims. | +| Publish an honest measured envelope and retain Pi separation | `docs/deployment.md`, `docs/raspberry-pi.md`, this record | pass | Nine-cell latency/throughput/resource summaries are documented for this host and synthetic profile; the Pi microSD solo reference remains separate and unchanged. | +| Preserve runtime, persistence, policy and compatibility invariants | changed-file review; focused Rust and tooling tests | pass | No runtime or schema change. WAL/NORMAL, the single SQLite gate/worker, and admission/budget semantics are unchanged. | + +### Measured matrix summary + +Median throughput and latency across three repetitions per cell: + +| Named principals | Concurrency | Throughput (req/s) | p95 (ms) | p99 (ms) | Max observed (ms) | Peak RSS (MiB) | +|---:|---:|---:|---:|---:|---:|---:| +| 1 | 1 | 36.5 | 82 | 148 | 232 | 24.9 | +| 5 | 1 | 42.2 | 61 | 89 | 243 | 44.2 | +| 5 | 2 | 94.0 | 61 | 79 | 96 | 44.3 | +| 5 | 4 | 193.7 | 59 | 75 | 100 | 44.6 | +| 5 | 8 | 335.3 | 69 | 76 | 1,088 | 45.0 | +| 5 | 16 | 341.3 | 88 | 106 | 1,119 | 45.8 | +| 10 | 1 | 42.9 | 61 | 77 | 160 | 44.2 | +| 25 | 1 | 42.5 | 61 | 81 | 228 | 44.6 | +| 49 | 1 | 42.3 | 61 | 81 | 195 | 45.1 | + +The 8/16 concurrency maxima show occasional long tails despite moderate p95/p99; +they are retained rather than hidden by medians. This is a descriptive result, +not a latency threshold or service guarantee. ## 3. Production implementation evidence -No production behavior changed. M002 tooling adds ordinary-host inventory without exposing mount source paths or process command lines; root/database storage class and filesystem are bounded facts. The benchmark runner rejects tmpfs/ramfs/overlay and rotational storage, and standalone ordinary-server benchmark invocations enforce the same preflight I/O-pressure gate as the matrix driver. The deterministic latency profile cycles through 0, 0, 0, 10, 10, and 50 ms per synthetic upstream request. All provider traffic remains loopback-only. - -The frozen matrix contains 27 runs across nine unique cells. It uses a 1,000-request concurrent batch, three repetitions, 30 sequential samples per surface, native finite, native streaming, and translated streaming cases, synthetic managed principals, and hard user/global request budgets. No raw M002 report, summary, or capacity number exists because preflight did not proceed to collection. +No production behavior changed. The ordinary-server qualification profile +preserves the SBC runner's default and records sanitized host/build/storage +facts. The matrix uses a fixed 0/10/50 ms loopback provider profile, native +finite, native streaming, and translated streaming samples, hard user/global +request budgets, and fresh roots. The optional diagnostic build adds only +feature-gated scalar SQLite observations; it is not the measured release +candidate and did not alter the product binary. + +The committed artifact corpus is in +`artifacts/qualification/on-prem-concurrency-performance/ordinary-server/`. +It includes all 27 passing raw reports, `matrix.json`, generated `summary.json`, +`lifecycle.json`, the controlled `disk-contention.json`, the supplementary +SQLite gate report, and preserved failed attempts. A secret-pattern scan found +no credential, token, prompt, authorization, or cookie material in the corpus. ## 4. Verification executed ### Commands run ```bash -rtk cargo fmt --manifest-path rust/Cargo.toml --all -- --check -rtk cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings -rtk cargo check --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -rtk cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- -D warnings -rtk cargo test --manifest-path rust/Cargo.toml --workspace --all-targets -- --test-threads=1 -rtk proxy bash -c 'CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --workspace --all-targets -- --test-threads=1' -rtk uv sync --frozen -rtk uv run ruff format --check scripts/ tests/tooling/ +rtk cargo build --manifest-path rust/Cargo.toml --locked --release +rtk uv run python scripts/qualification_ordinary_server.py --binary rust/target/release/eggpool --output artifacts/qualification/on-prem-concurrency-performance/ordinary-server +rtk uv run python scripts/qualification_sbc.py --binary rust/target/release/eggpool --target-class ordinary-server --config-fixture tests/tooling/fixtures/qualification/sbc-benchmark.toml --benchmark-samples 30 --concurrency-workers 4 --concurrency-requests 1000 --benchmark-principals 5 --benchmark-budget-mode user-global --provider-latency-profile mixed-0-10-50ms --competing-io-mib-per-second 4 --competing-io-max-mib 32 --output artifacts/qualification/on-prem-concurrency-performance/ordinary-server/disk-contention.json +rtk proxy bash -c 'CARGO_BUILD_JOBS=1 cargo build --manifest-path rust/Cargo.toml --locked --release --features qualification-db-diagnostics' +rtk uv run python scripts/qualification_sbc.py --binary rust/target/release/eggpool --target-class ordinary-server --config-fixture tests/tooling/fixtures/qualification/sbc-benchmark.toml --diagnose-publication-phases --output artifacts/qualification/on-prem-concurrency-performance/ordinary-server/sqlite-gate-diagnostic-session.json +rtk proxy bash -c 'CARGO_BUILD_JOBS=2 cargo test --manifest-path rust/Cargo.toml --test on_prem_budgets --test on_prem_global_budgets --test on_prem_attribution -- --test-threads=1' +rtk uv run pytest tests/tooling/test_qualification_sbc.py tests/tooling/test_qualification_ordinary_server.py tests/tooling/test_on_prem_mmc_tail_attribution.py tests/tooling/test_qualification_stability.py -q --tb=short --maxfail=1 +rtk uv run pytest tests/tooling/ -q --tb=short --maxfail=1 rtk uv run ruff check scripts/ tests/tooling/ +rtk uv run ruff format --check scripts/ tests/tooling/ rtk uv run pyright scripts/ -rtk uv run pytest tests/tooling/test_qualification_sbc.py tests/tooling/test_on_prem_mmc_tail_attribution.py tests/tooling/test_qualification_ordinary_server.py -q rtk uv run python scripts/validate_release_docs.py rtk git diff --check ``` @@ -67,54 +107,101 @@ rtk git diff --check | Command | Result | |---|---| -| Rust fmt, default Clippy, no-default check, and no-default Clippy | pass | -| Workspace serial Rust tests, default parallel build | interrupted before test results; observed I/O PSI reached `some avg10=94.05%`, `full avg10=81.43%`, `full avg60=46.12%` during compile/link | -| Workspace serial Rust tests with one Cargo build job | interrupted before test results; observed I/O PSI reached `full avg10=43.31%`, `full avg60=42.09%` during compile/link | -| `uv sync --frozen` | pass; generated a local `requires-python` lockfile normalization, which was reverted because this task changes no Python dependencies | -| Ruff format/check and Pyright | pass; 65 files formatted, no Ruff findings, Pyright 0 errors | -| Focused Q008, M001 artifact, and M002 tooling tests | pass; 51 tests | -| Release-doc validator | pass; seven docs checked | -| Release build and ordinary-server matrix | not run; shared-host I/O pressure and lack of an isolated, provisioned server target triggered the plan stop condition | -| Hosted CI | not run from this local closure pass | +| Locked release build | pass; measured binary SHA-256 is recorded above. | +| Ordinary-server matrix and managed lifecycle | pass; 27/27 Q008 runs and both Q009 phases; generated summary validated all reports against the frozen candidate. | +| Bounded same-filesystem contention run | pass; 1,000/1,000 requests completed, zero HTTP failures, zero pending requests and active reservations. | +| Feature-gated SQLite phase diagnostic | pass; 60/60 requests; runtime state converged; gate timing reported above. | +| Rust budget/attribution targets | pass; 48 tests total (29 user budget, 16 global budget, 3 attribution), serial. | +| Focused qualification/tooling pytest | pass; 57 tests. | +| Ruff, Pyright, release-doc validator | pass; no Ruff findings, Pyright 0 errors, release-doc validator passed. | +| Full workspace Rust suite | not rerun; no Rust runtime/schema source changed. The directly relevant user/global budget and attribution targets passed here; integrated M022 closure `plans/closure/on-prem-user-access/022-status.md` records 1,039 default-feature tests and 1,040 no-default-feature tests passing (one ignored in each run). | +| Full tooling suite | pass; 197 passed, 3 skipped. The final exact-commit summary guard also passed its focused five-test file. | +| Hosted CI | not run in this local closure pass. | + +`uv run` normalizes `uv.lock`'s `requires-python` field in this environment; +that incidental change was reverted and is not part of this work. ## 5. Invariant review -- The runner uses a disposable root, loopback-only provider, and synthetic identities. No customer data, real provider key, prompt, or production database is used. -- Named principal count is reported independently from in-flight request concurrency. The plan makes no 49- or 50-concurrent service claim. -- The qualification database stays on durable non-rotational SSD/NVMe for ordinary-server measurements; tmpfs is rejected. -- No database topology, WAL/NORMAL, one-worker, routing, quota, admission, or budget semantics changed. -- Existing M001 and M012 physical artifacts and Pi guidance remain intact. +- All inference traffic and controlled contention stayed on loopback and a + disposable root. No real provider, external billing, production database, or + customer identity was used. +- Principal count and active concurrency remain separate in the manifest and + report. No 49-user simultaneous-inference claim is made. +- Ordinary-server measurements require durable non-rotational SSD/NVMe; + temporary and rotational storage are rejected. The controlled writer has + explicit rate/volume caps and remained below the frozen I/O stop gate. +- No storage topology, WAL/checkpoint policy, SQLite worker/gate, admission, + routing, provider quota, user budget, or migration behavior changed. +- Pi M001/M012 artifacts and the Pi solo envelope were not rewritten. ## 6. Failure and recovery review -The Q008 runner still checks isolated backup/recovery and graceful stop paths, and the matrix summary requires all requests to finish with no pending rows or active reservations. M002 did not exercise server shutdown or cancellation during active inference, retries/partial upstream failure, identity disable/revocation during load, or the separate observed 429 categories. These remain unverified and must be included before using the matrix as team-operation evidence. No active reservation or failure count is fabricated. - -The environment stop was safe: the Rust test build processes were interrupted, no qualification process or synthetic request batch was started, and no performance artifact was emitted. +The lifecycle corpus covers client cancellation, partial/disconnected streams, +provider failure classes, budget denial before dispatch, user disable during an +in-flight request, abrupt process termination, and restart reconciliation. The +disabled credential was rejected with HTTP 401 and did not add a provider +request. Restart did not replay unknown in-flight provider work; durable state +settled with zero pending requests and reservations. + +Failed attempts were retained: the initial Q008 run exposed insufficient +post-run budget headroom; the first lifecycle attempt used retired shared-key +inference; two initial SQLite diagnostic attempts exposed protected runtime +diagnostic access/quiescence assumptions. The qualification tools were +corrected to use a managed synthetic member for disable, an ephemeral browser +session for protected runtime diagnostics, and a larger budget headroom. The +final reruns passed; failed reports remain in the artifact directory. ## 7. Migration and compatibility review -No schema, migration, configuration, endpoint, CLI, or runtime compatibility change was made. The SBC runner's default target and schema remain unchanged; ordinary-server qualification is an explicit target profile. +No schema, migration, configuration, endpoint, CLI, or runtime compatibility +change was made. The Q008 SBC default remains unchanged. Ordinary-server +qualification is an explicit target profile. Diagnostic browser-session +material is held only in process memory and is never serialized. ## 8. Security review -The tools retain bounded aggregate host, timing, and storage-class facts. They do not serialize request bodies, hostnames, addresses, credentials, tokens, usernames, or environment values. Existing redaction guards and the new raw-to-summary tests pass. There are no M002 raw artifacts to audit. +The raw-to-summary tooling retains only bounded synthetic scenario labels, +scalar timings, host classes and aggregate state. Fixed synthetic account +labels may appear in CLI fixture records; no real usernames or customer +identities are used. Tokens, passwords, browser cookies, request/response +bodies, authorization headers, hostnames and environment values are not +written to reports. The synthetic member token and browser cookie stay in +memory. The committed artifact corpus passed the secret-pattern content audit. ## 9. Documentation and operations -`docs/deployment.md` documents the frozen workload, repeat command, safe storage requirements, preflight pressure stop, and no-SLA boundary. `docs/raspberry-pi.md` keeps the SBC solo envelope separate. `architecture/deep-dive-observability.md` distinguishes qualification artifacts from runtime telemetry. The release-documentation validator passes. +`docs/deployment.md` reports the measured host envelope, workload, p95/p99 and +maximum observations, and repeat command with an explicit no-SLA boundary. +`docs/raspberry-pi.md` links to the separate server evidence without changing +the Pi results. `architecture/deep-dive-observability.md` distinguishes +qualification artifacts and supplementary gate timing from runtime telemetry. +Release documentation validation passed. ## 10. Unresolved findings | Severity | Finding | Impact | Required action | |---|---|---|---| -| medium | No isolated ordinary-server run, release candidate identity, or capacity observations were collected | There is no ordinary-server team sizing envelope; downstream work cannot cite M002 as measured capacity | On an explicitly provisioned, isolated Linux host, build the locked release candidate, retain its exact SHA and host/storage attestation, pass the 25% full-I/O-PSI preflight, and complete all 27 fresh-root runs. Preserve all failures and calculate the report from raw reports. | -| medium | Active-load cancellation/restart, retries/partial provider failures, token revocation, policy denies, and classified 429s are absent from the matrix execution | Budget recovery and failure behavior under simultaneous team load remain unknown | Extend the bounded synthetic scenarios and their sanitized guard before using M002 as operational readiness evidence; exercise and report provider, identity, and budget responses separately. | -| low | The full Rust workspace suite did not complete because both compile attempts crossed the I/O stop gate | This tooling-only change has focused Python coverage and compile/lint gates, but no serial workspace test result | Re-run the exact serial workspace suite on a stable, low-pressure host. | +| low | The capacity envelope comes from one host and one deterministic synthetic provider profile; the competing-I/O and SQLite-gate diagnostics are supplementary single runs | Other hardware, provider latency, request size, background load and policy can produce different tails; the optional phase diagnostic is not per-cell p99 evidence | Treat the published numbers as a relative reference only. Run the same frozen corpus on any materially different deployment class before making sizing or SLA claims. | +| low | Hosted CI was not run in this local closure pass | This documentation/tooling/evidence closure has local verification only | Standard CI remains available for the next code-bearing change; no runtime release depends on M002. | + +No high- or medium-severity M002 finding remains. The plan's acceptance +criteria are met and its stop conditions were not triggered during collection. ## 11. Roadmap disposition -M002 is **conditionally closed for its tooling/setup slice only**. The ordinary-server measurements and deployment envelope are explicitly unmeasured. A future isolated hardware pass must provide the raw corpus and failure/recovery evidence above before any capacity recommendation is made. The Pi microSD solo benchmark remains a distinct reference and does not satisfy or block M002. +M002 is closed. The result supplies the ordinary-server operational evidence +required by on-prem user-access M023 and supports host-specific relative sizing. +It authorizes no production performance change. M001 remains conditionally +closed on its own Pi/microSD attribution conditions; M002 does not reopen or +resolve them. ## 12. Registry updates -The implementation plan moves to `implemented` with this conditional closure; the roadmap remains `active` because no future production optimization is registered and M002's operational evidence is outstanding. The dependency-ready M002 row is removed and the conditional closure is recorded. Unblock audit: searched the on-prem implementation plans, both relevant roadmap dependency sections, and registry ready/active/blocked tables. No future plan declares M002 as a hard, interface, or soft dependency. On-prem user-access M023 names ordinary-server evidence as an operational requirement but remains blocked on its hard M022 dependency; M002 has not supplied that operational evidence, so M023 is not promoted and no other plan is unblocked. +The implementation plan remains `implemented`; the closure record and roadmap +mark M002 `closed`. Unblock audit searched `plans/implementation/`, both +on-prem roadmap dependency sections, and the ready/active/blocked registry +tables. M023's hard M022 dependency is closed and its operational M002 +requirement is satisfied, so M023 moves `blocked` to `ready`. No other future +plan declares M002 as a hard, interface, or soft dependency; nothing else is +promoted or blocked. diff --git a/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md b/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md index cfaf97fa..1823cc3c 100644 --- a/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md +++ b/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md @@ -1,6 +1,6 @@ # On-Prem User Access M023 — Short-window team fairness and admission controls -Status: blocked +Status: ready Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). @@ -18,11 +18,11 @@ Prevent one user's bursty coding agents from monopolizing shared provider capaci ## 2. Dependencies -Hard: M022 client/budget interoperability closure. Interface: existing principal authentication, M008/M009 durable reservations and `coordinator/endpoints.rs` admission order. Operational: qualify on representative ordinary on-prem Linux server; Pi/microSD is a solo low-end reference, not the team load-release gate. +Hard: M022 client/budget interoperability closure (closed). Interface: existing principal authentication, M008/M009 durable reservations and `coordinator/endpoints.rs` admission order. Operational: satisfied by the measured ordinary-server baseline in `plans/closure/on-prem-concurrency-performance/002-status.md`; Pi/microSD is a solo low-end reference, not the team load-release gate. ## 3. Current evidence -`rust/src/user_budget.rs` and `rust/src/instance_budget.rs` already handle daily/monthly and max-concurrent rules transactionally. `routing/` owns provider-account quota fairness, not downstream-user fairness. An authorized user's burst may exhaust provider throughput or dominate latency without violating day/month budgets. Existing single-gate SQLite performance, particularly Pi microSD write tails, argues against persisting per-request rolling-window rows by default. +`rust/src/user_budget.rs` and `rust/src/instance_budget.rs` already handle daily/monthly and max-concurrent rules transactionally. `routing/` owns provider-account quota fairness, not downstream-user fairness. An authorized user's burst may exhaust provider throughput or dominate latency without violating day/month budgets. The ordinary-server M002 baseline supplies a host-specific shared-team reference, while its bounded SQLite publication diagnostic recorded low gate wait on that target. Existing single-gate SQLite performance, particularly Pi microSD write tails, argues against persisting per-request rolling-window rows by default. ## 4. Invariants diff --git a/plans/registry.md b/plans/registry.md index 75e9f902..a4d663fe 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -58,14 +58,15 @@ No new runtime, schema or performance claim lands with these plans. | Persistence | active | `plans/subsystems/persistence-roadmap.md` | M011 ready — post-M010 roadmap/documentation reconciliation | M009/M010 closures accepted. Documentation/tooling-only corrective; production remains WAL/NORMAL. No technical persistence successor is registered. | | Deployment and packaging | closed | `plans/subsystems/deployment-packaging-roadmap.md` | M005 closed — active PATH integration detection corrective | M001–M005 closed; no successor registered. | | Dashboard | closed | `plans/subsystems/dashboard-roadmap.md` | M012 closed — lifecycle closure and documentation polish | M001–M012 closed; no registered successor; future dashboard work requires a new bounded plan. | -| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M023 blocked — short-window fairness controls; M022 closed | M017–M022 closed; M023 awaits ordinary-server operational evidence. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | -| On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 conditionally closed — qualification tooling implemented; isolated ordinary-server baseline still unmeasured | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 authorizes no WAL/SQLite topology change. | +| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M023 ready — short-window fairness controls; M022 and ordinary-server operational gate closed | M017–M022 closed; M023 may begin against measured M002 host evidence. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | +| On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 closed — ordinary-server synthetic baseline recorded | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 authorizes no WAL/SQLite topology change. | ## Dependency-ready implementation plans | Subsystem | Milestone | Status | Implementation plan | Dependencies / handoff note | |---|---|---|---|---| | Persistence | M011 — post-M010 roadmap and documentation reconciliation | ready | `plans/implementation/persistence/011-post-m010-roadmap-documentation-reconciliation.md` | Hard: M009/M010 closures accepted. Documentation/tooling only; no runtime or physical target dependency. | +| On-prem user access | M023 — short-window team fairness | ready | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | Hard M022 is closed; the operational ordinary-server gate is satisfied by `plans/closure/on-prem-concurrency-performance/002-status.md`. | ## Active implementation plans @@ -77,7 +78,6 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Blocker | |---|---|---| -| On-prem user access | M023 — short-window team fairness | Ordinary-server M002 operational evidence: representative Linux server with durable SSD/NVMe and measured baseline; M022 is closed. | Unblock audit (on-prem user access reopening after M012 closure, baseline @@ -109,7 +109,8 @@ Historical M007 blocker assessment at baseline `7e241ad` (`plans/closure/persist | Subsystem / plan | Disposition | Evidence | |---|---|---| -| On-prem user access M022 — hard-budget agent interoperability | closed — Codex CLI 0.162.1 omitted a Responses output cap; OpenCode 1.18.35 sent configured `max_output_tokens`; stable proof-failure and exhausted-budget codes added; default/no-default verification passes; M023 remains blocked on ordinary-server evidence | `plans/closure/on-prem-user-access/022-status.md`, implementation `dee65ae` | +| On-prem concurrency performance M002 — ordinary-server team baseline | closed — 27 passing Q008 runs across nine cells; Q009 hard-budget, provider-fault, user-disable and restart evidence; bounded competing-I/O and SQLite gate diagnostics; measured host-specific sizing guidance; M023 promoted to ready | `plans/closure/on-prem-concurrency-performance/002-status.md`, artifacts under `artifacts/qualification/on-prem-concurrency-performance/ordinary-server/` | +| On-prem user access M022 — hard-budget agent interoperability | closed — Codex CLI 0.162.1 omitted a Responses output cap; OpenCode 1.18.35 sent configured `max_output_tokens`; stable proof-failure and exhausted-budget codes added; default/no-default verification passes; M023 is now ready after M002 operational evidence | `plans/closure/on-prem-user-access/022-status.md`, implementation `dee65ae` | | On-prem user access M021 — default member policy and provisioning | closed — transactional future-only default profile selection, CLI/browser creation, explicit no-profile override, additive schema 63 migration, policy visibility and budget-separation behavior pass; M022 subsequently closed; M023 remains blocked on ordinary-server evidence | `plans/closure/on-prem-user-access/021-status.md`, implementation `b76d359`, CLI journey `ee563be`, race test `3f1ca54` | | On-prem user access M020 — managed transition and headless administration | closed — first member transition, profile assignment and audit are atomic; headless add/show/disable/enable workflow and stale-config auth rejection pass; M021 promoted to ready | `plans/closure/on-prem-user-access/020-status.md`, implementation `8a875a1` | | On-prem user access M019 — login transport and solo LAN operability corrective | closed — remote plaintext login GET returns no credential form; loopback and configured HTTPS-proxy paths pass; solo SSH tunnel and team proxy guidance documented; M020 promoted to ready | `plans/closure/on-prem-user-access/019-status.md`, implementation `b18a561` | @@ -119,7 +120,6 @@ Historical M007 blocker assessment at baseline `7e241ad` (`plans/closure/persist | On-prem user access M015 — independent authentication/authorization threat review | closed — independent review findings on hard-cost proof bounds and browser-session access corrected and re-reviewed; default 1,024 / no-default 1,025 tests pass (one ignored each); exact-SHA CI, installer and dependency audit pass on `d8fd9c2a`; no M015 security findings remain; M016 promoted to ready | `plans/closure/on-prem-user-access/015-status.md`, implementation `a526980c`, `deb1622f`, `d8fd9c2a`, security scan `52bd79cc`, CI `38720507683`, installer `38720509034`, audit `38720507720` | | On-prem user access M014 — Rustls dependency security-advisory corrective | closed — lockfile-only Rustls 0.23.46 patch, unchanged transport feature graph; local default/no-default and 1,009/1,010-test workspace suites, provider TLS (trusted root, wrong-root and hostname rejection), budget/coordinator suites, release build and tooling pass; hosted cargo-deny, CI and both installers pass on `7462263ec482c5fda4bd1a70a9a55b0efc313cd3`; main remains pending PR merge | `plans/closure/on-prem-user-access/014-status.md`, implementation `7462263e`, audit `38671345587`, CI `38671345679`, installers `38671345597` | | On-prem concurrency performance M001 — Pi 5/microSD concurrent tail attribution | conditionally closed — three paired physical microSD/tmpfs runs identify durable publication/SQLite/storage as the dominant tested four-way tail contributor; no production change or capacity claim; SSD/NVMe is not required | `plans/closure/on-prem-concurrency-performance/001-status.md`, six Q008 reports + manifest under `artifacts/qualification/on-prem-concurrency-performance/`, artifact guard `tests/tooling/test_on_prem_mmc_tail_attribution.py` | -| On-prem concurrency performance M002 — ordinary-server team baseline | conditionally closed — ordinary-host attestation, safe frozen 27-run driver, raw-to-summary guard, and deployment guidance implemented; no release build or measurement because the shared host crossed the I/O-pressure stop gate; no capacity or SLA claim | `plans/closure/on-prem-concurrency-performance/002-status.md`, implementation `cdfb6fa8`, tests `tests/tooling/test_qualification_ordinary_server.py` | | On-prem user access M012 — physical SBC small-team qualification and final disposition | closed — Pi 5 managed envelope measured (functional 25 cells + 3×30 benchmark batches all pass, 145/145 each, 0 pending/active; Argon2id 51 ms 1-way / 71–73 ms 2-way; 49 named users at 51 ms p50 with spot logins passing; concurrency 1/2/4/8 completes with 8-way saturating at p50 4.1 s); runner managed-auth corrective + 2 contract tests; solo practical, 49 users inexpensive metadata, no 50-concurrent claim; one medium (MMC tails exceed Sep-21 gates, envelope restricted) + six low findings, no high | `plans/closure/on-prem-user-access/012-status.md`, artifacts `artifacts/qualification/on-prem-m012-pi5-functional.json` + `on-prem-m012-pi5-benchmark-run-{1,2,3}.json`, docs `docs/raspberry-pi.md` envelope | | On-prem user access M011 — admin/member dashboard browser qualification | closed — frozen Python oracle rebuilt and run against head with 54 parity gaps, identical to the pre-M011 baseline (0 new, 0 resolved); on-prem admin/member pages qualified against a Rust-owned approved reference (70 observations, 22 captures) covering both roles, the negative matrix, no-JS sign-in, HiDPI/mobile/themed viewports, and a DOM credential-leak guard; browser sign-in, top-nav overflow at 1280px, unstyled usage pages, missing viewport declarations, unbounded tables, and budget labelling all corrected; 967 serial Rust tests and 175 tooling tests pass at the M011 baseline (re-verified post-integration — see integration note) | `plans/closure/on-prem-user-access/011-status.md` | | On-prem user access M010 — budget crash recovery and schema-54 restore | closed @@ -550,3 +550,14 @@ requirement, but the evidence remains unmeasured and M023 remains blocked on its hard M022 dependency. No plan was promoted or unblocked. A future isolated server run remains required before M023 can use M002 as its operational capacity evidence. + +Unblock audit (on-prem concurrency performance M002 closed, +`plans/closure/on-prem-concurrency-performance/002-status.md`): searched +`plans/implementation/`, the on-prem concurrency and user-access roadmap +dependency sections, and the dependency-ready, active, and blocked registry +tables. M023's hard M022 dependency is closed and its named operational M002 +requirement is satisfied by the durable-NVMe host matrix and Q009 lifecycle +evidence, so M023 moves `blocked` → `ready`. No other registered future plan +declares M002 as a hard, interface, or soft dependency. M001 remains +conditionally closed; no production storage optimization is authorized by +this evidence. diff --git a/plans/subsystems/on-prem-concurrency-performance-roadmap.md b/plans/subsystems/on-prem-concurrency-performance-roadmap.md index a88b1a62..602b0ca6 100644 --- a/plans/subsystems/on-prem-concurrency-performance-roadmap.md +++ b/plans/subsystems/on-prem-concurrency-performance-roadmap.md @@ -138,15 +138,13 @@ claims. Source: ### M002 — Ordinary-server small-team baseline -Class: polish. Conditionally closed for the tooling/setup slice. The physical -measurement and operating envelope remain unmeasured because the available -host was shared and crossed the frozen I/O-pressure stop gate during required -build verification. A future isolated run must use durable SSD/NVMe and cover -1/5/10/25/49 named principal records separately from 1/2/4/8/16 concurrent -requests, with three fresh-root repetitions per condition, adequate p99 sample -counts, active-load recovery evidence, and sanitized summary output. SBC -microSD remains the solo low-end performance/durability reference; it is -**not** a gate for team capacity or server release. +Class: polish. Closed with a measured one-host synthetic envelope across +1/5/10/25/49 named principals and 1/2/4/8/16 concurrent requests, three +fresh-root repetitions per condition, managed lifecycle/recovery checks, and a +bounded competing-I/O observation. Use the host-specific result for relative +sizing only; it is not an SLA or a provider-throughput promise. SBC microSD +remains the solo low-end performance/durability reference; it is **not** a +gate for team capacity or server release. Source: `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md`. Closure: `plans/closure/on-prem-concurrency-performance/002-status.md`. @@ -212,4 +210,4 @@ the Pi/MMC solo latency tail. No optimization promise or new | Milestone | Status | Implementation plan | Closure record | Blockers | |---|---|---|---|---| | M001 | conditionally closed | `plans/implementation/on-prem-concurrency-performance/001-pi5-mmc-tail-latency-attribution.md` | `plans/closure/on-prem-concurrency-performance/001-status.md` | Four-way microSD/MMC tail attributed to durable storage; controlled competing-I/O and broader account/budget matrix remain conditions. SSD/NVMe is optional and not a closure dependency. No dependency edge to post-merge on-prem operability plans. | -| M002 | conditionally closed | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | `plans/closure/on-prem-concurrency-performance/002-status.md` | Tooling/setup complete; ordinary-server performance, active-load recovery, and capacity envelope remain unmeasured pending an isolated target below the frozen I/O-pressure stop gate. | +| M002 | closed | `plans/implementation/on-prem-concurrency-performance/002-ordinary-server-team-baseline.md` | `plans/closure/on-prem-concurrency-performance/002-status.md` | One-host synthetic ordinary-server baseline and managed recovery evidence recorded; no universal capacity promise and no production optimization authorized. | diff --git a/plans/subsystems/on-prem-user-access-roadmap.md b/plans/subsystems/on-prem-user-access-roadmap.md index 6b517b16..6b47b1e0 100644 --- a/plans/subsystems/on-prem-user-access-roadmap.md +++ b/plans/subsystems/on-prem-user-access-roadmap.md @@ -346,14 +346,14 @@ behavior and adding actionable hard-budget diagnostics. No output cap is injected; M008/M015 proof-or-reject guarantees remain in force. Source: `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md`. -#### M023 — Short-window team fairness controls - -Class: capability. Hard M022 dependency is closed. Remains blocked by the -source plan's stop condition until ordinary-server Linux/SSD-or-NVMe evidence -is recorded through concurrency M002. Then implement an optional -per-principal short-window admission throttle and stable 429/Retry-After, -bounded in-memory state, no unbounded queues or new SQLite write pressure; -preserve solo no-rule behavior. +#### M023 — Short-window team fairness controls (ready) + +Class: capability. Hard M022 dependency is closed, and the operational +ordinary-server Linux/SSD-or-NVMe evidence gate is satisfied by concurrency +M002. Implement an optional per-principal short-window admission throttle and +stable 429/Retry-After, bounded in-memory state, no unbounded queues or new +SQLite write pressure; preserve solo no-rule behavior. M002 provides a +host-specific synthetic reference, not a universal capacity guarantee. Source: `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md`. ### Post-merge dependency and hardware policy @@ -461,4 +461,4 @@ A ready plan is not an implemented capability. | M020 | closed | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | `plans/closure/on-prem-user-access/020-status.md` | Headless member lifecycle and first-member transition are atomic and audited; M021 unblocked. | | M021 | closed | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | `plans/closure/on-prem-user-access/021-status.md` | Transactional future-only profile default; CLI/browser and no-profile journeys, migration, policy and budget boundaries verified. | | M022 | closed | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | `plans/closure/on-prem-user-access/022-status.md` | Captured Codex/OpenCode output-cap behavior; preserved hard proof-or-reject semantics and added actionable error codes. | -| M023 | blocked | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Hard M022 is closed; ordinary-server M002 Linux/SSD-or-NVMe operational evidence remains outstanding. | +| M023 | ready | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Hard M022 is closed; ordinary-server M002 operational evidence is recorded in its closure. | From 3722ae64ede200da69415c1da711dd6c5248c519 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Fri, 16 Oct 2026 02:51:09 +0000 Subject: [PATCH 56/62] Activate M023 short-window fairness controls --- .../on-prem-user-access/023-short-window-fairness-controls.md | 2 +- plans/registry.md | 4 ++-- plans/subsystems/on-prem-user-access-roadmap.md | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md b/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md index 1823cc3c..a4dc3aa3 100644 --- a/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md +++ b/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md @@ -1,6 +1,6 @@ # On-Prem User Access M023 — Short-window team fairness and admission controls -Status: ready +Status: active Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index a4d663fe..bb740228 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -58,7 +58,7 @@ No new runtime, schema or performance claim lands with these plans. | Persistence | active | `plans/subsystems/persistence-roadmap.md` | M011 ready — post-M010 roadmap/documentation reconciliation | M009/M010 closures accepted. Documentation/tooling-only corrective; production remains WAL/NORMAL. No technical persistence successor is registered. | | Deployment and packaging | closed | `plans/subsystems/deployment-packaging-roadmap.md` | M005 closed — active PATH integration detection corrective | M001–M005 closed; no successor registered. | | Dashboard | closed | `plans/subsystems/dashboard-roadmap.md` | M012 closed — lifecycle closure and documentation polish | M001–M012 closed; no registered successor; future dashboard work requires a new bounded plan. | -| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M023 ready — short-window fairness controls; M022 and ordinary-server operational gate closed | M017–M022 closed; M023 may begin against measured M002 host evidence. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | +| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M023 active — short-window fairness controls | M017–M022 closed; M002 ordinary-server gate satisfied. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | | On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 closed — ordinary-server synthetic baseline recorded | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 authorizes no WAL/SQLite topology change. | ## Dependency-ready implementation plans @@ -66,12 +66,12 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Dependencies / handoff note | |---|---|---|---|---| | Persistence | M011 — post-M010 roadmap and documentation reconciliation | ready | `plans/implementation/persistence/011-post-m010-roadmap-documentation-reconciliation.md` | Hard: M009/M010 closures accepted. Documentation/tooling only; no runtime or physical target dependency. | -| On-prem user access | M023 — short-window team fairness | ready | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | Hard M022 is closed; the operational ordinary-server gate is satisfied by `plans/closure/on-prem-concurrency-performance/002-status.md`. | ## Active implementation plans | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| +| On-prem user access | M023 — short-window team fairness | active | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | Implement opt-in bounded principal/global admission before body collection; validate against measured ordinary-server evidence. | ## Blocked work diff --git a/plans/subsystems/on-prem-user-access-roadmap.md b/plans/subsystems/on-prem-user-access-roadmap.md index 6b47b1e0..3bd701ff 100644 --- a/plans/subsystems/on-prem-user-access-roadmap.md +++ b/plans/subsystems/on-prem-user-access-roadmap.md @@ -346,7 +346,7 @@ behavior and adding actionable hard-budget diagnostics. No output cap is injected; M008/M015 proof-or-reject guarantees remain in force. Source: `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md`. -#### M023 — Short-window team fairness controls (ready) +#### M023 — Short-window team fairness controls (active) Class: capability. Hard M022 dependency is closed, and the operational ordinary-server Linux/SSD-or-NVMe evidence gate is satisfied by concurrency @@ -461,4 +461,4 @@ A ready plan is not an implemented capability. | M020 | closed | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | `plans/closure/on-prem-user-access/020-status.md` | Headless member lifecycle and first-member transition are atomic and audited; M021 unblocked. | | M021 | closed | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | `plans/closure/on-prem-user-access/021-status.md` | Transactional future-only profile default; CLI/browser and no-profile journeys, migration, policy and budget boundaries verified. | | M022 | closed | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | `plans/closure/on-prem-user-access/022-status.md` | Captured Codex/OpenCode output-cap behavior; preserved hard proof-or-reject semantics and added actionable error codes. | -| M023 | ready | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Hard M022 is closed; ordinary-server M002 operational evidence is recorded in its closure. | +| M023 | active | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Hard M022 is closed; ordinary-server M002 operational evidence is recorded in its closure. | From 84482e91b84fe7424144dcf916441479c5b492d1 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Fri, 16 Oct 2026 03:12:18 +0000 Subject: [PATCH 57/62] Implement bounded short-window admission fairness --- architecture/deep-dive-request-lifecycle.md | 10 + architecture/deep-dive-security.md | 6 + docs/api-reference.md | 7 + docs/deployment.md | 7 + docs/model-limits.md | 30 ++ rust/assets/runtime-manifest.json | 2 +- rust/config.example.toml | 7 + rust/src/config.rs | 73 ++++ rust/src/config_reload_policy.rs | 12 + rust/src/request/admission_fairness.rs | 381 ++++++++++++++++++ rust/src/request/mod.rs | 4 + rust/src/server/middleware.rs | 82 +++- rust/src/server/mod.rs | 5 + rust/tests/runtime_lifecycle_r005.rs | 28 +- rust/tests/server_transport.rs | 88 ++++ .../runtime/compatibility-observations.json | 2 +- 16 files changed, 739 insertions(+), 5 deletions(-) create mode 100644 rust/src/request/admission_fairness.rs diff --git a/architecture/deep-dive-request-lifecycle.md b/architecture/deep-dive-request-lifecycle.md index 4a02f8e5..50de7f6f 100644 --- a/architecture/deep-dive-request-lifecycle.md +++ b/architecture/deep-dive-request-lifecycle.md @@ -22,6 +22,16 @@ adapts the response, and converges durable state. extractor: it acquires a `GenerationLease`, reads the live `server.max_request_body_bytes`, and enforces it while the body streams. `is_inference_path()` covers exactly the four public inference routes. +- The same middleware consults the process-local short-window fairness + limiter after authentication and generation acquisition but before reading + the body or reserving durable user/global budget. It keys buckets by + authenticated user ID, shares allowance across that user's credentials, + applies optional per-principal/global 60-second token buckets, and bounds + retained principal state. It never queues, writes SQLite, or changes + provider routing. State resets on restart; absent rules allocate no bucket. + Throttles are 429 with `Retry-After` and + `short_window_rate_limited`; successful admissions expose bounded remaining + allowance headers. - Declared `Content-Length` above the live limit is rejected before polling (413); malformed `Content-Length` is 400. A 32 KiB initial `RawBodyReservation` is taken against `effective_ceiling()` = diff --git a/architecture/deep-dive-security.md b/architecture/deep-dive-security.md index 2c3179b4..92a97cc4 100644 --- a/architecture/deep-dive-security.md +++ b/architecture/deep-dive-security.md @@ -96,6 +96,12 @@ Failed personal-token lookups use a 256-entry, one-second negative cache keyed by token digest. Successful token lookups are not cached, so revocation takes effect on the next request. +Optional short-window fairness uses only the authenticated principal's +database-owned user ID; credentials and request headers cannot choose a bucket. +The process-local table is capacity-bounded, stale entries are reclaimed only +after the full refill window, and a full table fails closed for new principals. +No token, prompt, body, IP, or provider outcome is retained in limiter state. + ## Headers, bodies, and redaction Provider credentials are added only at dispatch-header construction. Incoming diff --git a/docs/api-reference.md b/docs/api-reference.md index 9bc14e35..a8ef66e2 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -138,6 +138,13 @@ intersected with every per-user rule. Members never receive instance fields, so one member cannot observe the shared ceiling or infer other users' consumption from it. +An optional `[admission_fairness]` policy can add process-local short-window +request throttles before request-body collection. A throttle returns HTTP 429 +with `Retry-After`, code `short_window_rate_limited`, and +`X-EggPool-Admission-Scope`; accepted responses expose the remaining allowance +in `X-EggPool-Admission-Remaining`. These short-window counters reset on +restart and do not replace durable daily/monthly user or instance budgets. + With `[dashboard].enabled = true`, the dashboard also serves browser-session pages for `/users`, `/profiles`, `/usage`, `/admin/audit`, `/my-account`, and `/my-account/models`. Mutations use same-origin, CSRF-protected form posts diff --git a/docs/deployment.md b/docs/deployment.md index 832b4f14..8b899d1b 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -334,6 +334,13 @@ use `eggpool restart` for those. A mixed live + restart-required change is rejected entirely (exit code `2`); no partial application. The CLI is fail-closed and never implicitly restarts the service. +The optional `[admission_fairness]` short-window rates and principal-state +bound are live-reloadable. They take effect for requests using the newly +published generation; in-memory buckets are shared across generations and +clamped to the active policy. Restart clears the ephemeral limiter state. +See [model limits](model-limits.md#short-window-admission-fairness) for the +rate bounds and client response contract. + Sticky model-router affinity remains process-local across these live swaps only when the router's semantic fingerprint is unchanged. Policy edits naturally make prior decisions unreachable; no persisted cache state or restart step is diff --git a/docs/model-limits.md b/docs/model-limits.md index 0459b2bf..a0eb4ee1 100644 --- a/docs/model-limits.md +++ b/docs/model-limits.md @@ -153,6 +153,36 @@ nothing is silently upgraded to a guarantee. identifies an exhausted rule. Observed provider usage above a hard hold is settled visibly, never clamped. +### Short-window admission fairness + +Operators can optionally cap bursty inference traffic before EggPool reads the +request body or enters durable budget reservation. The rule is process-local, +uses a monotonic 60-second token bucket, and defaults to disabled: + +```toml +[admission_fairness] +requests_per_minute_per_principal = 120 +global_requests_per_minute = 1200 +max_tracked_principals = 4096 +``` + +Each principal's bucket is keyed by its authenticated user ID, so multiple +agent credentials for one user share the same allowance. The global bucket, +when configured, intersects the per-user rule. Rates are bounded to 1–60,000 +requests per minute; tracked principal state is bounded to 1–65,536 entries. +If the state table is full of recently active principals, a new principal is +rejected with 429 until stale state can be reclaimed; EggPool does not evict a +live bucket to grant extra traffic. Limiter state resets on restart, while +daily/monthly budgets remain durable and authoritative. + +Throttles return HTTP 429, `Retry-After`, machine code +`short_window_rate_limited`, and `X-EggPool-Admission-Scope` (`principal`, +`global`, or `principal_state_capacity`). Admitted responses expose remaining +short-window allowance in `X-EggPool-Admission-Remaining`, plus the scope and +60-second window. Malformed or oversized requests consume an admission token +because the decision is made before body collection. No-rule installations +take the existing admission path without allocating principal state. + ### Coding-agent compatibility The supported request fields used as output bounds are: diff --git a/rust/assets/runtime-manifest.json b/rust/assets/runtime-manifest.json index 5356fc3f..06916f34 100644 --- a/rust/assets/runtime-manifest.json +++ b/rust/assets/runtime-manifest.json @@ -26,7 +26,7 @@ "path": "config/config.example.toml", "category": "configuration-template", "source": "rust/config.example.toml", - "sha256": "046355f5469c730125d3179a190dc605bcdde344a7b1f1ef7ab3ad304dfce7b4", + "sha256": "bf1bbe6c23475bbb0f6a9f309c02bdc29bd9e01c22d39ea182f5353226bc83d6", "identity": "exact" }, { diff --git a/rust/config.example.toml b/rust/config.example.toml index d6dee941..ce5a13b9 100644 --- a/rust/config.example.toml +++ b/rust/config.example.toml @@ -267,6 +267,13 @@ five_hour_microdollars = 12000000 weekly_microdollars = 30000000 monthly_microdollars = 60000000 +# Optional process-local short-window admission controls. Omitted rates leave +# that rule disabled. Each configured rate is a 60-second token-bucket capacity. +[admission_fairness] +# requests_per_minute_per_principal = 120 +# global_requests_per_minute = 1200 +# max_tracked_principals = 4096 + # --- Pricing resolution ---------------------------------------------- # External pricing catalogs supplement upstream model metadata when # providers do not include pricing in /v1/models responses. The per-catalog diff --git a/rust/src/config.rs b/rust/src/config.rs index 30f2c70e..66d69578 100644 --- a/rust/src/config.rs +++ b/rust/src/config.rs @@ -367,6 +367,26 @@ impl Default for LimitsConfig { } } } + +/// Optional process-local fairness controls for authenticated inference. +/// Both rates use a 60-second token bucket; `None` keeps that rule disabled. +#[derive(Debug, Clone, Deserialize, Serialize)] +#[serde(deny_unknown_fields, default)] +pub struct AdmissionFairnessConfig { + pub requests_per_minute_per_principal: Option, + pub global_requests_per_minute: Option, + pub max_tracked_principals: usize, +} + +impl Default for AdmissionFairnessConfig { + fn default() -> Self { + Self { + requests_per_minute_per_principal: None, + global_requests_per_minute: None, + max_tracked_principals: 4096, + } + } +} #[derive(Debug, Clone, Deserialize, Serialize)] #[serde(deny_unknown_fields, default)] pub struct DispatchSpansConfig { @@ -1430,6 +1450,7 @@ pub struct Config { pub model_routers: BTreeMap, pub routing: RoutingConfig, pub limits: LimitsConfig, + pub admission_fairness: AdmissionFairnessConfig, pub pricing: PricingConfig, pub dashboard: DashboardConfig, pub security: SecurityConfig, @@ -1571,6 +1592,27 @@ impl Config { } pub fn validate(&mut self) -> Result<(), ConfigError> { + for (name, rate) in [ + ( + "admission_fairness.requests_per_minute_per_principal", + self.admission_fairness.requests_per_minute_per_principal, + ), + ( + "admission_fairness.global_requests_per_minute", + self.admission_fairness.global_requests_per_minute, + ), + ] { + if rate.is_some_and(|rate| rate == 0 || rate > 60_000) { + return Err(ConfigError::validation(format!( + "{name} must be between 1 and 60000 when set" + ))); + } + } + if !(1..=65_536).contains(&self.admission_fairness.max_tracked_principals) { + return Err(ConfigError::validation( + "admission_fairness.max_tracked_principals must be between 1 and 65536", + )); + } for proxy in &self.security.trusted_proxies { if proxy.parse::().is_err() { return Err(ConfigError::validation( @@ -2450,6 +2492,11 @@ mod tests { assert!(config.transcoder.enabled); assert!(config.model_info.enabled); assert_eq!(config.server.max_request_body_bytes, 10 * 1024 * 1024); + assert_eq!(config.admission_fairness.max_tracked_principals, 4096); + assert_eq!( + config.admission_fairness.requests_per_minute_per_principal, + None + ); } #[test] @@ -2461,6 +2508,32 @@ mod tests { assert!(config.validate().is_err()); } + #[test] + fn short_window_fairness_values_are_bounded_and_optional() { + let mut config = Config::default(); + assert!(config.validate().is_ok()); + + config.admission_fairness.requests_per_minute_per_principal = Some(1); + config.admission_fairness.global_requests_per_minute = Some(60_000); + config.admission_fairness.max_tracked_principals = 65_536; + assert!(config.validate().is_ok()); + + for rate in [Some(0), Some(60_001)] { + let mut invalid = Config::default(); + invalid.admission_fairness.requests_per_minute_per_principal = rate; + assert!(invalid.validate().is_err()); + let mut invalid = Config::default(); + invalid.admission_fairness.global_requests_per_minute = rate; + assert!(invalid.validate().is_err()); + } + + for limit in [0, 65_537] { + let mut invalid = Config::default(); + invalid.admission_fairness.max_tracked_principals = limit; + assert!(invalid.validate().is_err()); + } + } + #[test] fn loopback_detection_never_fails_open_on_a_hostname() { // The bind-address API-key gate skips the key only for a loopback diff --git a/rust/src/config_reload_policy.rs b/rust/src/config_reload_policy.rs index 5990935c..7bae4506 100644 --- a/rust/src/config_reload_policy.rs +++ b/rust/src/config_reload_policy.rs @@ -534,6 +534,18 @@ const DYNAMIC_RULES: &[(&str, ReloadDisposition)] = &[ // Rust table. const FIELD_DISPOSITIONS: &[(&str, ReloadDisposition)] = &[ ("accounts", ReloadDisposition::Live), + ( + "admission_fairness.global_requests_per_minute", + ReloadDisposition::Live, + ), + ( + "admission_fairness.max_tracked_principals", + ReloadDisposition::Live, + ), + ( + "admission_fairness.requests_per_minute_per_principal", + ReloadDisposition::Live, + ), ("backup.directory", ReloadDisposition::RestartRequired), ("backup.enabled", ReloadDisposition::Live), ("backup.include_env", ReloadDisposition::RestartRequired), diff --git a/rust/src/request/admission_fairness.rs b/rust/src/request/admission_fairness.rs new file mode 100644 index 00000000..2fc5c2d3 --- /dev/null +++ b/rust/src/request/admission_fairness.rs @@ -0,0 +1,381 @@ +//! Bounded process-local short-window inference admission fairness. +//! +//! The limiter keys principals by the authenticated user id, so credentials +//! owned by one user share one bucket. It is consulted before the inference +//! body is collected and never touches durable budget state or provider +//! routing. State is intentionally ephemeral across process restarts. + +use std::{ + collections::HashMap, + sync::{Arc, Mutex, MutexGuard}, + time::{Duration, Instant}, +}; + +const WINDOW: Duration = Duration::from_secs(60); +const STALE_AFTER: Duration = Duration::from_secs(120); +const LEGACY_PRINCIPAL: &str = "\0legacy-shared-key"; + +/// Validated policy projected from the active runtime generation. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub(crate) struct AdmissionFairnessPolicy { + pub(crate) requests_per_minute_per_principal: Option, + pub(crate) global_requests_per_minute: Option, + pub(crate) max_tracked_principals: usize, +} + +impl AdmissionFairnessPolicy { + fn is_enabled(self) -> bool { + self.requests_per_minute_per_principal.is_some() + || self.global_requests_per_minute.is_some() + } +} + +/// Result of one authenticated admission decision. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum AdmissionFairnessDecision { + Disabled, + Allowed { + remaining: u32, + scope: &'static str, + }, + Limited { + retry_after_seconds: u64, + scope: &'static str, + }, +} + +#[derive(Clone, Default)] +pub(crate) struct AdmissionFairnessLimiter { + inner: Arc>, +} + +#[derive(Default)] +struct LimiterState { + principals: HashMap, + global: Option, +} + +#[derive(Clone, Copy)] +struct Bucket { + tokens: f64, + updated_at: Instant, + last_seen: Instant, +} + +impl Bucket { + fn full(capacity: u32, now: Instant) -> Self { + Self { + tokens: f64::from(capacity), + updated_at: now, + last_seen: now, + } + } + + fn refill(&mut self, capacity: u32, now: Instant) { + let elapsed = now.saturating_duration_since(self.updated_at).as_secs_f64(); + let refill_per_second = f64::from(capacity) / WINDOW.as_secs_f64(); + self.tokens = (self.tokens + elapsed * refill_per_second).min(f64::from(capacity)); + self.updated_at = now; + self.last_seen = now; + } + + fn retry_after(self, capacity: u32) -> u64 { + let refill_per_second = f64::from(capacity) / WINDOW.as_secs_f64(); + let wait = ((1.0 - self.tokens).max(0.0) / refill_per_second).ceil(); + wait.max(1.0) as u64 + } +} + +impl AdmissionFairnessLimiter { + /// Admit one request using a supplied monotonic instant. The clock seam + /// makes window, simultaneous-slot, and stale-eviction behavior testable. + pub(crate) fn admit_at( + &self, + policy: AdmissionFairnessPolicy, + principal_id: Option<&str>, + now: Instant, + ) -> AdmissionFairnessDecision { + if !policy.is_enabled() { + return AdmissionFairnessDecision::Disabled; + } + let mut state = lock(&self.inner); + let principal_key = policy + .requests_per_minute_per_principal + .map(|_| principal_id.unwrap_or(LEGACY_PRINCIPAL)); + let principal_existing = principal_key + .as_ref() + .and_then(|key| state.principals.get(*key).copied()); + + if let (Some(_), None) = (principal_key.as_ref(), principal_existing) + && state.principals.len() >= policy.max_tracked_principals + { + let stale = state + .principals + .iter() + .filter(|(_, bucket)| { + now.saturating_duration_since(bucket.last_seen) >= STALE_AFTER + }) + .min_by_key(|(_, bucket)| bucket.last_seen) + .map(|(id, bucket)| (id.clone(), bucket.last_seen)); + if let Some((stale_id, _)) = stale { + state.principals.remove(&stale_id); + } else { + let retry_after_seconds = state + .principals + .values() + .map(|bucket| { + STALE_AFTER + .saturating_sub(now.saturating_duration_since(bucket.last_seen)) + .as_secs() + .max(1) + }) + .min() + .unwrap_or(1); + return AdmissionFairnessDecision::Limited { + retry_after_seconds, + scope: "principal_state_capacity", + }; + } + } + + let user_capacity = policy.requests_per_minute_per_principal; + let mut user_bucket = match (&principal_key, principal_existing, user_capacity) { + (Some(_), Some(mut bucket), Some(capacity)) => { + bucket.refill(capacity, now); + Some(bucket) + } + (Some(_), None, Some(capacity)) => Some(Bucket::full(capacity, now)), + _ => None, + }; + let mut global_bucket = match (policy.global_requests_per_minute, state.global) { + (Some(capacity), Some(mut bucket)) => { + bucket.refill(capacity, now); + Some(bucket) + } + (Some(capacity), None) => Some(Bucket::full(capacity, now)), + _ => None, + }; + + if let (Some(capacity), Some(bucket)) = (user_capacity, user_bucket) + && bucket.tokens < 1.0 + { + return AdmissionFairnessDecision::Limited { + retry_after_seconds: bucket.retry_after(capacity), + scope: "principal", + }; + } + if let (Some(capacity), Some(bucket)) = (policy.global_requests_per_minute, global_bucket) + && bucket.tokens < 1.0 + { + return AdmissionFairnessDecision::Limited { + retry_after_seconds: bucket.retry_after(capacity), + scope: "global", + }; + } + + let mut remaining = u32::MAX; + if let (Some(key), Some(mut bucket)) = (principal_key, user_bucket.take()) { + bucket.tokens -= 1.0; + bucket.last_seen = now; + remaining = remaining.min(bucket.tokens.floor() as u32); + state.principals.insert(key.to_owned(), bucket); + } + if let Some(mut bucket) = global_bucket.take() { + bucket.tokens -= 1.0; + state.global = Some(bucket); + remaining = remaining.min(bucket.tokens.floor() as u32); + } + let scope = match ( + user_capacity.is_some(), + policy.global_requests_per_minute.is_some(), + ) { + (true, true) => "principal,global", + (true, false) => "principal", + (false, true) => "global", + (false, false) => "none", + }; + AdmissionFairnessDecision::Allowed { remaining, scope } + } + + #[cfg(test)] + fn principal_count(&self) -> usize { + lock(&self.inner).principals.len() + } +} + +fn lock(mutex: &Mutex) -> MutexGuard<'_, LimiterState> { + mutex + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()) +} + +#[cfg(test)] +mod tests { + use super::{AdmissionFairnessDecision as Decision, AdmissionFairnessLimiter as Limiter, *}; + use std::thread; + + fn policy(per_user: Option, global: Option, max: usize) -> AdmissionFairnessPolicy { + AdmissionFairnessPolicy { + requests_per_minute_per_principal: per_user, + global_requests_per_minute: global, + max_tracked_principals: max, + } + } + + #[test] + fn absent_policy_keeps_the_no_rule_fast_path() { + let limiter = Limiter::default(); + assert_eq!( + limiter.admit_at(policy(None, None, 8), Some("u1"), Instant::now()), + Decision::Disabled + ); + assert_eq!(limiter.principal_count(), 0); + } + + #[test] + fn all_credentials_for_one_principal_share_a_bucket() { + let limiter = Limiter::default(); + let start = Instant::now(); + let policy = policy(Some(2), None, 8); + assert!(matches!( + limiter.admit_at(policy, Some("user-a"), start), + Decision::Allowed { .. } + )); + assert!(matches!( + limiter.admit_at(policy, Some("user-a"), start), + Decision::Allowed { .. } + )); + assert_eq!( + limiter.admit_at(policy, Some("user-a"), start), + Decision::Limited { + retry_after_seconds: 30, + scope: "principal" + } + ); + assert!(matches!( + limiter.admit_at(policy, Some("user-b"), start), + Decision::Allowed { .. } + )); + } + + #[test] + fn global_and_principal_rules_intersect_without_consuming_denied_slots() { + let limiter = Limiter::default(); + let start = Instant::now(); + let policy = policy(Some(4), Some(1), 8); + assert!(matches!( + limiter.admit_at(policy, Some("a"), start), + Decision::Allowed { .. } + )); + assert_eq!( + limiter.admit_at(policy, Some("a"), start), + Decision::Limited { + retry_after_seconds: 60, + scope: "global" + } + ); + assert_eq!(limiter.principal_count(), 1); + assert_eq!( + limiter.admit_at(policy, Some("b"), start), + Decision::Limited { + retry_after_seconds: 60, + scope: "global" + } + ); + assert_eq!(limiter.principal_count(), 1); + } + + #[test] + fn monotonic_refill_admits_again_at_the_window_boundary() { + let limiter = Limiter::default(); + let start = Instant::now(); + let policy = policy(Some(1), None, 8); + assert!(matches!( + limiter.admit_at(policy, Some("a"), start), + Decision::Allowed { .. } + )); + assert!(matches!( + limiter.admit_at(policy, Some("a"), start + Duration::from_secs(60)), + Decision::Allowed { .. } + )); + } + + #[test] + fn principal_table_is_bounded_and_evicts_only_fully_stale_state() { + let limiter = Limiter::default(); + let start = Instant::now(); + let policy = policy(Some(10), None, 2); + for id in ["a", "b"] { + assert!(matches!( + limiter.admit_at(policy, Some(id), start), + Decision::Allowed { .. } + )); + } + assert_eq!( + limiter.admit_at(policy, Some("c"), start + Duration::from_secs(1)), + Decision::Limited { + retry_after_seconds: 119, + scope: "principal_state_capacity" + } + ); + assert!(matches!( + limiter.admit_at(policy, Some("c"), start + STALE_AFTER), + Decision::Allowed { .. } + )); + assert_eq!(limiter.principal_count(), 2); + } + + #[test] + fn simultaneous_last_slot_is_granted_once() { + let limiter = Limiter::default(); + let policy = policy(Some(1), None, 8); + let start = Instant::now(); + let handles = (0..32) + .map(|_| { + let limiter = limiter.clone(); + thread::spawn(move || limiter.admit_at(policy, Some("same-user"), start)) + }) + .collect::>(); + let allowed = handles + .into_iter() + .map(|handle| handle.join().expect("worker")) + .filter(|decision| matches!(decision, Decision::Allowed { .. })) + .count(); + assert_eq!(allowed, 1); + } + + #[test] + fn many_principals_remain_within_declared_capacity() { + let limiter = Limiter::default(); + let policy = policy(Some(2), None, 49); + let start = Instant::now(); + for index in 0..100 { + let _ = limiter.admit_at(policy, Some(&format!("principal-{index}")), start); + } + assert_eq!(limiter.principal_count(), 49); + } + + #[test] + fn qualification_user_counts_keep_buckets_separate() { + for principal_count in [1, 5, 25, 49] { + let limiter = Limiter::default(); + let policy = policy(Some(1), None, principal_count); + let start = Instant::now(); + for index in 0..principal_count { + let id = format!("member-{index}"); + assert!(matches!( + limiter.admit_at(policy, Some(&id), start), + Decision::Allowed { .. } + )); + assert!(matches!( + limiter.admit_at(policy, Some(&id), start), + Decision::Limited { + scope: "principal", + .. + } + )); + } + assert_eq!(limiter.principal_count(), principal_count); + } + } +} diff --git a/rust/src/request/mod.rs b/rust/src/request/mod.rs index 2c78a124..d8f478c7 100644 --- a/rust/src/request/mod.rs +++ b/rust/src/request/mod.rs @@ -4,6 +4,7 @@ //! owns the bounded parse and the values that later layers may safely reuse. pub mod admission; +pub(crate) mod admission_fairness; pub mod body; pub mod limits; pub(crate) mod resource_budget; @@ -15,6 +16,9 @@ pub use admission::{ has_compaction_trigger, routing_request_facts, thinking_requirement_from_intent, validate_responses_stateless_policy, }; +pub(crate) use admission_fairness::{ + AdmissionFairnessDecision, AdmissionFairnessLimiter, AdmissionFairnessPolicy, +}; pub(crate) use admission::{ ParsedRequestBody, admit_compact_parsed_request, admit_parsed_request, diff --git a/rust/src/server/middleware.rs b/rust/src/server/middleware.rs index 76d60587..11829b75 100644 --- a/rust/src/server/middleware.rs +++ b/rust/src/server/middleware.rs @@ -351,6 +351,32 @@ pub(super) async fn admit_inference_body( endpoint_error_body(surface, "Invalid body limit"), ); }; + let active_fairness = &lease.generation().config().admission_fairness; + let fairness_policy = crate::request::AdmissionFairnessPolicy { + requests_per_minute_per_principal: active_fairness.requests_per_minute_per_principal, + global_requests_per_minute: active_fairness.global_requests_per_minute, + max_tracked_principals: active_fairness.max_tracked_principals, + }; + let principal_id = request + .extensions() + .get::() + .and_then(|principal| principal.user_id.as_deref()); + let fairness_headers = + match state + .admission_fairness + .admit_at(fairness_policy, principal_id, Instant::now()) + { + crate::request::AdmissionFairnessDecision::Disabled => None, + crate::request::AdmissionFairnessDecision::Allowed { remaining, scope } => { + Some((remaining, scope)) + } + crate::request::AdmissionFairnessDecision::Limited { + retry_after_seconds, + scope, + } => { + return fairness_limited_response(surface, retry_after_seconds, scope); + } + }; let ceiling = crate::request::resource_budget::effective_ceiling(limit); // Malformed Content-Length is a client error, not an absent value: fail // closed with 400 instead of granting the cheap 32 KiB reservation path. @@ -437,7 +463,21 @@ pub(super) async fn admit_inference_body( request.extensions_mut().insert(Arc::new(lease)); request.extensions_mut().insert(Arc::new(reservation)); *request.body_mut() = Body::from(collected); - next.run(request).await + let mut response = next.run(request).await; + if let Some((remaining, scope)) = fairness_headers { + let headers = response.headers_mut(); + headers.insert( + "x-eggpool-admission-remaining", + HeaderValue::from_str(&remaining.to_string()) + .unwrap_or_else(|_| HeaderValue::from_static("0")), + ); + headers.insert("x-eggpool-admission-scope", HeaderValue::from_static(scope)); + headers.insert( + "x-eggpool-admission-window-seconds", + HeaderValue::from_static("60"), + ); + } + response } pub(super) fn is_inference_path(path: &str) -> bool { @@ -458,6 +498,46 @@ fn backpressure_response(surface: crate::wire::ir::ClientSurface, detail: Vec Response { + let message = "Short-window inference admission limit reached"; + let body = if surface == crate::wire::ir::ClientSurface::Messages { + serde_json::json!({ + "type": "error", + "error": { + "type": "api_error", + "message": message, + "code": "short_window_rate_limited" + } + }) + } else { + serde_json::json!({ + "error": { + "type": "rate_limit_error", + "message": message, + "code": "short_window_rate_limited" + } + }) + }; + let bytes = serde_json::to_vec(&body) + .unwrap_or_else(|_| br#"{"error":{"code":"short_window_rate_limited"}}"#.to_vec()); + let mut headers = HeaderMap::new(); + headers.insert( + header::CONTENT_TYPE, + HeaderValue::from_static("application/json"), + ); + headers.insert( + header::RETRY_AFTER, + HeaderValue::from_str(&retry_after_seconds.to_string()) + .unwrap_or_else(|_| HeaderValue::from_static("1")), + ); + headers.insert("x-eggpool-admission-scope", HeaderValue::from_static(scope)); + (StatusCode::TOO_MANY_REQUESTS, headers, Bytes::from(bytes)).into_response() +} + pub(super) fn validate_server_key(config: &Config) -> Result<(), ServerError> { let key = config.resolved_server_api_key(); if key.as_deref().is_some_and(|value| !valid_key_shape(value)) { diff --git a/rust/src/server/mod.rs b/rust/src/server/mod.rs index a030661c..97d14f10 100644 --- a/rust/src/server/mod.rs +++ b/rust/src/server/mod.rs @@ -407,6 +407,7 @@ impl ServerRuntime { runtime: Arc::clone(&self.inner.manager), process: Some(self.inner.process.clone()), body_tasks: self.inner.body_tasks.clone(), + admission_fairness: crate::request::AdmissionFairnessLimiter::default(), }); let runtime_config = eggserve_runtime_config().map_err(|error| ServerError::EggServe(Box::new(error)))?; @@ -704,6 +705,7 @@ pub struct AppState { pub runtime: Arc, process: Option, body_tasks: BodyTaskTracker, + admission_fairness: crate::request::AdmissionFairnessLimiter, } impl AppState { @@ -721,6 +723,7 @@ impl AppState { runtime, process: None, body_tasks: BodyTaskTracker::new(), + admission_fairness: crate::request::AdmissionFairnessLimiter::default(), } } @@ -746,6 +749,7 @@ impl AppState { runtime: Arc::new(RuntimeManager::new(generation)), process: None, body_tasks: BodyTaskTracker::new(), + admission_fairness: crate::request::AdmissionFairnessLimiter::default(), } } } @@ -903,6 +907,7 @@ pub async fn run_with_digest( runtime: Arc::clone(&manager), process: Some(process.clone()), body_tasks: BodyTaskTracker::new(), + admission_fairness: crate::request::AdmissionFairnessLimiter::default(), }; let control_server = match crate::operations::control::start(control_path, move |request| { let reload_service = reload_service.clone(); diff --git a/rust/tests/runtime_lifecycle_r005.rs b/rust/tests/runtime_lifecycle_r005.rs index d6690bdf..e7a9bd2d 100644 --- a/rust/tests/runtime_lifecycle_r005.rs +++ b/rust/tests/runtime_lifecycle_r005.rs @@ -86,13 +86,13 @@ fn exact_r001_field_dispositions_are_ported() { .map(|(path, disposition)| ((*path).to_owned(), disposition.to_string())) .collect::>(); assert_eq!(actual, expected); - assert_eq!(actual.len(), 155); + assert_eq!(actual.len(), 158); assert_eq!( actual .iter() .filter(|(_, disposition)| disposition == "live") .count(), - 61 + 64 ); assert_eq!( actual @@ -176,6 +176,30 @@ fn live_restart_and_mixed_mutations_are_classified_without_partial_semantics() { ); } +#[test] +fn admission_fairness_policy_is_live_reloaded_as_one_generation_contract() { + let old = Config::default(); + let mut new = old.clone(); + new.admission_fairness.requests_per_minute_per_principal = Some(120); + new.admission_fairness.global_requests_per_minute = Some(1200); + new.admission_fairness.max_tracked_principals = 2048; + let transition = classify_transition(&old, &new).expect("fairness transition"); + assert!(!transition.has_restart_required()); + assert_eq!( + transition + .diff() + .live() + .iter() + .map(|change| change.path.as_str()) + .collect::>(), + vec![ + "admission_fairness.global_requests_per_minute", + "admission_fairness.max_tracked_principals", + "admission_fairness.requests_per_minute_per_principal", + ] + ); +} + #[test] fn advertised_endpoint_change_is_live_and_noop_is_empty() { let old = Config::default(); diff --git a/rust/tests/server_transport.rs b/rust/tests/server_transport.rs index 2d13e886..c9333534 100644 --- a/rust/tests/server_transport.rs +++ b/rust/tests/server_transport.rs @@ -33,6 +33,49 @@ async fn runtime_fixture_with_body_limit( (directory, database, runtime) } +async fn runtime_fixture_with_short_window_limit( + requests_per_minute_per_principal: Option, + global_requests_per_minute: Option, +) -> (tempfile::TempDir, Database, ServerRuntime) { + let directory = tempfile::tempdir().expect("temporary state directory"); + let database = Database::open(DatabaseConfig { + path: directory + .path() + .join("eggpool.db") + .to_string_lossy() + .into_owned(), + ..DatabaseConfig::default() + }) + .await + .expect("database opens"); + MigrationRunner::new(&database) + .run() + .await + .expect("migrations run"); + let process = ProcessRuntime::new(database.clone()).expect("process runtime builds"); + let mut config = Config::default(); + config.server.api_key = Some("test-key-transport".to_owned()); + config.admission_fairness.requests_per_minute_per_principal = requests_per_minute_per_principal; + config.admission_fairness.global_requests_per_minute = global_requests_per_minute; + let candidate = RuntimeGenerationFactory::prepare( + &process, + config.clone(), + "server-transport-fairness".to_owned(), + 1, + ) + .await + .expect("initial generation prepares"); + let manager = Arc::new(RuntimeManager::new( + candidate.transfer().expect("generation transfers"), + )); + process + .install_initial_tasks((*manager).clone(), &config) + .await + .expect("initial tasks install"); + let runtime = ServerRuntime::new(process, Arc::clone(&manager), config); + (directory, database, runtime) +} + async fn runtime_fixture_with_generation_components( max_request_body_bytes: u64, ) -> ( @@ -207,6 +250,51 @@ async fn dashboard_timeseries_routes_preserve_auth_and_validate_periods() { .expect("database close is idempotent"); } +#[tokio::test] +async fn short_window_fairness_rejects_before_dispatch_and_reports_retry_scope() { + let (_directory, database, runtime) = + runtime_fixture_with_short_window_limit(Some(1), None).await; + let handle = runtime.handle(); + let listener = TcpListener::bind(("127.0.0.1", 0)) + .await + .expect("bind listener"); + let address = listener.local_addr().expect("listener address"); + let task = tokio::spawn(async move { runtime.serve_listener(listener).await }); + let body = br#"{"model":"missing-model","messages":[]}"#; + let raw = format!( + "POST /v1/chat/completions HTTP/1.1\r\nHost: localhost\r\nX-API-Key: test-key-transport\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{}", + body.len(), + std::str::from_utf8(body).expect("fixture JSON") + ); + let admitted = request(address, raw.as_bytes()).await; + assert!(admitted.starts_with("HTTP/1.1 503"), "{admitted}"); + assert!( + admitted.contains("x-eggpool-admission-scope: principal"), + "{admitted}" + ); + assert!( + admitted.contains("x-eggpool-admission-remaining: 0"), + "{admitted}" + ); + + let limited_without_body = format!( + "POST /v1/chat/completions HTTP/1.1\r\nHost: localhost\r\nX-API-Key: test-key-transport\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n", + body.len() + ); + let limited = request(address, limited_without_body.as_bytes()).await; + assert!(limited.starts_with("HTTP/1.1 429"), "{limited}"); + assert!(limited.contains("retry-after: 60"), "{limited}"); + assert!( + limited.contains("x-eggpool-admission-scope: principal"), + "{limited}" + ); + assert!(limited.contains("short_window_rate_limited"), "{limited}"); + + handle.request_shutdown(ShutdownReason::Requested); + let _ = task.await; + database.close().await.expect("database closes"); +} + async fn streaming_fixture_provider( listener: TcpListener, stopped: oneshot::Sender<()>, diff --git a/tests/fixtures/runtime/compatibility-observations.json b/tests/fixtures/runtime/compatibility-observations.json index 96e3452c..8ca7d1c1 100644 --- a/tests/fixtures/runtime/compatibility-observations.json +++ b/tests/fixtures/runtime/compatibility-observations.json @@ -1 +1 @@ -{"active_authority":{"active_state_after_shutdown":"no active generation is acquirable","app_state_rule":"mirrors are compatibility snapshots, never authority for leased request work","generation_dataclass_fields":["generation_id","config","config_digest","registry","catalog","router","coordinator","client_pool","outbound_manager","health_manager","cost_calculator","transcoder_policy","dispatch_overhead_recorder","dispatch_span_recorder","account_backoff_repo","stats_service","supervisor","routing_trace_guard","routing_trace_writer","created_at_monotonic","created_at_epoch","effects_applier","model_quarantine","finalization_supervisor","model_info","model_router_registry","local_pre_upstream_recorder","stream_diagnostics","immutable_request_state","generation_cleanup_callbacks"],"generation_owned_app_state_mirrors":["account_backoff_repo","catalog","client_pool","cost_calculator","dashboard_telemetry","dispatch_overhead_recorder","dispatch_span_recorder","health_manager","httpx_client","local_pre_upstream_recorder","metrics_coalescer","model_info","model_router_registry","outbound_manager","registry","router","routing_trace_guard","runtime_metrics","stats","stream_diagnostics","supervisor","task_monitor","transcoder_policy"],"live_request_authority":["providers","accounts","model_routers","model_overrides","model_capabilities","server.max_request_body_bytes","models.expose_mode","models.collapse_models","models.stale_after_s","models.allow_stale_catalog","routing.* request/retry/health/fairness settings","routing.wire_negotiation.*","transcoder.*","security.persist_redacted_error_detail","task intervals and retention settings consumed per tick"],"process_runtime_fields":["db","stats_db","config_path","metrics_coalescer","process_supervisor","task_spec_version","last_task_transition","routing_trace_writer","maintenance_state","event_loop_lag_monitor","readiness_probe","wire_profile_resolver","model_router_affinity","model_router_metrics","model_info"],"request_generation_rule":"one generation from admission through finite terminal registration or stream disconnect","restart_required_constructor_authority":["server.host/port/api_key/api_key_env/log_level/access_log/threads","upstream transport settings","database constructor and path settings","dashboard route topology and public/auth mode","security middleware settings","readiness probe constructor settings","listener and route-topology state"]},"config_policy":{"change_order":"lexicographic path order","default_unknown_disposition":"restart_required","disposition_values":["live","restart_required","ignored"],"dynamic_rules":[{"disposition":"live","pattern":"providers."},{"disposition":"live","pattern":"accounts./"},{"disposition":"live","pattern":"model_overrides."},{"disposition":"live","pattern":"model_capabilities."},{"disposition":"live","pattern":"transcoder."},{"disposition":"restart_required","pattern":"cache."},{"disposition":"live","pattern":"models."}],"dynamic_values_are_collapsed":true,"field_dispositions":[{"disposition":"live","path":"accounts"},{"disposition":"restart_required","path":"backup.directory"},{"disposition":"live","path":"backup.enabled"},{"disposition":"restart_required","path":"backup.include_env"},{"disposition":"live","path":"backup.interval_s"},{"disposition":"live","path":"backup.retain_count"},{"disposition":"live","path":"backup.startup_delay_s"},{"disposition":"restart_required","path":"dashboard.enabled"},{"disposition":"restart_required","path":"dashboard.public"},{"disposition":"restart_required","path":"dashboard.refresh_interval_s"},{"disposition":"live","path":"dashboard.retain_event_days"},{"disposition":"live","path":"dashboard.retain_request_stats_days"},{"disposition":"restart_required","path":"dashboard.store_request_content"},{"disposition":"restart_required","path":"dashboard.theme"},{"disposition":"live","path":"dashboard.themes_dir"},{"disposition":"restart_required","path":"database.busy_timeout_ms"},{"disposition":"restart_required","path":"database.journal_size_limit"},{"disposition":"restart_required","path":"database.path"},{"disposition":"restart_required","path":"database.synchronous"},{"disposition":"restart_required","path":"database.wal"},{"disposition":"restart_required","path":"database.worker_threads"},{"disposition":"live","path":"integrations.advertise_base_url"},{"disposition":"restart_required","path":"limits.five_hour_microdollars"},{"disposition":"restart_required","path":"limits.monthly_microdollars"},{"disposition":"restart_required","path":"limits.weekly_microdollars"},{"disposition":"live","path":"maintenance.contention_defer_above_lock_wait_p95_ms"},{"disposition":"live","path":"maintenance.max_batches_per_tick"},{"disposition":"live","path":"maintenance.max_deferral_age_s"},{"disposition":"live","path":"maintenance.max_rows_per_batch"},{"disposition":"live","path":"maintenance.max_tick_duration_ms"},{"disposition":"live","path":"maintenance.p0_max_batches_per_tick"},{"disposition":"live","path":"maintenance.p0_max_rows_per_batch"},{"disposition":"live","path":"maintenance.p0_max_tick_duration_ms"},{"disposition":"restart_required","path":"metrics.aggregate_only"},{"disposition":"restart_required","path":"metrics.cleanup_interval_s"},{"disposition":"restart_required","path":"metrics.cleanup_max_rows_per_pass"},{"disposition":"live","path":"metrics.detailed_span_sample_rate"},{"disposition":"live","path":"metrics.dispatch_spans.sample_rate"},{"disposition":"restart_required","path":"metrics.dispatch_spans.window_size"},{"disposition":"restart_required","path":"metrics.event_loop_lag_enabled"},{"disposition":"live","path":"metrics.flush_interval_s"},{"disposition":"restart_required","path":"metrics.max_buffered_events"},{"disposition":"live","path":"metrics.operational_event_retain_days"},{"disposition":"restart_required","path":"metrics.rollup_retain_days"},{"disposition":"live","path":"metrics.routing_decision_retain_days"},{"disposition":"restart_required","path":"metrics.timeseries_bucket_s"},{"disposition":"restart_required","path":"metrics.trace_sample_rate"},{"disposition":"restart_required","path":"metrics.write_mode"},{"disposition":"live","path":"model_capabilities"},{"disposition":"restart_required","path":"model_info.aliases"},{"disposition":"restart_required","path":"model_info.conflict_ttl_s"},{"disposition":"live","path":"model_info.enabled"},{"disposition":"restart_required","path":"model_info.include_in_models_endpoint"},{"disposition":"restart_required","path":"model_info.known_ttl_s"},{"disposition":"restart_required","path":"model_info.max_models_per_cycle"},{"disposition":"restart_required","path":"model_info.overrides"},{"disposition":"restart_required","path":"model_info.partial_ttl_s"},{"disposition":"restart_required","path":"model_info.refresh_interval_s"},{"disposition":"restart_required","path":"model_info.sources"},{"disposition":"restart_required","path":"model_info.sparse_new_accelerated_days"},{"disposition":"restart_required","path":"model_info.sparse_new_initial_ttl_s"},{"disposition":"restart_required","path":"model_info.sparse_new_later_ttl_s"},{"disposition":"restart_required","path":"model_info.startup_refresh"},{"disposition":"restart_required","path":"model_info.store_raw_observations"},{"disposition":"live","path":"model_overrides"},{"disposition":"live","path":"model_routers"},{"disposition":"live","path":"models.allow_stale_catalog"},{"disposition":"restart_required","path":"models.catalog_withdrawal_policy"},{"disposition":"live","path":"models.collapse_models"},{"disposition":"live","path":"models.expose_mode"},{"disposition":"live","path":"models.ping_retain_days"},{"disposition":"live","path":"models.refresh_interval_s"},{"disposition":"live","path":"models.stale_after_s"},{"disposition":"restart_required","path":"models.startup_refresh"},{"disposition":"restart_required","path":"network.connect_timeout_s"},{"disposition":"restart_required","path":"network.keepalive_expiry_s"},{"disposition":"restart_required","path":"network.max_connections"},{"disposition":"restart_required","path":"network.max_keepalive"},{"disposition":"restart_required","path":"network.read_timeout_s"},{"disposition":"restart_required","path":"pricing.catalogs.aliases"},{"disposition":"restart_required","path":"pricing.catalogs.opencode_zen.api_key"},{"disposition":"restart_required","path":"pricing.catalogs.opencode_zen.base_url"},{"disposition":"restart_required","path":"pricing.catalogs.opencode_zen.enabled"},{"disposition":"restart_required","path":"pricing.catalogs.opencode_zen.max_entries"},{"disposition":"restart_required","path":"pricing.catalogs.opencode_zen.options"},{"disposition":"restart_required","path":"pricing.catalogs.opencode_zen.priority"},{"disposition":"restart_required","path":"pricing.catalogs.opencode_zen.ttl_seconds"},{"disposition":"restart_required","path":"pricing.catalogs.openrouter.api_key"},{"disposition":"restart_required","path":"pricing.catalogs.openrouter.base_url"},{"disposition":"restart_required","path":"pricing.catalogs.openrouter.enabled"},{"disposition":"restart_required","path":"pricing.catalogs.openrouter.max_entries"},{"disposition":"restart_required","path":"pricing.catalogs.openrouter.options"},{"disposition":"restart_required","path":"pricing.catalogs.openrouter.priority"},{"disposition":"restart_required","path":"pricing.catalogs.openrouter.ttl_seconds"},{"disposition":"restart_required","path":"pricing.fallback"},{"disposition":"live","path":"providers"},{"disposition":"restart_required","path":"proxies"},{"disposition":"restart_required","path":"readiness_probe.enabled"},{"disposition":"restart_required","path":"readiness_probe.freshness_s"},{"disposition":"restart_required","path":"readiness_probe.initial_probe"},{"disposition":"restart_required","path":"readiness_probe.interval_s"},{"disposition":"restart_required","path":"readiness_probe.timeout_s"},{"disposition":"live","path":"routing.fairness_epsilon"},{"disposition":"live","path":"routing.fairness_mode"},{"disposition":"live","path":"routing.fairness_scope"},{"disposition":"live","path":"routing.health_penalty"},{"disposition":"live","path":"routing.inflight_penalty"},{"disposition":"live","path":"routing.local_quota_mode"},{"disposition":"live","path":"routing.max_retries_before_stream"},{"disposition":"live","path":"routing.near_tie_epsilon"},{"disposition":"live","path":"routing.quota_exhausted_cooldown_seconds"},{"disposition":"live","path":"routing.randomize_near_ties"},{"disposition":"live","path":"routing.strategy"},{"disposition":"restart_required","path":"routing.trace.flush_interval_s"},{"disposition":"live","path":"routing.trace.guard_cooldown_s"},{"disposition":"live","path":"routing.trace.guard_oldest_event_age_s"},{"disposition":"live","path":"routing.trace.guard_queue_occupancy_threshold"},{"disposition":"live","path":"routing.trace.include_score_components"},{"disposition":"restart_required","path":"routing.trace.max_batch_size"},{"disposition":"live","path":"routing.trace.mode"},{"disposition":"restart_required","path":"routing.trace.queue_capacity"},{"disposition":"live","path":"routing.trace.sample_rate"},{"disposition":"restart_required","path":"routing.trace.shutdown_flush_timeout_s"},{"disposition":"live","path":"routing.trace.skip_above_lock_wait_p95_ms"},{"disposition":"live","path":"routing.unknown_request_reservation_microdollars"},{"disposition":"live","path":"routing.wire_negotiation.cache_max_entries"},{"disposition":"live","path":"routing.wire_negotiation.enabled"},{"disposition":"live","path":"routing.wire_negotiation.learned_preference_ttl_s"},{"disposition":"live","path":"routing.wire_negotiation.max_concurrent_per_provider"},{"disposition":"live","path":"routing.wire_negotiation.min_negotiation_interval_s"},{"disposition":"live","path":"routing.wire_negotiation.rejection_cooldown_s"},{"disposition":"restart_required","path":"security.allowed_hosts"},{"path":"security.allow_insecure_web_auth","disposition":"restart_required"},{"disposition":"restart_required","path":"security.cors_origins"},{"disposition":"live","path":"security.persist_redacted_error_detail"},{"disposition":"restart_required","path":"security.redact_headers"},{"disposition":"restart_required","path":"security.trusted_proxies"},{"disposition":"restart_required","path":"server.access_log"},{"disposition":"restart_required","path":"server.api_key"},{"disposition":"restart_required","path":"server.api_key_env"},{"disposition":"restart_required","path":"server.host"},{"disposition":"restart_required","path":"server.log_level"},{"disposition":"live","path":"server.max_request_body_bytes"},{"disposition":"restart_required","path":"server.port"},{"disposition":"restart_required","path":"server.threads"},{"disposition":"live","path":"transcoder"},{"disposition":"restart_required","path":"update_checker.enabled"},{"disposition":"restart_required","path":"upstream.base_url"},{"disposition":"restart_required","path":"upstream.connect_timeout_s"},{"disposition":"restart_required","path":"upstream.keepalive_timeout_s"},{"disposition":"restart_required","path":"upstream.max_connections"},{"disposition":"restart_required","path":"upstream.max_keepalive"},{"disposition":"restart_required","path":"upstream.pool_timeout_s"},{"disposition":"restart_required","path":"upstream.read_timeout_s"},{"disposition":"restart_required","path":"upstream.write_timeout_s"}],"free_text_redaction_probe":"Authorization: ","free_text_redaction_token":"","ignored_paths":[],"runtime_tracked_default_paths":["backup.directory","backup.enabled","backup.include_env","backup.interval_s","backup.retain_count","backup.startup_delay_s","dashboard.enabled","dashboard.public","dashboard.refresh_interval_s","dashboard.retain_event_days","dashboard.retain_request_stats_days","dashboard.store_request_content","dashboard.theme","dashboard.themes_dir","database.busy_timeout_ms","database.journal_size_limit","database.path","database.synchronous","database.wal","database.worker_threads","integrations.advertise_base_url","limits.five_hour_microdollars","limits.monthly_microdollars","limits.weekly_microdollars","maintenance.contention_defer_above_lock_wait_p95_ms","maintenance.max_batches_per_tick","maintenance.max_deferral_age_s","maintenance.max_rows_per_batch","maintenance.max_tick_duration_ms","maintenance.p0_max_batches_per_tick","maintenance.p0_max_rows_per_batch","maintenance.p0_max_tick_duration_ms","metrics.aggregate_only","metrics.cleanup_interval_s","metrics.cleanup_max_rows_per_pass","metrics.detailed_span_sample_rate","metrics.dispatch_spans.sample_rate","metrics.dispatch_spans.window_size","metrics.event_loop_lag_enabled","metrics.flush_interval_s","metrics.max_buffered_events","metrics.operational_event_retain_days","metrics.rollup_retain_days","metrics.routing_decision_retain_days","metrics.timeseries_bucket_s","metrics.trace_sample_rate","metrics.write_mode","model_capabilities","model_info.aliases","model_info.conflict_ttl_s","model_info.enabled","model_info.include_in_models_endpoint","model_info.known_ttl_s","model_info.max_models_per_cycle","model_info.overrides","model_info.partial_ttl_s","model_info.refresh_interval_s","model_info.sources","model_info.sparse_new_accelerated_days","model_info.sparse_new_initial_ttl_s","model_info.sparse_new_later_ttl_s","model_info.startup_refresh","model_info.store_raw_observations","model_overrides","model_routers","models.allow_stale_catalog","models.catalog_withdrawal_policy","models.collapse_models","models.expose_mode","models.ping_retain_days","models.refresh_interval_s","models.stale_after_s","models.startup_refresh","network.connect_timeout_s","network.keepalive_expiry_s","network.max_connections","network.max_keepalive","network.read_timeout_s","pricing.catalogs.aliases","pricing.catalogs.opencode_zen.api_key","pricing.catalogs.opencode_zen.base_url","pricing.catalogs.opencode_zen.enabled","pricing.catalogs.opencode_zen.max_entries","pricing.catalogs.opencode_zen.options","pricing.catalogs.opencode_zen.priority","pricing.catalogs.opencode_zen.ttl_seconds","pricing.catalogs.openrouter.api_key","pricing.catalogs.openrouter.base_url","pricing.catalogs.openrouter.enabled","pricing.catalogs.openrouter.max_entries","pricing.catalogs.openrouter.options","pricing.catalogs.openrouter.priority","pricing.catalogs.openrouter.ttl_seconds","pricing.fallback","proxies","readiness_probe.enabled","readiness_probe.freshness_s","readiness_probe.initial_probe","readiness_probe.interval_s","readiness_probe.timeout_s","routing.fairness_epsilon","routing.fairness_mode","routing.fairness_scope","routing.health_penalty","routing.inflight_penalty","routing.local_quota_mode","routing.max_retries_before_stream","routing.near_tie_epsilon","routing.quota_exhausted_cooldown_seconds","routing.randomize_near_ties","routing.strategy","routing.trace.flush_interval_s","routing.trace.guard_cooldown_s","routing.trace.guard_oldest_event_age_s","routing.trace.guard_queue_occupancy_threshold","routing.trace.include_score_components","routing.trace.max_batch_size","routing.trace.mode","routing.trace.queue_capacity","routing.trace.sample_rate","routing.trace.shutdown_flush_timeout_s","routing.trace.skip_above_lock_wait_p95_ms","routing.unknown_request_reservation_microdollars","routing.wire_negotiation.cache_max_entries","routing.wire_negotiation.enabled","routing.wire_negotiation.learned_preference_ttl_s","routing.wire_negotiation.max_concurrent_per_provider","routing.wire_negotiation.min_negotiation_interval_s","routing.wire_negotiation.rejection_cooldown_s","security.allowed_hosts","security.cors_origins","security.persist_redacted_error_detail","security.redact_headers","security.trusted_proxies","server.access_log","server.api_key","server.api_key_env","server.host","server.log_level","server.max_request_body_bytes","server.port","server.threads","transcoder","update_checker.enabled","upstream.base_url","upstream.connect_timeout_s","upstream.keepalive_timeout_s","upstream.max_connections","upstream.max_keepalive","upstream.pool_timeout_s","upstream.read_timeout_s","upstream.write_timeout_s"],"secret_display_token":"","secret_field_names":["api_key","api_key_env","proxy_url","proxy_url_env"],"top_level_config_sections":["accounts","backup","dashboard","database","integrations","limits","maintenance","metrics","model_capabilities","model_info","model_overrides","model_routers","models","network","pricing","providers","proxies","readiness_probe","routing","security","server","transcoder","update_checker","upstream"],"unknown_path_probe":"restart_required"},"lifecycle":{"candidate":{"abort":{"abort_is_idempotent":true,"cleanup_errors_preserve_primary_failure":true,"closed_order":["supervisor","outbound_manager","client_pool"],"primary_failure_stage":"build","process_owned_resources_touched":false,"registered_order":["client_pool","outbound_manager","supervisor"]},"legal_transitions":[["building","prepared"],["prepared","transferred"],["building","aborted"],["prepared","aborted"]],"states":["building","prepared","transferred","aborted"]},"close_order":["generation supervisor stop_all","generation cleanup callbacks","finalization supervisor shutdown","provider client pool close","outbound client manager aclose"],"generation_slot":{"legal_transitions":[["active","retiring"],["retiring","closing"],["closing","closed"],["closing","failed_close"]],"retirement_ready_when":["active_leases == 0","terminal_references == 0"],"states":["active","retiring","closing","closed","failed_close"]},"lease":{"claim_and_increment_are_one_critical_section":true,"claim_predicate":"shutdown_in_progress OR (NOT lease_admission_gated AND active_slot_accepting)","claim_timeout_s":30.0,"release_is_exactly_once":true,"stream_release_boundary":"iterator finally after handler return"},"pending_swap":{"commit_effects":["candidate_becomes_active","old_slot_stops_accepting_leases","publication_epoch_increments","lease_admission_reopens"],"legal_transitions":[["prepared","staged"],["staged","committed"],["staged","rolled_back"],["committed","finalized"]],"rollback_effects":["old_slot_restored","candidate_slot_dropped","lease_admission_reopens","candidate_requires_abort"],"stage_effects":["lease_admission_gated","candidate_slot_non_accepting","active_slot_unchanged"],"states":["prepared","staged","committed","rolled_back","finalized"]},"publication_gate":{"forbidden_inside_gate":["provider/network calls","catalog refresh","backup","unbounded work"],"inside_gate":["stage candidate","apply bounded mandatory acceptance changes","commit runtime pointer","commit task-spec state"],"pre_gate_work":["config parsing and validation","digest check and diff","candidate construction","persistence delta preparation","task diff preflight"]},"shutdown":{"adopts_unfinalized_committed_swap_old_slot":true,"forced_close_abandons_terminal_references_only_on_process_shutdown":true,"live_rehash_preserves_accepted_work":true,"manager_join_deadline_s":10.0,"retirement_drain_deadline_s":5.0,"stops_new_publication_and_acquisition":true},"transaction":{"acceptance_states":["not_accepted","persistence_committed_runtime_pending","accepted"],"accepted_reload_cannot_abort":true,"states":["created","validated","diffed","candidate_prepared","persistence_prepared","process_transitions_prepared","process_transitions_preflighted","commit_started","runtime_staged","runtime_swap_committed","runtime_published","process_transitions_applied","persistence_committed","observable_state_updated","retirement_scheduled","completed","aborting","aborted","compensation_failed"],"transitions":{"aborted":[],"aborting":["aborted","compensation_failed"],"candidate_prepared":["aborting","persistence_prepared"],"commit_started":["aborting","runtime_published","runtime_staged"],"compensation_failed":[],"completed":[],"created":["aborting","validated"],"diffed":["aborting","candidate_prepared"],"observable_state_updated":["aborting","retirement_scheduled"],"persistence_committed":["aborting","observable_state_updated"],"persistence_prepared":["aborting","process_transitions_prepared"],"process_transitions_applied":["aborting","persistence_committed"],"process_transitions_preflighted":["aborting","commit_started"],"process_transitions_prepared":["aborting","process_transitions_preflighted"],"retirement_scheduled":["aborting","completed"],"runtime_published":["aborting","process_transitions_applied"],"runtime_staged":["aborting","runtime_swap_committed"],"runtime_swap_committed":["aborting","process_transitions_applied"],"validated":["aborting","diffed"]}}},"normalization":{"excluded":["API keys","proxy credentials","raw config values","request/provider bodies"],"normalized":["timestamps","runtime identifiers"],"preserved":["state names and order","booleans","changed/restart path ordering","task names and specs","generation increments","retirement pending state","error category distinctions"]},"oracle_modules":["eggpool.runtime_manager","eggpool.generation_factory","eggpool.config_reload_policy","eggpool.reload_transaction","eggpool.runtime_task_inventory","eggpool.runtime_tasks","eggpool.reload_diagnostics","eggpool.app","eggpool.cli_rehash_helper"],"ownership_inventory":[{"authority":"ProcessRuntime.db/stats_db","lifetime":"process","name":"database_and_repositories","ownership":"process-owned"},{"authority":"RuntimeGeneration.client_pool","lifetime":"generation retirement","name":"provider_client_pool","ownership":"generation-owned"},{"authority":"RuntimeGeneration.outbound_manager","lifetime":"generation retirement","name":"outbound_client_manager","ownership":"generation-owned"},{"authority":"RuntimeGeneration.registry","lifetime":"generation retirement","name":"account_registry","ownership":"generation-owned"},{"authority":"RuntimeGeneration.catalog","lifetime":"generation retirement","name":"catalog_service","ownership":"generation-owned"},{"authority":"RuntimeGeneration.router/coordinator","lifetime":"generation retirement","name":"router_and_coordinator","ownership":"generation-owned"},{"authority":"RuntimeGeneration.finalization_supervisor","lifetime":"after leases and terminal references drain","name":"finalization_supervisor","ownership":"generation-owned"},{"authority":"ProcessRuntime.model_router_affinity","lifetime":"process; revalidated per generation","name":"model_router_affinity","ownership":"process-owned"},{"authority":"ProcessRuntime.wire_profile_resolver","lifetime":"process; fingerprint-qualified","name":"wire_profile_resolver","ownership":"process-owned"},{"authority":"RuntimeGeneration.supervisor","lifetime":"generation retirement","name":"generation_task_supervisor","ownership":"generation-owned"},{"authority":"ProcessRuntime.process_supervisor","lifetime":"process singleton","name":"process_task_supervisor","ownership":"process-owned"},{"authority":"startup lifecycle entry point","lifetime":"one bounded startup pass","name":"crash_reconciler","ownership":"process-owned/startup-only"},{"authority":"RuntimeMetricsService and reload metadata","lifetime":"process; bounded snapshots","name":"runtime_metrics_and_reload_diagnostics","ownership":"process-owned"},{"authority":"listener, middleware, route topology, auth, DB constructor","lifetime":"process construction","name":"server_constructor_state","ownership":"startup-only/restart-required"},{"authority":"RuntimeGenerationCandidate","lifetime":"building/prepared until transfer or abort","name":"candidate_resources_before_publication","ownership":"candidate-owned"},{"authority":"GenerationLease","lifetime":"finite completion or stream disconnect","name":"request_and_stream_lease","ownership":"request/lease-owned"},{"authority":"terminal reference on generation slot","lifetime":"durable/runtime convergence","name":"retained_finalization","ownership":"retained-finalization-owned"}],"reload":{"cases":[{"accepted":true,"active_generation_unchanged":true,"category":"success_noop","changed_sections":[],"generation_delta":0,"legacy_wire_stage":"commit","name":"identical_digest_noop","publication_occurred":false,"restart_required_paths":[],"retirement_pending":false,"terminal_stage":"idle"},{"accepted":true,"active_generation_unchanged":false,"category":"success_committed","changed_sections":["server"],"generation_delta":1,"legacy_wire_stage":"retirement","name":"valid_live_only_change","publication_occurred":true,"restart_required_paths":[],"retirement_pending":"true_if_old_work_remains","terminal_stage":"retirement"},{"accepted":false,"active_generation_unchanged":true,"category":"rejected_restart_required","changed_sections":["server"],"generation_delta":0,"legacy_wire_stage":"diff","name":"restart_required_change","publication_occurred":false,"restart_required_paths":["server.port"],"retirement_pending":false,"terminal_stage":"diff"},{"accepted":false,"active_generation_unchanged":true,"category":"rejected_restart_required","changed_sections":["server"],"generation_delta":0,"legacy_wire_stage":"diff","name":"mixed_live_and_restart_required_change","publication_occurred":false,"restart_required_paths":["server.port"],"retirement_pending":false,"terminal_stage":"diff"},{"accepted":false,"active_generation_unchanged":true,"category":"rejected_validation","changed_sections":[],"generation_delta":0,"legacy_wire_stage":"validation","name":"invalid_config","publication_occurred":false,"restart_required_paths":[],"retirement_pending":false,"terminal_stage":"validation"},{"accepted":false,"active_generation_unchanged":true,"category":"rejected_validation","changed_sections":[],"generation_delta":0,"legacy_wire_stage":"digest_check","name":"expected_digest_mismatch","publication_occurred":false,"restart_required_paths":[],"retirement_pending":false,"terminal_stage":"validation"},{"accepted":false,"active_generation_unchanged":true,"category":"rejected_validation","changed_sections":[],"generation_delta":0,"legacy_wire_stage":"preparation","name":"stale_caller_generation","publication_occurred":false,"restart_required_paths":[],"retirement_pending":false,"terminal_stage":"preparation"},{"accepted":false,"active_generation_unchanged":true,"category":"failed_candidate_prepare","changed_sections":[],"generation_delta":0,"legacy_wire_stage":"preparation","name":"candidate_construction_failure","publication_occurred":false,"restart_required_paths":[],"retirement_pending":false,"terminal_stage":"preparation"},{"accepted":false,"active_generation_unchanged":true,"category":"failed_process_transition_prepare","changed_sections":[],"generation_delta":0,"legacy_wire_stage":"preparation","name":"preflight_process_transition_failure","publication_occurred":false,"restart_required_paths":[],"retirement_pending":false,"terminal_stage":"preparation"},{"accepted":false,"active_generation_unchanged":true,"category":"aborted_cancelled","changed_sections":[],"generation_delta":0,"legacy_wire_stage":"preparation","name":"cancellation_before_acceptance","publication_occurred":false,"restart_required_paths":[],"retirement_pending":false,"terminal_stage":"preparation"},{"accepted":true,"active_generation_unchanged":false,"category":"post_commit_finalization_pending","changed_sections":[],"generation_delta":1,"legacy_wire_stage":"retirement","name":"cancellation_during_or_after_acceptance","publication_occurred":true,"restart_required_paths":[],"retirement_pending":true,"terminal_stage":"retirement"},{"accepted":true,"active_generation_unchanged":false,"category":"post_commit_finalization_pending","changed_sections":[],"generation_delta":1,"legacy_wire_stage":"retirement","name":"successful_publication_retirement_pending","publication_occurred":true,"restart_required_paths":[],"retirement_pending":true,"terminal_stage":"retirement"},{"accepted":true,"active_generation_unchanged":false,"category":"success_committed","changed_sections":[],"generation_delta":1,"legacy_wire_stage":"retirement","name":"successful_publication_already_drained","publication_occurred":true,"restart_required_paths":[],"retirement_pending":false,"terminal_stage":"retirement"}],"classification_probes":{"retry_pending":"post_commit_finalization_pending","success_committed":"success_committed","success_noop":"success_noop","validation":"rejected_validation"},"diff_observations":{"identical":{"changes":[],"dispositions":[],"live":[],"paths":[],"restart_required":[],"sections":[]},"live_only":{"changes":[{"disposition":"live","new_display":"10485761","old_display":"10485760","path":"server.max_request_body_bytes","secret":false,"section":"server"}],"dispositions":["live"],"live":["server.max_request_body_bytes"],"paths":["server.max_request_body_bytes"],"restart_required":[],"sections":["server"]},"mixed":{"changes":[{"disposition":"live","new_display":"10485761","old_display":"10485760","path":"server.max_request_body_bytes","secret":false,"section":"server"},{"disposition":"restart_required","new_display":"11301","old_display":"11300","path":"server.port","secret":false,"section":"server"}],"dispositions":["live","restart_required"],"live":["server.max_request_body_bytes"],"paths":["server.max_request_body_bytes","server.port"],"restart_required":["server.port"],"sections":["server"]},"restart_only":{"changes":[{"disposition":"restart_required","new_display":"11301","old_display":"11300","path":"server.port","secret":false,"section":"server"}],"dispositions":["restart_required"],"live":[],"paths":["server.port"],"restart_required":["server.port"],"sections":["server"]},"secret":{"changes":[{"disposition":"restart_required","new_display":"","old_display":"","path":"pricing.catalogs.openrouter.api_key","secret":true,"section":"pricing"}],"dispositions":["restart_required"],"live":[],"paths":["pricing.catalogs.openrouter.api_key"],"restart_required":["pricing.catalogs.openrouter.api_key"],"sections":["pricing"]}},"generation_id_rule":"monotonic; no-op and rejected reloads do not publish","legacy_stages":["validation","digest_check","diff","preparation","reconciliation","commit","retirement"],"result_categories":["success_committed","success_noop","success_ignored_only","rejected_busy","rejected_validation","rejected_restart_required","failed_candidate_prepare","failed_persistence_prepare","failed_process_transition_prepare","failed_commit","failed_publication","failed_process_transition_apply","failed_persistence_commit","failed_runtime_stage","failed_runtime_stage_rollback","post_commit_finalization_pending","retirement_schedule_failed","shutdown_adopted","aborted_cancelled","aborted_shutdown","compensation_failed","internal_error"],"retirement_pending_rule":"accepted publication may be pending while old leases or terminal references drain","terminal_stages":["validation","diff","preparation","reconciliation","commit","retirement","idle"]},"schema_version":"m8-runtime-lifecycle-r001-observations/v1","tasks":{"conditions":{"automatic_backup":"backup.enabled and interval_s > 0 enables","catalog_refresh":"models.refresh_interval_s <= 0 disables","generation_leased_callbacks":"acquire active generation for every tick","metrics_flush":"metrics.write_mode == immediate disables","retention_cleanup":"always registered; reads active generation config per tick","update_checker":"startup-only process task; update_checker.enabled and outbound handle required"},"config_variants":{"automatic_backup_enabled":[{"enabled":true,"initial_delay_s":null,"interval_s":300.0,"name":"catalog_refresh","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":86400.0,"name":"retention_cleanup","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":60.0,"name":"checkpoint","run_immediately":true},{"enabled":true,"initial_delay_s":5.0,"interval_s":120.0,"name":"metrics_flush","run_immediately":false},{"enabled":false,"initial_delay_s":null,"interval_s":86400.0,"name":"update_checker","run_immediately":true},{"enabled":true,"initial_delay_s":5.0,"interval_s":600.0,"name":"automatic_backup","run_immediately":false}],"catalog_refresh_disabled":[{"enabled":false,"initial_delay_s":null,"interval_s":0.0,"name":"catalog_refresh","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":86400.0,"name":"retention_cleanup","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":60.0,"name":"checkpoint","run_immediately":true},{"enabled":true,"initial_delay_s":5.0,"interval_s":120.0,"name":"metrics_flush","run_immediately":false},{"enabled":false,"initial_delay_s":null,"interval_s":86400.0,"name":"update_checker","run_immediately":true},{"enabled":false,"initial_delay_s":300.0,"interval_s":86400.0,"name":"automatic_backup","run_immediately":false}],"default":[{"enabled":true,"initial_delay_s":null,"interval_s":300.0,"name":"catalog_refresh","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":86400.0,"name":"retention_cleanup","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":60.0,"name":"checkpoint","run_immediately":true},{"enabled":true,"initial_delay_s":5.0,"interval_s":120.0,"name":"metrics_flush","run_immediately":false},{"enabled":false,"initial_delay_s":null,"interval_s":86400.0,"name":"update_checker","run_immediately":true},{"enabled":false,"initial_delay_s":300.0,"interval_s":86400.0,"name":"automatic_backup","run_immediately":false}],"metrics_immediate":[{"enabled":true,"initial_delay_s":null,"interval_s":300.0,"name":"catalog_refresh","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":86400.0,"name":"retention_cleanup","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":60.0,"name":"checkpoint","run_immediately":true},{"enabled":false,"initial_delay_s":5.0,"interval_s":120.0,"name":"metrics_flush","run_immediately":false},{"enabled":false,"initial_delay_s":null,"interval_s":86400.0,"name":"update_checker","run_immediately":true},{"enabled":false,"initial_delay_s":300.0,"interval_s":86400.0,"name":"automatic_backup","run_immediately":false}],"update_checker_enabled":[{"enabled":true,"initial_delay_s":null,"interval_s":300.0,"name":"catalog_refresh","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":86400.0,"name":"retention_cleanup","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":60.0,"name":"checkpoint","run_immediately":true},{"enabled":true,"initial_delay_s":5.0,"interval_s":120.0,"name":"metrics_flush","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":86400.0,"name":"update_checker","run_immediately":true},{"enabled":false,"initial_delay_s":300.0,"interval_s":86400.0,"name":"automatic_backup","run_immediately":false}]},"inventory":[{"callback_kind":"catalog_refresh","default_enabled":true,"description":"Periodically refresh the model catalog from enabled accounts","generation_dependencies":["catalog","model_info","registry","health_manager"],"initial_delay_s":null,"interval_s":300.0,"name":"catalog_refresh","ownership":"generation_leased","process_dependencies":["db"],"reloadable_fields":["models.refresh_interval_s"],"run_immediately":false,"timeout_s":null},{"callback_kind":"retention_cleanup","default_enabled":true,"description":"Clean up old requests, events, pings, and expired reservations","generation_dependencies":["router"],"initial_delay_s":null,"interval_s":86400.0,"name":"retention_cleanup","ownership":"generation_leased","process_dependencies":["db"],"reloadable_fields":["dashboard.retain_request_stats_days","dashboard.retain_event_days","metrics.operational_event_retain_days","metrics.routing_decision_retain_days","models.ping_retain_days"],"run_immediately":false,"timeout_s":null},{"callback_kind":"checkpoint","default_enabled":true,"description":"Periodic SQLite WAL checkpoint","generation_dependencies":[],"initial_delay_s":null,"interval_s":60.0,"name":"checkpoint","ownership":"process","process_dependencies":["db"],"reloadable_fields":[],"run_immediately":true,"timeout_s":null},{"callback_kind":"metrics_flush","default_enabled":true,"description":"Flush buffered metrics analytics to SQLite","generation_dependencies":[],"initial_delay_s":5.0,"interval_s":30.0,"name":"metrics_flush","ownership":"process","process_dependencies":["metrics_coalescer"],"reloadable_fields":["metrics.write_mode","metrics.flush_interval_s"],"run_immediately":false,"timeout_s":null},{"callback_kind":"update_checker","default_enabled":true,"description":"Periodically check PyPI for new EggPool releases","generation_dependencies":[],"initial_delay_s":null,"interval_s":86400.0,"name":"update_checker","ownership":"process","process_dependencies":["outbound_manager"],"reloadable_fields":[],"run_immediately":true,"timeout_s":null},{"callback_kind":"automatic_backup","default_enabled":true,"description":"Create periodic backup archives of config and database","generation_dependencies":[],"initial_delay_s":300.0,"interval_s":86400.0,"name":"automatic_backup","ownership":"process","process_dependencies":["db"],"reloadable_fields":["backup.enabled","backup.interval_s","backup.startup_delay_s","backup.retain_count"],"run_immediately":false,"timeout_s":null}],"names_unique":true,"singleton_process_tasks":["checkpoint","metrics_flush","update_checker","automatic_backup"]}} +{"active_authority":{"active_state_after_shutdown":"no active generation is acquirable","app_state_rule":"mirrors are compatibility snapshots, never authority for leased request work","generation_dataclass_fields":["generation_id","config","config_digest","registry","catalog","router","coordinator","client_pool","outbound_manager","health_manager","cost_calculator","transcoder_policy","dispatch_overhead_recorder","dispatch_span_recorder","account_backoff_repo","stats_service","supervisor","routing_trace_guard","routing_trace_writer","created_at_monotonic","created_at_epoch","effects_applier","model_quarantine","finalization_supervisor","model_info","model_router_registry","local_pre_upstream_recorder","stream_diagnostics","immutable_request_state","generation_cleanup_callbacks"],"generation_owned_app_state_mirrors":["account_backoff_repo","catalog","client_pool","cost_calculator","dashboard_telemetry","dispatch_overhead_recorder","dispatch_span_recorder","health_manager","httpx_client","local_pre_upstream_recorder","metrics_coalescer","model_info","model_router_registry","outbound_manager","registry","router","routing_trace_guard","runtime_metrics","stats","stream_diagnostics","supervisor","task_monitor","transcoder_policy"],"live_request_authority":["providers","accounts","admission_fairness.* short-window controls consumed at authenticated request admission","model_routers","model_overrides","model_capabilities","server.max_request_body_bytes","models.expose_mode","models.collapse_models","models.stale_after_s","models.allow_stale_catalog","routing.* request/retry/health/fairness settings","routing.wire_negotiation.*","transcoder.*","security.persist_redacted_error_detail","task intervals and retention settings consumed per tick"],"process_runtime_fields":["db","stats_db","config_path","metrics_coalescer","process_supervisor","task_spec_version","last_task_transition","routing_trace_writer","maintenance_state","event_loop_lag_monitor","readiness_probe","wire_profile_resolver","model_router_affinity","model_router_metrics","model_info"],"request_generation_rule":"one generation from admission through finite terminal registration or stream disconnect","restart_required_constructor_authority":["server.host/port/api_key/api_key_env/log_level/access_log/threads","upstream transport settings","database constructor and path settings","dashboard route topology and public/auth mode","security middleware settings","readiness probe constructor settings","listener and route-topology state"]},"config_policy":{"change_order":"lexicographic path order","default_unknown_disposition":"restart_required","disposition_values":["live","restart_required","ignored"],"dynamic_rules":[{"disposition":"live","pattern":"providers."},{"disposition":"live","pattern":"accounts./"},{"disposition":"live","pattern":"model_overrides."},{"disposition":"live","pattern":"model_capabilities."},{"disposition":"live","pattern":"transcoder."},{"disposition":"restart_required","pattern":"cache."},{"disposition":"live","pattern":"models."}],"dynamic_values_are_collapsed":true,"field_dispositions":[{"disposition":"live","path":"accounts"},{"disposition":"live","path":"admission_fairness.global_requests_per_minute"},{"disposition":"live","path":"admission_fairness.max_tracked_principals"},{"disposition":"live","path":"admission_fairness.requests_per_minute_per_principal"},{"disposition":"restart_required","path":"backup.directory"},{"disposition":"live","path":"backup.enabled"},{"disposition":"restart_required","path":"backup.include_env"},{"disposition":"live","path":"backup.interval_s"},{"disposition":"live","path":"backup.retain_count"},{"disposition":"live","path":"backup.startup_delay_s"},{"disposition":"restart_required","path":"dashboard.enabled"},{"disposition":"restart_required","path":"dashboard.public"},{"disposition":"restart_required","path":"dashboard.refresh_interval_s"},{"disposition":"live","path":"dashboard.retain_event_days"},{"disposition":"live","path":"dashboard.retain_request_stats_days"},{"disposition":"restart_required","path":"dashboard.store_request_content"},{"disposition":"restart_required","path":"dashboard.theme"},{"disposition":"live","path":"dashboard.themes_dir"},{"disposition":"restart_required","path":"database.busy_timeout_ms"},{"disposition":"restart_required","path":"database.journal_size_limit"},{"disposition":"restart_required","path":"database.path"},{"disposition":"restart_required","path":"database.synchronous"},{"disposition":"restart_required","path":"database.wal"},{"disposition":"restart_required","path":"database.worker_threads"},{"disposition":"live","path":"integrations.advertise_base_url"},{"disposition":"restart_required","path":"limits.five_hour_microdollars"},{"disposition":"restart_required","path":"limits.monthly_microdollars"},{"disposition":"restart_required","path":"limits.weekly_microdollars"},{"disposition":"live","path":"maintenance.contention_defer_above_lock_wait_p95_ms"},{"disposition":"live","path":"maintenance.max_batches_per_tick"},{"disposition":"live","path":"maintenance.max_deferral_age_s"},{"disposition":"live","path":"maintenance.max_rows_per_batch"},{"disposition":"live","path":"maintenance.max_tick_duration_ms"},{"disposition":"live","path":"maintenance.p0_max_batches_per_tick"},{"disposition":"live","path":"maintenance.p0_max_rows_per_batch"},{"disposition":"live","path":"maintenance.p0_max_tick_duration_ms"},{"disposition":"restart_required","path":"metrics.aggregate_only"},{"disposition":"restart_required","path":"metrics.cleanup_interval_s"},{"disposition":"restart_required","path":"metrics.cleanup_max_rows_per_pass"},{"disposition":"live","path":"metrics.detailed_span_sample_rate"},{"disposition":"live","path":"metrics.dispatch_spans.sample_rate"},{"disposition":"restart_required","path":"metrics.dispatch_spans.window_size"},{"disposition":"restart_required","path":"metrics.event_loop_lag_enabled"},{"disposition":"live","path":"metrics.flush_interval_s"},{"disposition":"restart_required","path":"metrics.max_buffered_events"},{"disposition":"live","path":"metrics.operational_event_retain_days"},{"disposition":"restart_required","path":"metrics.rollup_retain_days"},{"disposition":"live","path":"metrics.routing_decision_retain_days"},{"disposition":"restart_required","path":"metrics.timeseries_bucket_s"},{"disposition":"restart_required","path":"metrics.trace_sample_rate"},{"disposition":"restart_required","path":"metrics.write_mode"},{"disposition":"live","path":"model_capabilities"},{"disposition":"restart_required","path":"model_info.aliases"},{"disposition":"restart_required","path":"model_info.conflict_ttl_s"},{"disposition":"live","path":"model_info.enabled"},{"disposition":"restart_required","path":"model_info.include_in_models_endpoint"},{"disposition":"restart_required","path":"model_info.known_ttl_s"},{"disposition":"restart_required","path":"model_info.max_models_per_cycle"},{"disposition":"restart_required","path":"model_info.overrides"},{"disposition":"restart_required","path":"model_info.partial_ttl_s"},{"disposition":"restart_required","path":"model_info.refresh_interval_s"},{"disposition":"restart_required","path":"model_info.sources"},{"disposition":"restart_required","path":"model_info.sparse_new_accelerated_days"},{"disposition":"restart_required","path":"model_info.sparse_new_initial_ttl_s"},{"disposition":"restart_required","path":"model_info.sparse_new_later_ttl_s"},{"disposition":"restart_required","path":"model_info.startup_refresh"},{"disposition":"restart_required","path":"model_info.store_raw_observations"},{"disposition":"live","path":"model_overrides"},{"disposition":"live","path":"model_routers"},{"disposition":"live","path":"models.allow_stale_catalog"},{"disposition":"restart_required","path":"models.catalog_withdrawal_policy"},{"disposition":"live","path":"models.collapse_models"},{"disposition":"live","path":"models.expose_mode"},{"disposition":"live","path":"models.ping_retain_days"},{"disposition":"live","path":"models.refresh_interval_s"},{"disposition":"live","path":"models.stale_after_s"},{"disposition":"restart_required","path":"models.startup_refresh"},{"disposition":"restart_required","path":"network.connect_timeout_s"},{"disposition":"restart_required","path":"network.keepalive_expiry_s"},{"disposition":"restart_required","path":"network.max_connections"},{"disposition":"restart_required","path":"network.max_keepalive"},{"disposition":"restart_required","path":"network.read_timeout_s"},{"disposition":"restart_required","path":"pricing.catalogs.aliases"},{"disposition":"restart_required","path":"pricing.catalogs.opencode_zen.api_key"},{"disposition":"restart_required","path":"pricing.catalogs.opencode_zen.base_url"},{"disposition":"restart_required","path":"pricing.catalogs.opencode_zen.enabled"},{"disposition":"restart_required","path":"pricing.catalogs.opencode_zen.max_entries"},{"disposition":"restart_required","path":"pricing.catalogs.opencode_zen.options"},{"disposition":"restart_required","path":"pricing.catalogs.opencode_zen.priority"},{"disposition":"restart_required","path":"pricing.catalogs.opencode_zen.ttl_seconds"},{"disposition":"restart_required","path":"pricing.catalogs.openrouter.api_key"},{"disposition":"restart_required","path":"pricing.catalogs.openrouter.base_url"},{"disposition":"restart_required","path":"pricing.catalogs.openrouter.enabled"},{"disposition":"restart_required","path":"pricing.catalogs.openrouter.max_entries"},{"disposition":"restart_required","path":"pricing.catalogs.openrouter.options"},{"disposition":"restart_required","path":"pricing.catalogs.openrouter.priority"},{"disposition":"restart_required","path":"pricing.catalogs.openrouter.ttl_seconds"},{"disposition":"restart_required","path":"pricing.fallback"},{"disposition":"live","path":"providers"},{"disposition":"restart_required","path":"proxies"},{"disposition":"restart_required","path":"readiness_probe.enabled"},{"disposition":"restart_required","path":"readiness_probe.freshness_s"},{"disposition":"restart_required","path":"readiness_probe.initial_probe"},{"disposition":"restart_required","path":"readiness_probe.interval_s"},{"disposition":"restart_required","path":"readiness_probe.timeout_s"},{"disposition":"live","path":"routing.fairness_epsilon"},{"disposition":"live","path":"routing.fairness_mode"},{"disposition":"live","path":"routing.fairness_scope"},{"disposition":"live","path":"routing.health_penalty"},{"disposition":"live","path":"routing.inflight_penalty"},{"disposition":"live","path":"routing.local_quota_mode"},{"disposition":"live","path":"routing.max_retries_before_stream"},{"disposition":"live","path":"routing.near_tie_epsilon"},{"disposition":"live","path":"routing.quota_exhausted_cooldown_seconds"},{"disposition":"live","path":"routing.randomize_near_ties"},{"disposition":"live","path":"routing.strategy"},{"disposition":"restart_required","path":"routing.trace.flush_interval_s"},{"disposition":"live","path":"routing.trace.guard_cooldown_s"},{"disposition":"live","path":"routing.trace.guard_oldest_event_age_s"},{"disposition":"live","path":"routing.trace.guard_queue_occupancy_threshold"},{"disposition":"live","path":"routing.trace.include_score_components"},{"disposition":"restart_required","path":"routing.trace.max_batch_size"},{"disposition":"live","path":"routing.trace.mode"},{"disposition":"restart_required","path":"routing.trace.queue_capacity"},{"disposition":"live","path":"routing.trace.sample_rate"},{"disposition":"restart_required","path":"routing.trace.shutdown_flush_timeout_s"},{"disposition":"live","path":"routing.trace.skip_above_lock_wait_p95_ms"},{"disposition":"live","path":"routing.unknown_request_reservation_microdollars"},{"disposition":"live","path":"routing.wire_negotiation.cache_max_entries"},{"disposition":"live","path":"routing.wire_negotiation.enabled"},{"disposition":"live","path":"routing.wire_negotiation.learned_preference_ttl_s"},{"disposition":"live","path":"routing.wire_negotiation.max_concurrent_per_provider"},{"disposition":"live","path":"routing.wire_negotiation.min_negotiation_interval_s"},{"disposition":"live","path":"routing.wire_negotiation.rejection_cooldown_s"},{"disposition":"restart_required","path":"security.allowed_hosts"},{"path":"security.allow_insecure_web_auth","disposition":"restart_required"},{"disposition":"restart_required","path":"security.cors_origins"},{"disposition":"live","path":"security.persist_redacted_error_detail"},{"disposition":"restart_required","path":"security.redact_headers"},{"disposition":"restart_required","path":"security.trusted_proxies"},{"disposition":"restart_required","path":"server.access_log"},{"disposition":"restart_required","path":"server.api_key"},{"disposition":"restart_required","path":"server.api_key_env"},{"disposition":"restart_required","path":"server.host"},{"disposition":"restart_required","path":"server.log_level"},{"disposition":"live","path":"server.max_request_body_bytes"},{"disposition":"restart_required","path":"server.port"},{"disposition":"restart_required","path":"server.threads"},{"disposition":"live","path":"transcoder"},{"disposition":"restart_required","path":"update_checker.enabled"},{"disposition":"restart_required","path":"upstream.base_url"},{"disposition":"restart_required","path":"upstream.connect_timeout_s"},{"disposition":"restart_required","path":"upstream.keepalive_timeout_s"},{"disposition":"restart_required","path":"upstream.max_connections"},{"disposition":"restart_required","path":"upstream.max_keepalive"},{"disposition":"restart_required","path":"upstream.pool_timeout_s"},{"disposition":"restart_required","path":"upstream.read_timeout_s"},{"disposition":"restart_required","path":"upstream.write_timeout_s"}],"free_text_redaction_probe":"Authorization: ","free_text_redaction_token":"","ignored_paths":[],"runtime_tracked_default_paths":["backup.directory","backup.enabled","backup.include_env","backup.interval_s","backup.retain_count","backup.startup_delay_s","dashboard.enabled","dashboard.public","dashboard.refresh_interval_s","dashboard.retain_event_days","dashboard.retain_request_stats_days","dashboard.store_request_content","dashboard.theme","dashboard.themes_dir","database.busy_timeout_ms","database.journal_size_limit","database.path","database.synchronous","database.wal","database.worker_threads","integrations.advertise_base_url","limits.five_hour_microdollars","limits.monthly_microdollars","limits.weekly_microdollars","maintenance.contention_defer_above_lock_wait_p95_ms","maintenance.max_batches_per_tick","maintenance.max_deferral_age_s","maintenance.max_rows_per_batch","maintenance.max_tick_duration_ms","maintenance.p0_max_batches_per_tick","maintenance.p0_max_rows_per_batch","maintenance.p0_max_tick_duration_ms","metrics.aggregate_only","metrics.cleanup_interval_s","metrics.cleanup_max_rows_per_pass","metrics.detailed_span_sample_rate","metrics.dispatch_spans.sample_rate","metrics.dispatch_spans.window_size","metrics.event_loop_lag_enabled","metrics.flush_interval_s","metrics.max_buffered_events","metrics.operational_event_retain_days","metrics.rollup_retain_days","metrics.routing_decision_retain_days","metrics.timeseries_bucket_s","metrics.trace_sample_rate","metrics.write_mode","model_capabilities","model_info.aliases","model_info.conflict_ttl_s","model_info.enabled","model_info.include_in_models_endpoint","model_info.known_ttl_s","model_info.max_models_per_cycle","model_info.overrides","model_info.partial_ttl_s","model_info.refresh_interval_s","model_info.sources","model_info.sparse_new_accelerated_days","model_info.sparse_new_initial_ttl_s","model_info.sparse_new_later_ttl_s","model_info.startup_refresh","model_info.store_raw_observations","model_overrides","model_routers","models.allow_stale_catalog","models.catalog_withdrawal_policy","models.collapse_models","models.expose_mode","models.ping_retain_days","models.refresh_interval_s","models.stale_after_s","models.startup_refresh","network.connect_timeout_s","network.keepalive_expiry_s","network.max_connections","network.max_keepalive","network.read_timeout_s","pricing.catalogs.aliases","pricing.catalogs.opencode_zen.api_key","pricing.catalogs.opencode_zen.base_url","pricing.catalogs.opencode_zen.enabled","pricing.catalogs.opencode_zen.max_entries","pricing.catalogs.opencode_zen.options","pricing.catalogs.opencode_zen.priority","pricing.catalogs.opencode_zen.ttl_seconds","pricing.catalogs.openrouter.api_key","pricing.catalogs.openrouter.base_url","pricing.catalogs.openrouter.enabled","pricing.catalogs.openrouter.max_entries","pricing.catalogs.openrouter.options","pricing.catalogs.openrouter.priority","pricing.catalogs.openrouter.ttl_seconds","pricing.fallback","proxies","readiness_probe.enabled","readiness_probe.freshness_s","readiness_probe.initial_probe","readiness_probe.interval_s","readiness_probe.timeout_s","routing.fairness_epsilon","routing.fairness_mode","routing.fairness_scope","routing.health_penalty","routing.inflight_penalty","routing.local_quota_mode","routing.max_retries_before_stream","routing.near_tie_epsilon","routing.quota_exhausted_cooldown_seconds","routing.randomize_near_ties","routing.strategy","routing.trace.flush_interval_s","routing.trace.guard_cooldown_s","routing.trace.guard_oldest_event_age_s","routing.trace.guard_queue_occupancy_threshold","routing.trace.include_score_components","routing.trace.max_batch_size","routing.trace.mode","routing.trace.queue_capacity","routing.trace.sample_rate","routing.trace.shutdown_flush_timeout_s","routing.trace.skip_above_lock_wait_p95_ms","routing.unknown_request_reservation_microdollars","routing.wire_negotiation.cache_max_entries","routing.wire_negotiation.enabled","routing.wire_negotiation.learned_preference_ttl_s","routing.wire_negotiation.max_concurrent_per_provider","routing.wire_negotiation.min_negotiation_interval_s","routing.wire_negotiation.rejection_cooldown_s","security.allowed_hosts","security.cors_origins","security.persist_redacted_error_detail","security.redact_headers","security.trusted_proxies","server.access_log","server.api_key","server.api_key_env","server.host","server.log_level","server.max_request_body_bytes","server.port","server.threads","transcoder","update_checker.enabled","upstream.base_url","upstream.connect_timeout_s","upstream.keepalive_timeout_s","upstream.max_connections","upstream.max_keepalive","upstream.pool_timeout_s","upstream.read_timeout_s","upstream.write_timeout_s"],"secret_display_token":"","secret_field_names":["api_key","api_key_env","proxy_url","proxy_url_env"],"top_level_config_sections":["accounts","backup","dashboard","database","integrations","limits","maintenance","metrics","model_capabilities","model_info","model_overrides","model_routers","models","network","pricing","providers","proxies","readiness_probe","routing","security","server","transcoder","update_checker","upstream"],"unknown_path_probe":"restart_required"},"lifecycle":{"candidate":{"abort":{"abort_is_idempotent":true,"cleanup_errors_preserve_primary_failure":true,"closed_order":["supervisor","outbound_manager","client_pool"],"primary_failure_stage":"build","process_owned_resources_touched":false,"registered_order":["client_pool","outbound_manager","supervisor"]},"legal_transitions":[["building","prepared"],["prepared","transferred"],["building","aborted"],["prepared","aborted"]],"states":["building","prepared","transferred","aborted"]},"close_order":["generation supervisor stop_all","generation cleanup callbacks","finalization supervisor shutdown","provider client pool close","outbound client manager aclose"],"generation_slot":{"legal_transitions":[["active","retiring"],["retiring","closing"],["closing","closed"],["closing","failed_close"]],"retirement_ready_when":["active_leases == 0","terminal_references == 0"],"states":["active","retiring","closing","closed","failed_close"]},"lease":{"claim_and_increment_are_one_critical_section":true,"claim_predicate":"shutdown_in_progress OR (NOT lease_admission_gated AND active_slot_accepting)","claim_timeout_s":30.0,"release_is_exactly_once":true,"stream_release_boundary":"iterator finally after handler return"},"pending_swap":{"commit_effects":["candidate_becomes_active","old_slot_stops_accepting_leases","publication_epoch_increments","lease_admission_reopens"],"legal_transitions":[["prepared","staged"],["staged","committed"],["staged","rolled_back"],["committed","finalized"]],"rollback_effects":["old_slot_restored","candidate_slot_dropped","lease_admission_reopens","candidate_requires_abort"],"stage_effects":["lease_admission_gated","candidate_slot_non_accepting","active_slot_unchanged"],"states":["prepared","staged","committed","rolled_back","finalized"]},"publication_gate":{"forbidden_inside_gate":["provider/network calls","catalog refresh","backup","unbounded work"],"inside_gate":["stage candidate","apply bounded mandatory acceptance changes","commit runtime pointer","commit task-spec state"],"pre_gate_work":["config parsing and validation","digest check and diff","candidate construction","persistence delta preparation","task diff preflight"]},"shutdown":{"adopts_unfinalized_committed_swap_old_slot":true,"forced_close_abandons_terminal_references_only_on_process_shutdown":true,"live_rehash_preserves_accepted_work":true,"manager_join_deadline_s":10.0,"retirement_drain_deadline_s":5.0,"stops_new_publication_and_acquisition":true},"transaction":{"acceptance_states":["not_accepted","persistence_committed_runtime_pending","accepted"],"accepted_reload_cannot_abort":true,"states":["created","validated","diffed","candidate_prepared","persistence_prepared","process_transitions_prepared","process_transitions_preflighted","commit_started","runtime_staged","runtime_swap_committed","runtime_published","process_transitions_applied","persistence_committed","observable_state_updated","retirement_scheduled","completed","aborting","aborted","compensation_failed"],"transitions":{"aborted":[],"aborting":["aborted","compensation_failed"],"candidate_prepared":["aborting","persistence_prepared"],"commit_started":["aborting","runtime_published","runtime_staged"],"compensation_failed":[],"completed":[],"created":["aborting","validated"],"diffed":["aborting","candidate_prepared"],"observable_state_updated":["aborting","retirement_scheduled"],"persistence_committed":["aborting","observable_state_updated"],"persistence_prepared":["aborting","process_transitions_prepared"],"process_transitions_applied":["aborting","persistence_committed"],"process_transitions_preflighted":["aborting","commit_started"],"process_transitions_prepared":["aborting","process_transitions_preflighted"],"retirement_scheduled":["aborting","completed"],"runtime_published":["aborting","process_transitions_applied"],"runtime_staged":["aborting","runtime_swap_committed"],"runtime_swap_committed":["aborting","process_transitions_applied"],"validated":["aborting","diffed"]}}},"normalization":{"excluded":["API keys","proxy credentials","raw config values","request/provider bodies"],"normalized":["timestamps","runtime identifiers"],"preserved":["state names and order","booleans","changed/restart path ordering","task names and specs","generation increments","retirement pending state","error category distinctions"]},"oracle_modules":["eggpool.runtime_manager","eggpool.generation_factory","eggpool.config_reload_policy","eggpool.reload_transaction","eggpool.runtime_task_inventory","eggpool.runtime_tasks","eggpool.reload_diagnostics","eggpool.app","eggpool.cli_rehash_helper"],"ownership_inventory":[{"authority":"ProcessRuntime.db/stats_db","lifetime":"process","name":"database_and_repositories","ownership":"process-owned"},{"authority":"RuntimeGeneration.client_pool","lifetime":"generation retirement","name":"provider_client_pool","ownership":"generation-owned"},{"authority":"RuntimeGeneration.outbound_manager","lifetime":"generation retirement","name":"outbound_client_manager","ownership":"generation-owned"},{"authority":"RuntimeGeneration.registry","lifetime":"generation retirement","name":"account_registry","ownership":"generation-owned"},{"authority":"RuntimeGeneration.catalog","lifetime":"generation retirement","name":"catalog_service","ownership":"generation-owned"},{"authority":"RuntimeGeneration.router/coordinator","lifetime":"generation retirement","name":"router_and_coordinator","ownership":"generation-owned"},{"authority":"RuntimeGeneration.finalization_supervisor","lifetime":"after leases and terminal references drain","name":"finalization_supervisor","ownership":"generation-owned"},{"authority":"ProcessRuntime.model_router_affinity","lifetime":"process; revalidated per generation","name":"model_router_affinity","ownership":"process-owned"},{"authority":"ProcessRuntime.wire_profile_resolver","lifetime":"process; fingerprint-qualified","name":"wire_profile_resolver","ownership":"process-owned"},{"authority":"RuntimeGeneration.supervisor","lifetime":"generation retirement","name":"generation_task_supervisor","ownership":"generation-owned"},{"authority":"ProcessRuntime.process_supervisor","lifetime":"process singleton","name":"process_task_supervisor","ownership":"process-owned"},{"authority":"startup lifecycle entry point","lifetime":"one bounded startup pass","name":"crash_reconciler","ownership":"process-owned/startup-only"},{"authority":"RuntimeMetricsService and reload metadata","lifetime":"process; bounded snapshots","name":"runtime_metrics_and_reload_diagnostics","ownership":"process-owned"},{"authority":"listener, middleware, route topology, auth, DB constructor","lifetime":"process construction","name":"server_constructor_state","ownership":"startup-only/restart-required"},{"authority":"RuntimeGenerationCandidate","lifetime":"building/prepared until transfer or abort","name":"candidate_resources_before_publication","ownership":"candidate-owned"},{"authority":"GenerationLease","lifetime":"finite completion or stream disconnect","name":"request_and_stream_lease","ownership":"request/lease-owned"},{"authority":"terminal reference on generation slot","lifetime":"durable/runtime convergence","name":"retained_finalization","ownership":"retained-finalization-owned"}],"reload":{"cases":[{"accepted":true,"active_generation_unchanged":true,"category":"success_noop","changed_sections":[],"generation_delta":0,"legacy_wire_stage":"commit","name":"identical_digest_noop","publication_occurred":false,"restart_required_paths":[],"retirement_pending":false,"terminal_stage":"idle"},{"accepted":true,"active_generation_unchanged":false,"category":"success_committed","changed_sections":["server"],"generation_delta":1,"legacy_wire_stage":"retirement","name":"valid_live_only_change","publication_occurred":true,"restart_required_paths":[],"retirement_pending":"true_if_old_work_remains","terminal_stage":"retirement"},{"accepted":false,"active_generation_unchanged":true,"category":"rejected_restart_required","changed_sections":["server"],"generation_delta":0,"legacy_wire_stage":"diff","name":"restart_required_change","publication_occurred":false,"restart_required_paths":["server.port"],"retirement_pending":false,"terminal_stage":"diff"},{"accepted":false,"active_generation_unchanged":true,"category":"rejected_restart_required","changed_sections":["server"],"generation_delta":0,"legacy_wire_stage":"diff","name":"mixed_live_and_restart_required_change","publication_occurred":false,"restart_required_paths":["server.port"],"retirement_pending":false,"terminal_stage":"diff"},{"accepted":false,"active_generation_unchanged":true,"category":"rejected_validation","changed_sections":[],"generation_delta":0,"legacy_wire_stage":"validation","name":"invalid_config","publication_occurred":false,"restart_required_paths":[],"retirement_pending":false,"terminal_stage":"validation"},{"accepted":false,"active_generation_unchanged":true,"category":"rejected_validation","changed_sections":[],"generation_delta":0,"legacy_wire_stage":"digest_check","name":"expected_digest_mismatch","publication_occurred":false,"restart_required_paths":[],"retirement_pending":false,"terminal_stage":"validation"},{"accepted":false,"active_generation_unchanged":true,"category":"rejected_validation","changed_sections":[],"generation_delta":0,"legacy_wire_stage":"preparation","name":"stale_caller_generation","publication_occurred":false,"restart_required_paths":[],"retirement_pending":false,"terminal_stage":"preparation"},{"accepted":false,"active_generation_unchanged":true,"category":"failed_candidate_prepare","changed_sections":[],"generation_delta":0,"legacy_wire_stage":"preparation","name":"candidate_construction_failure","publication_occurred":false,"restart_required_paths":[],"retirement_pending":false,"terminal_stage":"preparation"},{"accepted":false,"active_generation_unchanged":true,"category":"failed_process_transition_prepare","changed_sections":[],"generation_delta":0,"legacy_wire_stage":"preparation","name":"preflight_process_transition_failure","publication_occurred":false,"restart_required_paths":[],"retirement_pending":false,"terminal_stage":"preparation"},{"accepted":false,"active_generation_unchanged":true,"category":"aborted_cancelled","changed_sections":[],"generation_delta":0,"legacy_wire_stage":"preparation","name":"cancellation_before_acceptance","publication_occurred":false,"restart_required_paths":[],"retirement_pending":false,"terminal_stage":"preparation"},{"accepted":true,"active_generation_unchanged":false,"category":"post_commit_finalization_pending","changed_sections":[],"generation_delta":1,"legacy_wire_stage":"retirement","name":"cancellation_during_or_after_acceptance","publication_occurred":true,"restart_required_paths":[],"retirement_pending":true,"terminal_stage":"retirement"},{"accepted":true,"active_generation_unchanged":false,"category":"post_commit_finalization_pending","changed_sections":[],"generation_delta":1,"legacy_wire_stage":"retirement","name":"successful_publication_retirement_pending","publication_occurred":true,"restart_required_paths":[],"retirement_pending":true,"terminal_stage":"retirement"},{"accepted":true,"active_generation_unchanged":false,"category":"success_committed","changed_sections":[],"generation_delta":1,"legacy_wire_stage":"retirement","name":"successful_publication_already_drained","publication_occurred":true,"restart_required_paths":[],"retirement_pending":false,"terminal_stage":"retirement"}],"classification_probes":{"retry_pending":"post_commit_finalization_pending","success_committed":"success_committed","success_noop":"success_noop","validation":"rejected_validation"},"diff_observations":{"identical":{"changes":[],"dispositions":[],"live":[],"paths":[],"restart_required":[],"sections":[]},"live_only":{"changes":[{"disposition":"live","new_display":"10485761","old_display":"10485760","path":"server.max_request_body_bytes","secret":false,"section":"server"}],"dispositions":["live"],"live":["server.max_request_body_bytes"],"paths":["server.max_request_body_bytes"],"restart_required":[],"sections":["server"]},"mixed":{"changes":[{"disposition":"live","new_display":"10485761","old_display":"10485760","path":"server.max_request_body_bytes","secret":false,"section":"server"},{"disposition":"restart_required","new_display":"11301","old_display":"11300","path":"server.port","secret":false,"section":"server"}],"dispositions":["live","restart_required"],"live":["server.max_request_body_bytes"],"paths":["server.max_request_body_bytes","server.port"],"restart_required":["server.port"],"sections":["server"]},"restart_only":{"changes":[{"disposition":"restart_required","new_display":"11301","old_display":"11300","path":"server.port","secret":false,"section":"server"}],"dispositions":["restart_required"],"live":[],"paths":["server.port"],"restart_required":["server.port"],"sections":["server"]},"secret":{"changes":[{"disposition":"restart_required","new_display":"","old_display":"","path":"pricing.catalogs.openrouter.api_key","secret":true,"section":"pricing"}],"dispositions":["restart_required"],"live":[],"paths":["pricing.catalogs.openrouter.api_key"],"restart_required":["pricing.catalogs.openrouter.api_key"],"sections":["pricing"]}},"generation_id_rule":"monotonic; no-op and rejected reloads do not publish","legacy_stages":["validation","digest_check","diff","preparation","reconciliation","commit","retirement"],"result_categories":["success_committed","success_noop","success_ignored_only","rejected_busy","rejected_validation","rejected_restart_required","failed_candidate_prepare","failed_persistence_prepare","failed_process_transition_prepare","failed_commit","failed_publication","failed_process_transition_apply","failed_persistence_commit","failed_runtime_stage","failed_runtime_stage_rollback","post_commit_finalization_pending","retirement_schedule_failed","shutdown_adopted","aborted_cancelled","aborted_shutdown","compensation_failed","internal_error"],"retirement_pending_rule":"accepted publication may be pending while old leases or terminal references drain","terminal_stages":["validation","diff","preparation","reconciliation","commit","retirement","idle"]},"schema_version":"m8-runtime-lifecycle-r001-observations/v1","tasks":{"conditions":{"automatic_backup":"backup.enabled and interval_s > 0 enables","catalog_refresh":"models.refresh_interval_s <= 0 disables","generation_leased_callbacks":"acquire active generation for every tick","metrics_flush":"metrics.write_mode == immediate disables","retention_cleanup":"always registered; reads active generation config per tick","update_checker":"startup-only process task; update_checker.enabled and outbound handle required"},"config_variants":{"automatic_backup_enabled":[{"enabled":true,"initial_delay_s":null,"interval_s":300.0,"name":"catalog_refresh","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":86400.0,"name":"retention_cleanup","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":60.0,"name":"checkpoint","run_immediately":true},{"enabled":true,"initial_delay_s":5.0,"interval_s":120.0,"name":"metrics_flush","run_immediately":false},{"enabled":false,"initial_delay_s":null,"interval_s":86400.0,"name":"update_checker","run_immediately":true},{"enabled":true,"initial_delay_s":5.0,"interval_s":600.0,"name":"automatic_backup","run_immediately":false}],"catalog_refresh_disabled":[{"enabled":false,"initial_delay_s":null,"interval_s":0.0,"name":"catalog_refresh","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":86400.0,"name":"retention_cleanup","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":60.0,"name":"checkpoint","run_immediately":true},{"enabled":true,"initial_delay_s":5.0,"interval_s":120.0,"name":"metrics_flush","run_immediately":false},{"enabled":false,"initial_delay_s":null,"interval_s":86400.0,"name":"update_checker","run_immediately":true},{"enabled":false,"initial_delay_s":300.0,"interval_s":86400.0,"name":"automatic_backup","run_immediately":false}],"default":[{"enabled":true,"initial_delay_s":null,"interval_s":300.0,"name":"catalog_refresh","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":86400.0,"name":"retention_cleanup","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":60.0,"name":"checkpoint","run_immediately":true},{"enabled":true,"initial_delay_s":5.0,"interval_s":120.0,"name":"metrics_flush","run_immediately":false},{"enabled":false,"initial_delay_s":null,"interval_s":86400.0,"name":"update_checker","run_immediately":true},{"enabled":false,"initial_delay_s":300.0,"interval_s":86400.0,"name":"automatic_backup","run_immediately":false}],"metrics_immediate":[{"enabled":true,"initial_delay_s":null,"interval_s":300.0,"name":"catalog_refresh","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":86400.0,"name":"retention_cleanup","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":60.0,"name":"checkpoint","run_immediately":true},{"enabled":false,"initial_delay_s":5.0,"interval_s":120.0,"name":"metrics_flush","run_immediately":false},{"enabled":false,"initial_delay_s":null,"interval_s":86400.0,"name":"update_checker","run_immediately":true},{"enabled":false,"initial_delay_s":300.0,"interval_s":86400.0,"name":"automatic_backup","run_immediately":false}],"update_checker_enabled":[{"enabled":true,"initial_delay_s":null,"interval_s":300.0,"name":"catalog_refresh","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":86400.0,"name":"retention_cleanup","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":60.0,"name":"checkpoint","run_immediately":true},{"enabled":true,"initial_delay_s":5.0,"interval_s":120.0,"name":"metrics_flush","run_immediately":false},{"enabled":true,"initial_delay_s":null,"interval_s":86400.0,"name":"update_checker","run_immediately":true},{"enabled":false,"initial_delay_s":300.0,"interval_s":86400.0,"name":"automatic_backup","run_immediately":false}]},"inventory":[{"callback_kind":"catalog_refresh","default_enabled":true,"description":"Periodically refresh the model catalog from enabled accounts","generation_dependencies":["catalog","model_info","registry","health_manager"],"initial_delay_s":null,"interval_s":300.0,"name":"catalog_refresh","ownership":"generation_leased","process_dependencies":["db"],"reloadable_fields":["models.refresh_interval_s"],"run_immediately":false,"timeout_s":null},{"callback_kind":"retention_cleanup","default_enabled":true,"description":"Clean up old requests, events, pings, and expired reservations","generation_dependencies":["router"],"initial_delay_s":null,"interval_s":86400.0,"name":"retention_cleanup","ownership":"generation_leased","process_dependencies":["db"],"reloadable_fields":["dashboard.retain_request_stats_days","dashboard.retain_event_days","metrics.operational_event_retain_days","metrics.routing_decision_retain_days","models.ping_retain_days"],"run_immediately":false,"timeout_s":null},{"callback_kind":"checkpoint","default_enabled":true,"description":"Periodic SQLite WAL checkpoint","generation_dependencies":[],"initial_delay_s":null,"interval_s":60.0,"name":"checkpoint","ownership":"process","process_dependencies":["db"],"reloadable_fields":[],"run_immediately":true,"timeout_s":null},{"callback_kind":"metrics_flush","default_enabled":true,"description":"Flush buffered metrics analytics to SQLite","generation_dependencies":[],"initial_delay_s":5.0,"interval_s":30.0,"name":"metrics_flush","ownership":"process","process_dependencies":["metrics_coalescer"],"reloadable_fields":["metrics.write_mode","metrics.flush_interval_s"],"run_immediately":false,"timeout_s":null},{"callback_kind":"update_checker","default_enabled":true,"description":"Periodically check PyPI for new EggPool releases","generation_dependencies":[],"initial_delay_s":null,"interval_s":86400.0,"name":"update_checker","ownership":"process","process_dependencies":["outbound_manager"],"reloadable_fields":[],"run_immediately":true,"timeout_s":null},{"callback_kind":"automatic_backup","default_enabled":true,"description":"Create periodic backup archives of config and database","generation_dependencies":[],"initial_delay_s":300.0,"interval_s":86400.0,"name":"automatic_backup","ownership":"process","process_dependencies":["db"],"reloadable_fields":["backup.enabled","backup.interval_s","backup.startup_delay_s","backup.retain_count"],"run_immediately":false,"timeout_s":null}],"names_unique":true,"singleton_process_tasks":["checkpoint","metrics_flush","update_checker","automatic_backup"]}} From 472bef9f5dd8575f66416ac9cf07c1a766efc722 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Fri, 16 Oct 2026 03:12:29 +0000 Subject: [PATCH 58/62] Start closing M023 short-window fairness controls --- .../on-prem-user-access/023-short-window-fairness-controls.md | 2 +- plans/registry.md | 4 ++-- plans/subsystems/on-prem-user-access-roadmap.md | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md b/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md index a4dc3aa3..f8608f0d 100644 --- a/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md +++ b/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md @@ -1,6 +1,6 @@ # On-Prem User Access M023 — Short-window team fairness and admission controls -Status: active +Status: closing Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index bb740228..47860c9a 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -58,7 +58,7 @@ No new runtime, schema or performance claim lands with these plans. | Persistence | active | `plans/subsystems/persistence-roadmap.md` | M011 ready — post-M010 roadmap/documentation reconciliation | M009/M010 closures accepted. Documentation/tooling-only corrective; production remains WAL/NORMAL. No technical persistence successor is registered. | | Deployment and packaging | closed | `plans/subsystems/deployment-packaging-roadmap.md` | M005 closed — active PATH integration detection corrective | M001–M005 closed; no successor registered. | | Dashboard | closed | `plans/subsystems/dashboard-roadmap.md` | M012 closed — lifecycle closure and documentation polish | M001–M012 closed; no registered successor; future dashboard work requires a new bounded plan. | -| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M023 active — short-window fairness controls | M017–M022 closed; M002 ordinary-server gate satisfied. PR #7 and M001–M016 closure records remain immutable. Solo SBC/microSD is not the ordinary-server team-capacity gate. | +| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M023 closing — short-window fairness controls | M017–M022 closed; M002 ordinary-server gate satisfied. Production, test, and documentation work is committed; post-change Linux latency comparison remains an external condition. | | On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 closed — ordinary-server synthetic baseline recorded | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 authorizes no WAL/SQLite topology change. | ## Dependency-ready implementation plans @@ -71,7 +71,7 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| -| On-prem user access | M023 — short-window team fairness | active | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | Implement opt-in bounded principal/global admission before body collection; validate against measured ordinary-server evidence. | +| On-prem user access | M023 — short-window team fairness | closing | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | Production implementation and local verification are complete; final closure records the unavailable post-change ordinary-Linux comparison. | ## Blocked work diff --git a/plans/subsystems/on-prem-user-access-roadmap.md b/plans/subsystems/on-prem-user-access-roadmap.md index 3bd701ff..90b9afeb 100644 --- a/plans/subsystems/on-prem-user-access-roadmap.md +++ b/plans/subsystems/on-prem-user-access-roadmap.md @@ -346,7 +346,7 @@ behavior and adding actionable hard-budget diagnostics. No output cap is injected; M008/M015 proof-or-reject guarantees remain in force. Source: `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md`. -#### M023 — Short-window team fairness controls (active) +#### M023 — Short-window team fairness controls (closing) Class: capability. Hard M022 dependency is closed, and the operational ordinary-server Linux/SSD-or-NVMe evidence gate is satisfied by concurrency @@ -461,4 +461,4 @@ A ready plan is not an implemented capability. | M020 | closed | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | `plans/closure/on-prem-user-access/020-status.md` | Headless member lifecycle and first-member transition are atomic and audited; M021 unblocked. | | M021 | closed | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | `plans/closure/on-prem-user-access/021-status.md` | Transactional future-only profile default; CLI/browser and no-profile journeys, migration, policy and budget boundaries verified. | | M022 | closed | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | `plans/closure/on-prem-user-access/022-status.md` | Captured Codex/OpenCode output-cap behavior; preserved hard proof-or-reject semantics and added actionable error codes. | -| M023 | active | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Hard M022 is closed; ordinary-server M002 operational evidence is recorded in its closure. | +| M023 | closing | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Production implementation and local verification are complete; final closure records the external post-change Linux performance condition. | From 3776e4138d989010b2c72efe913f379fc081a145 Mon Sep 17 00:00:00 2001 From: David Bowman Date: Fri, 16 Oct 2026 03:14:37 +0000 Subject: [PATCH 59/62] Close M023 with Linux performance condition --- .../closure/on-prem-user-access/023-status.md | 190 ++++++++++++++++++ .../023-short-window-fairness-controls.md | 2 +- plans/registry.md | 14 +- .../subsystems/on-prem-user-access-roadmap.md | 17 +- 4 files changed, 212 insertions(+), 11 deletions(-) create mode 100644 plans/closure/on-prem-user-access/023-status.md diff --git a/plans/closure/on-prem-user-access/023-status.md b/plans/closure/on-prem-user-access/023-status.md new file mode 100644 index 00000000..6791a1ac --- /dev/null +++ b/plans/closure/on-prem-user-access/023-status.md @@ -0,0 +1,190 @@ +# On-Prem User Access M023 — Closure Status + +Status: conditionally closed + +Source implementation plan: + +- `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` + +Source subsystem roadmap: + +- `plans/subsystems/on-prem-user-access-roadmap.md#m023--short-window-team-fairness-controls` + +Repository baseline reviewed: `deda832d` (M022 closed and M002 ordinary-server +qualification closed; M023 dependency gates satisfied). + +Implementation commits: + +- `84482e91b84fe7424144dcf916441479c5b492d1` — implement bounded short-window admission fairness. +- `3722ae64ede200da69415c1da711dd6c5248c519` — move M023 from ready to active. +- `472bef9f5dd8575f66416ac9cf07c1a766efc722` — move M023 from active to closing. + +## 1. Executive finding + +M023's production capability is implemented. Optional per-principal and +instance-global 60-second token buckets now run after authentication and +generation acquisition, before body collection and durable budget admission. +They share allowance across credentials owned by one user, bound retained +principal state, fail closed at capacity, return a distinct 429 with +`Retry-After`, and expose remaining allowance headers on admitted responses. +No-rule behavior allocates no limiter state. No queue, database write, +migration, provider-routing change, or provider-health effect was added. + +The full local default/no-default Rust suites, strict Clippy, focused HTTP and +reload tests, and tooling suite pass. The plan's post-change p50/p95/p99 +comparison on an ordinary Linux server with durable SSD/NVMe storage could not +be run here: this checkout is executing on Darwin. The available Ubuntu M002 +measurements are a pre-change reference and cannot stand in for a measurement +of the M023 binary. M023 is therefore **conditionally closed** pending that +external operational comparison; no general capacity or latency claim is +made. + +## 2. Requirement-to-evidence matrix + +| Requirement | Evidence | Result | Notes | +|---|---|---|---| +| Keep no-rule behavior unchanged and avoid state allocation | `request::admission_fairness::tests::absent_policy_keeps_the_no_rule_fast_path`; `AdmissionFairnessLimiter::admit_at` | pass | Disabled policy returns before locking the shared limiter. | +| Share per-principal allowance across credentials and isolate 1/5/25/49 users | `all_credentials_for_one_principal_share_a_bucket`; `qualification_user_counts_keep_buckets_separate` | pass | Principal identity is the authenticated database-owned user ID, never a credential or request header. | +| Intersect per-principal and optional global limits | `global_and_principal_rules_intersect_without_consuming_denied_slots` | pass | A denied request does not consume the other bucket. | +| Bound concurrent admission and use a monotonic window | `simultaneous_last_slot_is_granted_once`; `monotonic_refill_admits_again_at_the_window_boundary` | pass | Thirty-two racing callers receive the single remaining token exactly once; refill uses `Instant`. | +| Bound principal state and define full-table behavior | `principal_table_is_bounded_and_evicts_only_fully_stale_state`; `many_principals_remain_within_declared_capacity` | pass | Stale buckets are eligible only after the 60-second bucket can fully refill (120-second retention); otherwise new principals fail closed with a computed retry delay. | +| Return an actionable throttle before reading or dispatching the body | `server_transport::short_window_fairness_rejects_before_dispatch_and_reports_retry_scope` | pass | The second request declares a body but sends none; it receives 429, `Retry-After: 60`, `short_window_rate_limited`, and the principal scope without body collection. | +| Keep policy bounded and live-generation consistent | `config::tests::short_window_fairness_values_are_bounded_and_optional`; `runtime_lifecycle_r005::admission_fairness_policy_is_live_reloaded_as_one_generation_contract` | pass | Optional rates validate from 1–60,000/minute; tracked principals validate from 1–65,536. Policy is read from the acquired request generation. | +| Measure post-change overhead and tails on the ordinary Linux target | M002 reference: `plans/closure/on-prem-concurrency-performance/002-status.md`; current local host: `uname -a` reports Darwin | partial | No post-change Linux p50/p95/p99 run or baseline delta is available from this host. This is the remaining condition. | + +## 3. Production implementation evidence + +`rust/src/request/admission_fairness.rs` owns the process-local token buckets. +Per-principal state is held in a bounded map; an optional global bucket is a +single value. Stale state is reclaimed only after it has had more than one +full refill window to become idle. A full table with no stale entry refuses a +new principal rather than evicting an active user's allowance. A new limiter +is created with each server AppState, so restart resets these ephemeral +counters. + +`server/middleware.rs::admit_inference_body` consults the limiter after auth +and generation acquisition but before `Content-Length` checks, body +collection, and the existing raw-body reservation. A denial returns a +protocol-shaped 429 with `Retry-After`, code `short_window_rate_limited`, and +`X-EggPool-Admission-Scope`. Admitted responses carry +`X-EggPool-Admission-Remaining`, scope, and the 60-second window. Requests +that later fail body validation consume one short-window attempt; cancellation +before upstream dispatch likewise consumes only that ephemeral attempt, which +refills normally. This behavior is documented. + +Configuration lives under `[admission_fairness]`. Both rates default to +disabled; changes are live-reloadable and apply according to the acquired +generation. The configuration template's runtime-manifest digest and the +R005 compatibility oracle were updated. There is no schema or migration. + +## 4. Verification executed + +### Commands run + +```bash +rtk cargo fmt --manifest-path rust/Cargo.toml --all -- --check +rtk cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings +rtk cargo check --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features +rtk cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- -D warnings +rtk cargo test --manifest-path rust/Cargo.toml --workspace --all-targets -- --test-threads=1 +rtk cargo test --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- --test-threads=1 +rtk cargo test --manifest-path rust/Cargo.toml --test runtime_lifecycle_r005 +rtk cargo test --manifest-path rust/Cargo.toml --test server_transport short_window_fairness_rejects_before_dispatch_and_reports_retry_scope -- --test-threads=1 +rtk cargo test --manifest-path rust/Cargo.toml --test build_manifest +rtk uv run pytest tests/tooling/ -q --tb=short --maxfail=1 +rtk git diff --check +``` + +### Results + +- Default workspace: 1,050 passed, 1 ignored across 80 suites. +- No-default workspace: 1,051 passed, 1 ignored across 80 suites. +- Strict Clippy passed with default and no-default features; no-default check passed. +- Focused config/reload target: 11 passed. Focused loopback HTTP target: 1 passed. Runtime asset manifest target: 4 passed. +- Tooling pytest: 199 passed, 1 skipped. +- Formatting and `git diff --check` passed. +- Verification is local. Hosted CI and post-change Linux hardware qualification were not run in this closure pass. + +## 5. Invariant review + +- Durable daily/monthly user and instance budgets remain authoritative and are + not read or modified by the short-window limiter. +- Requests rejected by this limiter do not reach endpoint parsing, provider + selection, provider health, quota, or upstream submission. +- Principal keys come from authenticated identity state. Tokens, credentials, + request bodies, prompts, IPs, and provider outcomes are not retained. +- Limiter state and critical sections are bounded. The no-rule fast path does + not acquire the limiter mutex. There is no per-client wait queue or extra + SQLite writer. +- Requests use one immutable configuration generation for body admission and + the downstream execution path. The limiter itself is process-local and + shared across generations. + +## 6. Failure and recovery review + +- User/global throttles return a stable, distinct 429 with a positive + `Retry-After`; neither is confused with provider or durable-budget 429s. +- If both buckets apply, a denial does not consume a token from the other + bucket. Concurrent callers serialize only the bounded bucket decision, so a + final token is granted once. +- If principal-state capacity is reached, active entries are not evicted to + grant a new caller excess traffic. Stale state becomes evictable only after + refill has completed. +- Restart discards short-window state. Durable holds, settled budgets, + provider retry state, and request recovery are unaffected. +- Body validation failure or cancellation after the admission decision may + consume one ephemeral token; the token then refills normally. No durable + counter or reservation is left behind by the limiter. + +## 7. Migration and compatibility review + +No migration was added. Existing configurations omit `[admission_fairness]` +and retain disabled throttling. Configured rate and table-bound changes are +classified live in the canonical reload policy. The HTTP error is distinct +from hard-budget, exhausted-budget, and provider rate-limit responses. + +## 8. Security review + +Authentication runs before this check. Bucket identity cannot be selected by +an API token value, header, model name, IP address, or body field. Unknown +authenticated principals cannot grow state without limit. Capacity exhaustion +fails closed for new principals, and response diagnostics contain only the +bounded scope and remaining-count facts. + +## 9. Documentation and operations + +Updated `docs/model-limits.md`, `docs/api-reference.md`, `docs/deployment.md`, +`architecture/deep-dive-request-lifecycle.md`, and +`architecture/deep-dive-security.md`. `rust/config.example.toml` documents the +disabled-by-default options. Operators can rehash the policy; restarting +clears its in-memory buckets. + +The M002 host reference is Ubuntu 24.04.5 with ext4 on Samsung SSD 980 NVMe. +It recorded the pre-M023 synthetic envelope and remains useful as the +comparison baseline. This checkout reports Darwin and did not produce +post-change measurements for that Linux host. + +## 10. Unresolved findings + +| Severity | Finding | Impact | Required action | +|---|---|---|---| +| medium | Post-change p50/p95/p99 overhead and admitted/rejected-rate comparison on durable ordinary Linux hardware was unavailable locally | Functional behavior is verified, but no measured no-rule overhead delta or ordinary-server fairness capacity result is claimed | Run the M023 fairness matrix and no-rule comparison on an ordinary Linux SSD/NVMe host against the M002 reference; record host/storage metadata, 1/5/25/49 principal and concurrency results, p50/p95/p99 overhead, and any envelope changes. | +| low | Hosted CI was not run in this local pass | The local default/no-default matrix passes; branch CI remains the hosted gate | Review hosted CI for the final pushed commit. | + +No high-severity correctness or security finding remains. + +## 11. Roadmap disposition + +M023's implementation is complete and may be used with operator-selected +limits. It is conditionally closed until the named ordinary-Linux performance +comparison is recorded. No capacity, latency-SLA, or universal team-size claim +is made. M023 has no registered successor, so this closure unblocks no future +on-prem user-access milestone. + +## 12. Registry updates + +The closure commit marked the implementation plan `implemented`, removed +M023 from the active implementation table, marked the roadmap milestone +`conditionally closed`, added this record to recent closures, and recorded +the ready/active/blocked audit. No successor was promoted or blocked; M023 is +terminal in the registered line. diff --git a/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md b/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md index f8608f0d..cdbf16f8 100644 --- a/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md +++ b/plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md @@ -1,6 +1,6 @@ # On-Prem User Access M023 — Short-window team fairness and admission controls -Status: closing +Status: implemented Repository baseline: `88f58bbf2e9095cf46437945edf83e580270bdb6` (`main`, post-PR #7). diff --git a/plans/registry.md b/plans/registry.md index 47860c9a..85b3fe5c 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -58,7 +58,7 @@ No new runtime, schema or performance claim lands with these plans. | Persistence | active | `plans/subsystems/persistence-roadmap.md` | M011 ready — post-M010 roadmap/documentation reconciliation | M009/M010 closures accepted. Documentation/tooling-only corrective; production remains WAL/NORMAL. No technical persistence successor is registered. | | Deployment and packaging | closed | `plans/subsystems/deployment-packaging-roadmap.md` | M005 closed — active PATH integration detection corrective | M001–M005 closed; no successor registered. | | Dashboard | closed | `plans/subsystems/dashboard-roadmap.md` | M012 closed — lifecycle closure and documentation polish | M001–M012 closed; no registered successor; future dashboard work requires a new bounded plan. | -| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M023 closing — short-window fairness controls | M017–M022 closed; M002 ordinary-server gate satisfied. Production, test, and documentation work is committed; post-change Linux latency comparison remains an external condition. | +| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M023 conditionally closed — short-window fairness controls | M017–M022 closed; M023 production and local evidence complete. Post-change ordinary-Linux latency comparison remains the named condition. No successor is registered. | | On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 closed — ordinary-server synthetic baseline recorded | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 authorizes no WAL/SQLite topology change. | ## Dependency-ready implementation plans @@ -71,7 +71,6 @@ No new runtime, schema or performance claim lands with these plans. | Subsystem | Milestone | Status | Implementation plan | Handoff note | |---|---|---|---|---| -| On-prem user access | M023 — short-window team fairness | closing | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | Production implementation and local verification are complete; final closure records the unavailable post-change ordinary-Linux comparison. | ## Blocked work @@ -109,6 +108,7 @@ Historical M007 blocker assessment at baseline `7e241ad` (`plans/closure/persist | Subsystem / plan | Disposition | Evidence | |---|---|---| +| On-prem user access M023 — short-window team fairness | conditionally closed — opt-in per-principal/global token buckets, pre-body 429/Retry-After, bounded process-local state, default/no-default suites and tooling pass; post-change ordinary-Linux p50/p95/p99 comparison remains outstanding; no registered successor | `plans/closure/on-prem-user-access/023-status.md`, implementation `84482e9` | | On-prem concurrency performance M002 — ordinary-server team baseline | closed — 27 passing Q008 runs across nine cells; Q009 hard-budget, provider-fault, user-disable and restart evidence; bounded competing-I/O and SQLite gate diagnostics; measured host-specific sizing guidance; M023 promoted to ready | `plans/closure/on-prem-concurrency-performance/002-status.md`, artifacts under `artifacts/qualification/on-prem-concurrency-performance/ordinary-server/` | | On-prem user access M022 — hard-budget agent interoperability | closed — Codex CLI 0.162.1 omitted a Responses output cap; OpenCode 1.18.35 sent configured `max_output_tokens`; stable proof-failure and exhausted-budget codes added; default/no-default verification passes; M023 is now ready after M002 operational evidence | `plans/closure/on-prem-user-access/022-status.md`, implementation `dee65ae` | | On-prem user access M021 — default member policy and provisioning | closed — transactional future-only default profile selection, CLI/browser creation, explicit no-profile override, additive schema 63 migration, policy visibility and budget-separation behavior pass; M022 subsequently closed; M023 remains blocked on ordinary-server evidence | `plans/closure/on-prem-user-access/021-status.md`, implementation `b76d359`, CLI journey `ee563be`, race test `3f1ca54` | @@ -561,3 +561,13 @@ evidence, so M023 moves `blocked` → `ready`. No other registered future plan declares M002 as a hard, interface, or soft dependency. M001 remains conditionally closed; no production storage optimization is authorized by this evidence. + + +Unblock audit (on-prem user access M023 conditionally closed, +`plans/closure/on-prem-user-access/023-status.md`): searched +`plans/implementation/`, the on-prem user-access roadmap dependency graph, +and the dependency-ready, active, and blocked registry tables. M023 has no +registered successor or downstream plan, so no future work is newly +unblocked and no status promotion is required. Its ordinary-Linux +post-change performance comparison remains a closure condition, not a +dependency for any registered plan. No capacity claim is introduced. diff --git a/plans/subsystems/on-prem-user-access-roadmap.md b/plans/subsystems/on-prem-user-access-roadmap.md index 90b9afeb..8202210c 100644 --- a/plans/subsystems/on-prem-user-access-roadmap.md +++ b/plans/subsystems/on-prem-user-access-roadmap.md @@ -346,19 +346,20 @@ behavior and adding actionable hard-budget diagnostics. No output cap is injected; M008/M015 proof-or-reject guarantees remain in force. Source: `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md`. -#### M023 — Short-window team fairness controls (closing) +#### M023 — Short-window team fairness controls (conditionally closed) Class: capability. Hard M022 dependency is closed, and the operational -ordinary-server Linux/SSD-or-NVMe evidence gate is satisfied by concurrency -M002. Implement an optional per-principal short-window admission throttle and -stable 429/Retry-After, bounded in-memory state, no unbounded queues or new -SQLite write pressure; preserve solo no-rule behavior. M002 provides a -host-specific synthetic reference, not a universal capacity guarantee. +ordinary-server Linux/SSD-or-NVMe baseline gate is satisfied by concurrency +M002. Production implementation and local verification are complete. The +post-change ordinary-Linux p50/p95/p99 comparison remains outstanding, so no +new capacity claim is made. M002 provides a host-specific synthetic reference, +not a universal capacity guarantee. Source: `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md`. ### Post-merge dependency and hardware policy -- M017–M022 are closed. M022 did not depend on a Pi/Le Potato run; it became +- M017–M022 are closed; M023 is conditionally closed on its post-change + ordinary-Linux comparison. M022 did not depend on a Pi/Le Potato run; it became ready after M017 closed. - Hard: M017 + M019 -> M020 -> M021; M017 -> M022 -> M023. - Interface: M018's privileged local transport must not broaden HTTP rights; @@ -461,4 +462,4 @@ A ready plan is not an implemented capability. | M020 | closed | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | `plans/closure/on-prem-user-access/020-status.md` | Headless member lifecycle and first-member transition are atomic and audited; M021 unblocked. | | M021 | closed | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | `plans/closure/on-prem-user-access/021-status.md` | Transactional future-only profile default; CLI/browser and no-profile journeys, migration, policy and budget boundaries verified. | | M022 | closed | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | `plans/closure/on-prem-user-access/022-status.md` | Captured Codex/OpenCode output-cap behavior; preserved hard proof-or-reject semantics and added actionable error codes. | -| M023 | closing | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | — | Production implementation and local verification are complete; final closure records the external post-change Linux performance condition. | +| M023 | conditionally closed | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | `plans/closure/on-prem-user-access/023-status.md` | Production and local verification pass; post-change ordinary-Linux p50/p95/p99 comparison remains the named condition. No successor is registered. | From f08a760e2cd995ac9b3416b571d092e3315b6524 Mon Sep 17 00:00:00 2001 From: wr3n-ai Date: Fri, 16 Oct 2026 05:05:11 +0000 Subject: [PATCH 60/62] Skip legacy key generation after admin bootstrap --- rust/src/runtime.rs | 46 +++++++++++++++++++++++++++++---------------- 1 file changed, 30 insertions(+), 16 deletions(-) diff --git a/rust/src/runtime.rs b/rust/src/runtime.rs index d7a71898..967dd0bb 100644 --- a/rust/src/runtime.rs +++ b/rust/src/runtime.rs @@ -2616,22 +2616,10 @@ async fn onboard(path: &Path, args: crate::cli::OnboardArgs) -> Result<(), Boots .map_err(|error| mutation_error(error, EXIT_VALIDATION))?; println!(" Created configuration at {}", path.display()); } - if config_mutation::read_server_key(path) - .map_err(|error| mutation_error(error, EXIT_VALIDATION))? - .is_none() - { - let key = config_mutation::generate_key() - .map_err(|error| mutation_error(error, EXIT_VALIDATION))?; - if config_mutation::write_server_key(path, &key) - .map_err(|error| mutation_error(error, EXIT_VALIDATION))? - { - println!(" Generated server API key"); - } - } let preflight_config = config::Config::from_toml(path).map_err(BootstrapError::from)?; let preflight_database_path = config::Config::runtime_path(&preflight_config.database.path); - let bootstrap_admin = if !preflight_database_path.exists() { - true + let (bootstrap_admin, identity_initialized) = if !preflight_database_path.exists() { + (true, false) } else { let database = open_operational_database(&preflight_config, true).await?; let access_state = crate::identity::access_state(&database) @@ -2644,8 +2632,24 @@ async fn onboard(path: &Path, args: crate::cli::OnboardArgs) -> Result<(), Boots .close() .await .map_err(|error| command_error(EXIT_VALIDATION, error.to_string()))?; - access_state == "fresh_uninitialized" && users.is_empty() + ( + access_state == "fresh_uninitialized" && users.is_empty(), + matches!(access_state.as_str(), "admin_initialized" | "managed"), + ) }; + if should_generate_legacy_server_key(identity_initialized) + && config_mutation::read_server_key(path) + .map_err(|error| mutation_error(error, EXIT_VALIDATION))? + .is_none() + { + let key = config_mutation::generate_key() + .map_err(|error| mutation_error(error, EXIT_VALIDATION))?; + if config_mutation::write_server_key(path, &key) + .map_err(|error| mutation_error(error, EXIT_VALIDATION))? + { + println!(" Generated server API key"); + } + } let admin_password = if bootstrap_admin { println!("Create the local administrator for this new installation."); Some(read_user_password( @@ -2728,6 +2732,10 @@ async fn onboard(path: &Path, args: crate::cli::OnboardArgs) -> Result<(), Boots .await } +fn should_generate_legacy_server_key(identity_initialized: bool) -> bool { + !identity_initialized +} + async fn serve(path: &Path, args: &ServeArgs) -> Result<(), BootstrapError> { validate_root(args.as_root)?; let config = config::Config::from_toml(path)?; @@ -3856,7 +3864,7 @@ fn format_bytes(value: Option<&Value>) -> String { #[cfg(test)] mod tests { - use super::{EXIT_VALIDATION, mutation_error}; + use super::{EXIT_VALIDATION, mutation_error, should_generate_legacy_server_key}; use crate::operations::config_mutation::MutationError; #[test] @@ -3872,4 +3880,10 @@ mod tests { let error = mutation_error(MutationError::Invalid("bad value".into()), EXIT_VALIDATION); assert_eq!(error.exit_code(), EXIT_VALIDATION); } + + #[test] + fn onboarding_does_not_create_a_legacy_key_after_admin_initialization() { + assert!(should_generate_legacy_server_key(false)); + assert!(!should_generate_legacy_server_key(true)); + } } From 83c0779ae4776a6b9d4d82571c7255bcb71246d6 Mon Sep 17 00:00:00 2001 From: wr3n-ai Date: Fri, 16 Oct 2026 05:21:40 +0000 Subject: [PATCH 61/62] Measure M023 short-window throttling --- scripts/qualification_sbc.py | 100 +++++++++++++++++++++++- tests/tooling/test_qualification_sbc.py | 16 ++++ 2 files changed, 115 insertions(+), 1 deletion(-) diff --git a/scripts/qualification_sbc.py b/scripts/qualification_sbc.py index 819d0057..c6c7d327 100644 --- a/scripts/qualification_sbc.py +++ b/scripts/qualification_sbc.py @@ -2626,6 +2626,7 @@ def _concurrent_finite_benchmark( competing_io_mib_per_second: int = 0, competing_io_max_mib: int = 0, api_tokens: Sequence[str] = (), + expected_rate_limited_count: int | None = None, ) -> dict[str, Any]: """Run a bounded finite batch with optional same-filesystem I/O load.""" if not 1 <= client_concurrency <= CONCURRENCY_WORKERS_MAX: @@ -2720,16 +2721,29 @@ def request_once(item: int) -> tuple[int, bytes, int, int | None]: elapsed_ms = [result[2] for result in results] ttft_ms = [result[3] for result in results if result[3] is not None] completed = sum(1 for status, body, _, _ in results if status == 200 and body) + rate_limited = sum( + 1 + for status, body, _, _ in results + if status == 429 and b"short_window_rate_limited" in body + ) status_counts: dict[str, int] = {} for status, _, _, _ in results: key = str(status) status_counts[key] = status_counts.get(key, 0) + 1 + expected_throttle_pass = _expected_short_window_throttle_counts( + results, expected_rate_limited_count, request_count + ) return { - "status": "measured" if completed == request_count else "fail", + "status": "measured" + if completed == request_count or expected_throttle_pass + else "fail", "request_count": request_count, "client_concurrency": client_concurrency, "completed_count": completed, "failed_count": request_count - completed, + "rate_limited_count": rate_limited, + "expected_rate_limited_count": expected_rate_limited_count, + "expected_throttle_pass": expected_throttle_pass, "http_status_counts": status_counts, "batch_elapsed_ms": round(elapsed_seconds * 1000), "requests_per_second": round(request_count / elapsed_seconds, 3), @@ -2751,6 +2765,25 @@ def request_once(item: int) -> tuple[int, bytes, int, int | None]: } +def _expected_short_window_throttle_counts( + results: Sequence[tuple[int, bytes, int, int | None]], + expected_rate_limited_count: int | None, + request_count: int, +) -> bool: + if expected_rate_limited_count is None or len(results) != request_count: + return False + admitted = sum(status == 200 and bool(body) for status, body, _, _ in results) + limited = sum( + status == 429 and b"short_window_rate_limited" in body + for status, body, _, _ in results + ) + return ( + admitted == request_count - expected_rate_limited_count + and limited == expected_rate_limited_count + and all(status in {200, 429} for status, _, _, _ in results) + ) + + def _command_token( binary: Path, config: Path, env: Mapping[str, str], username: str, timeout: float ) -> str: @@ -3203,6 +3236,7 @@ def run_qualification( competing_io_max_mib: int = 0, benchmark_principals: int = 1, benchmark_budget_mode: str = "none", + admission_fairness_rate_per_minute_per_principal: int | None = None, target_class: str = "sbc", provider_latency_profile: str = "instant", ) -> dict[str, Any]: @@ -3225,6 +3259,20 @@ def run_qualification( raise ValueError("benchmark principals must be between 1 and 49") if benchmark_budget_mode not in {"none", "user", "global", "user-global"}: raise ValueError("invalid benchmark budget mode") + if admission_fairness_rate_per_minute_per_principal is not None and not ( + 1 <= admission_fairness_rate_per_minute_per_principal <= 60_000 + ): + raise ValueError("admission fairness rate must be between 1 and 60000") + if admission_fairness_rate_per_minute_per_principal is not None and ( + benchmark_samples <= 0 + or benchmark_budget_mode != "user-global" + or benchmark_principals < 2 + or concurrency_requests < benchmark_principals + ): + raise ValueError( + "active fairness qualification requires benchmark mode, multiple " + "principals, and user-global budgets" + ) if target_class not in {"sbc", "ordinary-server"}: raise ValueError("target class must be sbc or ordinary-server") if provider_latency_profile not in {"instant", "mixed-0-10-50ms"}: @@ -4619,6 +4667,42 @@ def benchmark_sample( ) benchmark["principal_count"] = benchmark_principals benchmark["budget_mode"] = benchmark_budget_mode + expected_rate_limited_count = None + if admission_fairness_rate_per_minute_per_principal is not None: + # Keep setup and warm-up work outside the measured policy. + # Rehash installs the limiter immediately before the + # concurrent batch, whose fresh principals each start + # with exactly one token at the selected 1/minute rate. + with config.open("a", encoding="utf-8") as config_file: + config_file.write( + "\n[admission_fairness]\n" + "requests_per_minute_per_principal = " + f"{admission_fairness_rate_per_minute_per_principal}\n" + ) + rehash_result = _command( + "activate-admission-fairness", + binary, + config, + env, + ("rehash", "--json"), + timeout, + ) + commands.append(rehash_result) + if rehash_result.status != "pass": + raise QualificationError( + "could not activate the measured admission policy" + ) + expected_rate_limited_count = ( + concurrency_requests - benchmark_principals + ) + benchmark["admission_fairness"] = { + "requests_per_minute_per_principal": ( + admission_fairness_rate_per_minute_per_principal + ), + "configuration": "live rehash immediately before batch", + "expected_initially_admitted": benchmark_principals, + "expected_rate_limited": expected_rate_limited_count, + } concurrent = _concurrent_finite_benchmark( benchmark_process, port, @@ -4629,6 +4713,7 @@ def benchmark_sample( competing_io_mib_per_second=competing_io_mib_per_second, competing_io_max_mib=competing_io_max_mib, api_tokens=principal_tokens, + expected_rate_limited_count=expected_rate_limited_count, ) benchmark["runs"]["concurrent_native_responses_finite"] = concurrent if diagnose_finite_tail is not None: @@ -5056,6 +5141,16 @@ def _parser() -> argparse.ArgumentParser: default="none", help="Apply no budget, hard per-user, hard global, or both request budgets.", ) + parser.add_argument( + "--admission-fairness-rate-per-minute-per-principal", + type=int, + metavar="N", + help=( + "For an ordinary-server benchmark, enable the 60-second short-window " + "limiter immediately before the concurrent batch and measure expected " + "429 rejections at this per-principal rate." + ), + ) parser.add_argument( "--diagnose-finite-tail", type=_diagnose_sample_count, @@ -5212,6 +5307,9 @@ def main(argv: Sequence[str] | None = None) -> int: competing_io_max_mib=args.competing_io_max_mib, benchmark_principals=args.benchmark_principals, benchmark_budget_mode=args.benchmark_budget_mode, + admission_fairness_rate_per_minute_per_principal=( + args.admission_fairness_rate_per_minute_per_principal + ), target_class=args.target_class, provider_latency_profile=args.provider_latency_profile, ) diff --git a/tests/tooling/test_qualification_sbc.py b/tests/tooling/test_qualification_sbc.py index 9af6c3fa..6edddcb1 100644 --- a/tests/tooling/test_qualification_sbc.py +++ b/tests/tooling/test_qualification_sbc.py @@ -44,6 +44,7 @@ _diagnose_sample_count, _diagnostic_phase_summary, _direct_provider_control, + _expected_short_window_throttle_counts, _m007_candidate_gates_pass, _ns_to_ms, _percentile, @@ -76,6 +77,21 @@ ) +def test_m023_throttle_measurement_requires_expected_429_code_and_admission_count() -> ( + None +): + results = [ + (200, b"{}", 3, None), + (200, b"{}", 4, None), + (429, b'{"code":"short_window_rate_limited"}', 1, None), + (429, b'{"code":"short_window_rate_limited"}', 1, None), + ] + assert _expected_short_window_throttle_counts(results, 2, 4) + assert not _expected_short_window_throttle_counts(results, 1, 4) + wrong_code = [*results[:-1], (429, b'{"code":"budget_exceeded"}', 1, None)] + assert not _expected_short_window_throttle_counts(wrong_code, 2, 4) + + def test_q008_refuses_non_linux_or_non_aarch64_without_claiming_pass() -> None: report = run_qualification(binary=Path("/not/a/candidate")) if platform.system().lower() != "linux" or platform.machine().lower() not in { From e68a4021e52eae0e480dc1dd24e76edb4fb87982 Mon Sep 17 00:00:00 2001 From: wr3n-ai Date: Fri, 16 Oct 2026 05:40:33 +0000 Subject: [PATCH 62/62] Close M023 with ordinary Linux qualification --- .../run-1.json | 1104 +++++++++++++++++ .../run-2.json | 1104 +++++++++++++++++ .../run-3-blocked-attempt.json | 48 + .../run-3.json | 1104 +++++++++++++++++ .../summary.json | 86 ++ .../comparison.json | 308 +++++ .../controls/u1-c1-repeat-1.json | 1083 ++++++++++++++++ .../controls/u1-c1-repeat-2.json | 1083 ++++++++++++++++ .../controls/u1-c1-repeat-3.json | 1083 ++++++++++++++++ .../controls/u49-c1-repeat-1.json | 1083 ++++++++++++++++ .../controls/u49-c1-repeat-2.json | 1083 ++++++++++++++++ .../controls/u49-c1-repeat-3.json | 1083 ++++++++++++++++ .../controls/u5-c1-repeat-1.json | 1083 ++++++++++++++++ .../controls/u5-c1-repeat-2.json | 1083 ++++++++++++++++ .../controls/u5-c1-repeat-3.json | 1083 ++++++++++++++++ .../lifecycle.json | 502 ++++++++ .../m023-ordinary-server-no-limit/matrix.json | 174 +++ .../summary.json | 522 ++++++++ .../u1-c1-run-1.json | 1080 ++++++++++++++++ .../u1-c1-run-2.json | 1080 ++++++++++++++++ .../u1-c1-run-3.json | 1080 ++++++++++++++++ .../u10-c1-run-1.json | 1083 ++++++++++++++++ .../u10-c1-run-2.json | 1083 ++++++++++++++++ .../u10-c1-run-3.json | 1083 ++++++++++++++++ .../u25-c1-run-1.json | 1083 ++++++++++++++++ .../u25-c1-run-2.json | 1083 ++++++++++++++++ .../u25-c1-run-3.json | 1083 ++++++++++++++++ .../u49-c1-run-1.json | 1083 ++++++++++++++++ .../u49-c1-run-2.json | 1083 ++++++++++++++++ .../u49-c1-run-3.json | 1083 ++++++++++++++++ .../u5-c1-run-1.json | 1080 ++++++++++++++++ .../u5-c1-run-2.json | 1080 ++++++++++++++++ .../u5-c1-run-3.json | 1080 ++++++++++++++++ .../u5-c16-run-1.json | 1083 ++++++++++++++++ .../u5-c16-run-2.json | 1083 ++++++++++++++++ .../u5-c16-run-3.json | 1083 ++++++++++++++++ .../u5-c2-run-1.json | 1080 ++++++++++++++++ .../u5-c2-run-2.json | 1080 ++++++++++++++++ .../u5-c2-run-3.json | 1080 ++++++++++++++++ .../u5-c4-run-1.json | 1083 ++++++++++++++++ .../u5-c4-run-2.json | 1083 ++++++++++++++++ .../u5-c4-run-3.json | 1083 ++++++++++++++++ .../u5-c8-run-1.json | 1083 ++++++++++++++++ .../u5-c8-run-2.json | 1083 ++++++++++++++++ .../u5-c8-run-3.json | 1083 ++++++++++++++++ .../023-performance-qualification.md | 134 ++ plans/registry.md | 6 +- .../subsystems/on-prem-user-access-roadmap.md | 24 +- 48 files changed, 44063 insertions(+), 14 deletions(-) create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-1.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-2.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-3-blocked-attempt.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-3.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/summary.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/comparison.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u1-c1-repeat-1.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u1-c1-repeat-2.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u1-c1-repeat-3.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u49-c1-repeat-1.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u49-c1-repeat-2.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u49-c1-repeat-3.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u5-c1-repeat-1.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u5-c1-repeat-2.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u5-c1-repeat-3.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/lifecycle.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/matrix.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/summary.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u1-c1-run-1.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u1-c1-run-2.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u1-c1-run-3.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u10-c1-run-1.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u10-c1-run-2.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u10-c1-run-3.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u25-c1-run-1.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u25-c1-run-2.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u25-c1-run-3.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u49-c1-run-1.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u49-c1-run-2.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u49-c1-run-3.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c1-run-1.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c1-run-2.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c1-run-3.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c16-run-1.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c16-run-2.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c16-run-3.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c2-run-1.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c2-run-2.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c2-run-3.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c4-run-1.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c4-run-2.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c4-run-3.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c8-run-1.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c8-run-2.json create mode 100644 artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c8-run-3.json create mode 100644 plans/closure/on-prem-user-access/023-performance-qualification.md diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-1.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-1.json new file mode 100644 index 00000000..c893b5ee --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-1.json @@ -0,0 +1,1104 @@ +{ + "backup": { + "archive_sha256": "6e56ba937baabb05ae571903717f0e75263cb8bad47133937a8d420423bacccb", + "archive_size_bytes": 888279, + "elapsed_ms": 132, + "member_count": 3 + }, + "benchmark": { + "admission_fairness": { + "configuration": "live rehash immediately before batch", + "expected_initially_admitted": 5, + "expected_rate_limited": 995, + "requests_per_minute_per_principal": 1 + }, + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 16, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23773184, + "baseline_rss_bytes": 23773184, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 45158400, + "peak_rss_bytes": 45158400 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 977, + "client_concurrency": 16, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 5, + "cpu": { + "cpu_percent": 74.73, + "cpu_ticks": 73, + "cpu_time_ms": 730 + }, + "expected_rate_limited_count": 995, + "expected_throttle_pass": true, + "failed_count": 995, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 8.04, + "avg300": 10.22, + "avg60": 21.26 + }, + "some": { + "avg10": 8.11, + "avg300": 10.32, + "avg60": 21.48 + } + }, + "io_pressure_before": { + "full": { + "avg10": 9.6, + "avg300": 10.28, + "avg60": 21.95 + }, + "some": { + "avg10": 9.68, + "avg300": 10.39, + "avg60": 22.17 + } + }, + "root_device_read_delta_bytes": 4096, + "root_device_write_delta_bytes": 4665344 + }, + "http_status_counts": { + "200": 5, + "429": 995 + }, + "rate_limited_count": 995, + "request_count": 1000, + "requests_per_second": 1023.643, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 132, + "minimum_elapsed_ms": 1, + "p50_elapsed_ms": 13, + "p50_ttft_ms": 6, + "p95_elapsed_ms": 30, + "p95_ttft_ms": 128, + "p99_elapsed_ms": 113, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 55, + "minimum_elapsed_ms": 55, + "model": "q008-responses", + "p50_elapsed_ms": 55, + "p50_ttft_ms": 55, + "p95_elapsed_ms": 55, + "p95_ttft_ms": 55, + "sample_count": 1, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 59, + "model": "q008-responses", + "p50_elapsed_ms": 59, + "p50_ttft_ms": 57, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 1, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 55, + "minimum_elapsed_ms": 55, + "model": "q008-messages", + "p50_elapsed_ms": 55, + "p50_ttft_ms": 53, + "p95_elapsed_ms": 55, + "p95_ttft_ms": 53, + "sample_count": 1, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 6, + "warmup_count": 5 + } + }, + "sample_count": 1 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "83c0779ae4776a6b9d4d82571c7255bcb71246d6", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-8a2ai9ih/config.toml Server: 127.0.0.1:43873 Accounts: 1 Database: Content digest: 27ec78b142f46367f27c892af458dc6880ad54ff33207b4ca9c4f054c9b5ef15", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 160, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 67, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2544283, \"ppid\": 2544239, \"uptime_seconds\": 2.836687477, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "activate-admission-fairness", + "reason": "{ \"ok\": true, \"stage\": \"activation\", \"exit_code\": 0, \"generation\": 2, \"changed_sections\": [ \"admission_fairness\" ], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": true, \"message\": \"reload applied\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 4, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"activation\", \"exit_code\": 0, \"generation\": 3, \"changed_sections\": [ \"admission_fairness\" ], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": true, \"message\": \"reload applied\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 132, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-8a2ai9ih/backups/eggpool-backup-20261016-052156.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 55, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2544283)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 155, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-8a2ai9ih/recovery/config.toml db: /tmp/eggpool-q008-8a2ai9ih/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 91, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2544380)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 31, + "pending_requests": 0, + "request_statuses": { + "completed": 31 + }, + "requests": 31, + "reservations": 31 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 8, + "/responses": 20 + }, + "loopback_provider_requests": 31, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 12.77, + "avg300": 10.38, + "avg60": 23.22 + }, + "some": { + "avg10": 12.89, + "avg300": 10.48, + "avg60": 23.46 + } + }, + "root_device_read_bytes": 761832206336, + "root_device_write_bytes": 1362091955200 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 11, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 11 + }, + { + "body_bytes": 309, + "elapsed_ms": 10, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 213, + "elapsed_ms": 9, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 145, + "elapsed_ms": 20, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 270, + "elapsed_ms": 20, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 480, + "elapsed_ms": 60, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21884928, + 21884928, + 23773184, + 23773184, + 23810048, + 23834624, + 23851008, + 45158400, + 45158400, + 45158400, + 23121920 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 11, + 10, + 9 + ], + "finite_ttft_ms": [ + 11, + 9, + 9 + ], + "second_workload_elapsed_ms": [ + 64, + 11 + ], + "second_workload_ttft_ms": [ + 64, + 11 + ], + "streaming_elapsed_ms": [ + 20, + 20, + 60 + ], + "streaming_ttft_ms": [ + 17, + 17, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21884928, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21884928, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21884928, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21884928, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23773184, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23773184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172965888, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23773184, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23773184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172965888, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 12, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23810048, + "pending_requests": 0, + "request_statuses": { + "completed": 12 + }, + "requests": 12, + "reservations": 12, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23810048, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172965888, + "wal_bytes": 3201272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 18, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23834624, + "pending_requests": 0, + "request_statuses": { + "completed": 18 + }, + "requests": 18, + "reservations": 18, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23834624, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172965888, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 24, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23851008, + "pending_requests": 0, + "request_statuses": { + "completed": 24 + }, + "requests": 24, + "reservations": 24, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23851008, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172965888, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 29, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 45158400, + "pending_requests": 0, + "request_statuses": { + "completed": 29 + }, + "requests": 29, + "reservations": 29, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 45158400, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173928448, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 29, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 45158400, + "pending_requests": 0, + "request_statuses": { + "completed": 29 + }, + "requests": 29, + "reservations": 29, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 45158400, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173928448, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 29, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 45158400, + "pending_requests": 0, + "request_statuses": { + "completed": 29 + }, + "requests": 29, + "reservations": 29, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 45158400, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173928448, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 31, + "cpu_percent": 0.0, + "db_bytes": 872448, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23121920, + "pending_requests": 0, + "request_statuses": { + "completed": 31 + }, + "requests": 31, + "reservations": 31, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23121920, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 622152, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 34, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-2.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-2.json new file mode 100644 index 00000000..7dd3d8c6 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-2.json @@ -0,0 +1,1104 @@ +{ + "backup": { + "archive_sha256": "04c235b020d960c60fd6dd443814f801f3c906a9d2e2cbea4098230c6146c6ac", + "archive_size_bytes": 888279, + "elapsed_ms": 607, + "member_count": 3 + }, + "benchmark": { + "admission_fairness": { + "configuration": "live rehash immediately before batch", + "expected_initially_admitted": 5, + "expected_rate_limited": 995, + "requests_per_minute_per_principal": 1 + }, + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 16, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23629824, + "baseline_rss_bytes": 23629824, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 44855296, + "peak_rss_bytes": 44855296 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 830, + "client_concurrency": 16, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 5, + "cpu": { + "cpu_percent": 26.5, + "cpu_ticks": 22, + "cpu_time_ms": 220 + }, + "expected_rate_limited_count": 995, + "expected_throttle_pass": true, + "failed_count": 995, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 8.88, + "avg300": 9.96, + "avg60": 17.68 + }, + "some": { + "avg10": 9.13, + "avg300": 10.08, + "avg60": 17.9 + } + }, + "io_pressure_before": { + "full": { + "avg10": 8.88, + "avg300": 9.96, + "avg60": 17.68 + }, + "some": { + "avg10": 9.13, + "avg300": 10.08, + "avg60": 17.9 + } + }, + "root_device_read_delta_bytes": 81920, + "root_device_write_delta_bytes": 29990912 + }, + "http_status_counts": { + "200": 5, + "429": 995 + }, + "rate_limited_count": 995, + "request_count": 1000, + "requests_per_second": 1204.462, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 565, + "minimum_elapsed_ms": 1, + "p50_elapsed_ms": 4, + "p50_ttft_ms": 7, + "p95_elapsed_ms": 9, + "p95_ttft_ms": 564, + "p99_elapsed_ms": 553, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 53, + "minimum_elapsed_ms": 53, + "model": "q008-responses", + "p50_elapsed_ms": 53, + "p50_ttft_ms": 53, + "p95_elapsed_ms": 53, + "p95_ttft_ms": 53, + "sample_count": 1, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 57, + "minimum_elapsed_ms": 57, + "model": "q008-responses", + "p50_elapsed_ms": 57, + "p50_ttft_ms": 56, + "p95_elapsed_ms": 57, + "p95_ttft_ms": 56, + "sample_count": 1, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 55, + "minimum_elapsed_ms": 55, + "model": "q008-messages", + "p50_elapsed_ms": 55, + "p50_ttft_ms": 53, + "p95_elapsed_ms": 55, + "p95_ttft_ms": 53, + "sample_count": 1, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 6, + "warmup_count": 5 + } + }, + "sample_count": 1 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "83c0779ae4776a6b9d4d82571c7255bcb71246d6", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-mcrpqvgy/config.toml Server: 127.0.0.1:39457 Accounts: 1 Database: Content digest: 1d346aab41a2d2135a1c7f527e50404bf61809e17eeaa016885837b036e1f31a", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 150, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 73, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 5, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2544429, \"ppid\": 2544405, \"uptime_seconds\": 2.816843759, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "activate-admission-fairness", + "reason": "{ \"ok\": true, \"stage\": \"activation\", \"exit_code\": 0, \"generation\": 2, \"changed_sections\": [ \"admission_fairness\" ], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": true, \"message\": \"reload applied\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 51, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"activation\", \"exit_code\": 0, \"generation\": 3, \"changed_sections\": [ \"admission_fairness\" ], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": true, \"message\": \"reload applied\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 607, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-mcrpqvgy/backups/eggpool-backup-20261016-052213.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 310, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2544429)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 681, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-mcrpqvgy/recovery/config.toml db: /tmp/eggpool-q008-mcrpqvgy/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 593, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 419, + "id": "stop", + "reason": "Stopping server (PID 2544568)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 31, + "pending_requests": 0, + "request_statuses": { + "completed": 31 + }, + "requests": 31, + "reservations": 31 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 8, + "/responses": 20 + }, + "loopback_provider_requests": 31, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 3.18, + "avg300": 9.89, + "avg60": 18.06 + }, + "some": { + "avg10": 3.3, + "avg300": 10.0, + "avg60": 18.28 + } + }, + "root_device_read_bytes": 761844703232, + "root_device_write_bytes": 1362115228672 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 3, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21544960, + 21544960, + 23629824, + 23629824, + 23662592, + 23683072, + 23719936, + 44855296, + 44855296, + 44855296, + 23101440 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 3, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 61, + 3 + ], + "second_workload_ttft_ms": [ + 61, + 3 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 56 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21544960, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21544960, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21544960, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21544960, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23629824, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23629824, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23629824, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23629824, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 12, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23662592, + "pending_requests": 0, + "request_statuses": { + "completed": 12 + }, + "requests": 12, + "reservations": 12, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23662592, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 3201272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 18, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23683072, + "pending_requests": 0, + "request_statuses": { + "completed": 18 + }, + "requests": 18, + "reservations": 18, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23683072, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 24, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23719936, + "pending_requests": 0, + "request_statuses": { + "completed": 24 + }, + "requests": 24, + "reservations": 24, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23719936, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 29, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 44855296, + "pending_requests": 0, + "request_statuses": { + "completed": 29 + }, + "requests": 29, + "reservations": 29, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 44855296, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173928448, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 29, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 44855296, + "pending_requests": 0, + "request_statuses": { + "completed": 29 + }, + "requests": 29, + "reservations": 29, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 44855296, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173928448, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 29, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 44855296, + "pending_requests": 0, + "request_statuses": { + "completed": 29 + }, + "requests": 29, + "reservations": 29, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 44855296, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173928448, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 31, + "cpu_percent": 0.0, + "db_bytes": 872448, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23101440, + "pending_requests": 0, + "request_statuses": { + "completed": 31 + }, + "requests": 31, + "reservations": 31, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23101440, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 622152, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 51, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-3-blocked-attempt.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-3-blocked-attempt.json new file mode 100644 index 00000000..39cb74f8 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-3-blocked-attempt.json @@ -0,0 +1,48 @@ +{ + "commands": [], + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 28.33, + "avg300": 11.05, + "avg60": 21.38 + }, + "some": { + "avg10": 28.75, + "avg300": 11.18, + "avg60": 21.66 + } + }, + "root_device_read_bytes": 761853444096, + "root_device_write_bytes": 1362586076160 + }, + "product_name": "Z390 AORUS MASTER", + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [], + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "reason": "ordinary-server benchmark stopped before load because initial full I/O pressure is missing or at/above the 25% avg10 gate", + "resource_samples": [], + "schema_version": "runtime-q008.v2", + "status": "blocked" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-3.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-3.json new file mode 100644 index 00000000..a287c450 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-3.json @@ -0,0 +1,1104 @@ +{ + "backup": { + "archive_sha256": "605a973c99da2117052a236fb74e7db682f0f6d730a7796fbb92aa926af2a46e", + "archive_size_bytes": 888279, + "elapsed_ms": 112, + "member_count": 3 + }, + "benchmark": { + "admission_fairness": { + "configuration": "live rehash immediately before batch", + "expected_initially_admitted": 5, + "expected_rate_limited": 995, + "requests_per_minute_per_principal": 1 + }, + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 16, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23752704, + "baseline_rss_bytes": 23752704, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 45199360, + "peak_rss_bytes": 45199360 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 916, + "client_concurrency": 16, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 5, + "cpu": { + "cpu_percent": 79.7, + "cpu_ticks": 73, + "cpu_time_ms": 730 + }, + "expected_rate_limited_count": 995, + "expected_throttle_pass": true, + "failed_count": 995, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 0.15, + "avg300": 9.24, + "avg60": 8.99 + }, + "some": { + "avg10": 0.15, + "avg300": 9.35, + "avg60": 9.11 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.15, + "avg300": 9.24, + "avg60": 8.99 + }, + "some": { + "avg10": 0.15, + "avg300": 9.35, + "avg60": 9.11 + } + }, + "root_device_read_delta_bytes": 1626112, + "root_device_write_delta_bytes": 4673536 + }, + "http_status_counts": { + "200": 5, + "429": 995 + }, + "rate_limited_count": 995, + "request_count": 1000, + "requests_per_second": 1091.728, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 55, + "minimum_elapsed_ms": 1, + "p50_elapsed_ms": 13, + "p50_ttft_ms": 18, + "p95_elapsed_ms": 29, + "p95_ttft_ms": 49, + "p99_elapsed_ms": 39, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 52, + "minimum_elapsed_ms": 52, + "model": "q008-responses", + "p50_elapsed_ms": 52, + "p50_ttft_ms": 52, + "p95_elapsed_ms": 52, + "p95_ttft_ms": 52, + "sample_count": 1, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 53, + "minimum_elapsed_ms": 53, + "model": "q008-responses", + "p50_elapsed_ms": 53, + "p50_ttft_ms": 52, + "p95_elapsed_ms": 53, + "p95_ttft_ms": 52, + "sample_count": 1, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 60, + "model": "q008-messages", + "p50_elapsed_ms": 60, + "p50_ttft_ms": 57, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 1, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 6, + "warmup_count": 5 + } + }, + "sample_count": 1 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "83c0779ae4776a6b9d4d82571c7255bcb71246d6", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-931yufko/config.toml Server: 127.0.0.1:53629 Accounts: 1 Database: Content digest: 90f7c3ee25fe59f42de7a232feba4742af378826d070e4ecc0c9a34241be7d32", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 85, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 11, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 23, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 24, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 23, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 12, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2544809, \"ppid\": 2544785, \"uptime_seconds\": 2.873840163, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "activate-admission-fairness", + "reason": "{ \"ok\": true, \"stage\": \"activation\", \"exit_code\": 0, \"generation\": 2, \"changed_sections\": [ \"admission_fairness\" ], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": true, \"message\": \"reload applied\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"activation\", \"exit_code\": 0, \"generation\": 3, \"changed_sections\": [ \"admission_fairness\" ], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": true, \"message\": \"reload applied\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 112, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-931yufko/backups/eggpool-backup-20261016-052315.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2544809)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 78, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-931yufko/recovery/config.toml db: /tmp/eggpool-q008-931yufko/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 29, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2545019)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 31, + "pending_requests": 0, + "request_statuses": { + "completed": 31 + }, + "requests": 31, + "reservations": 31 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 8, + "/responses": 20 + }, + "loopback_provider_requests": 31, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.28, + "avg300": 9.43, + "avg60": 9.94 + }, + "some": { + "avg10": 0.28, + "avg300": 9.54, + "avg60": 10.07 + } + }, + "root_device_read_bytes": 761863606272, + "root_device_write_bytes": 1362598687744 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 6, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 6 + }, + { + "body_bytes": 309, + "elapsed_ms": 5, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 5 + }, + { + "body_bytes": 213, + "elapsed_ms": 5, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 5 + }, + { + "body_bytes": 145, + "elapsed_ms": 15, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 14 + }, + { + "body_bytes": 270, + "elapsed_ms": 14, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 13 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 54 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21856256, + 21856256, + 23752704, + 23752704, + 23773184, + 23814144, + 23834624, + 45199360, + 45199360, + 45199360, + 22765568 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 6, + 5, + 5 + ], + "finite_ttft_ms": [ + 6, + 5, + 5 + ], + "second_workload_elapsed_ms": [ + 64, + 11 + ], + "second_workload_ttft_ms": [ + 64, + 11 + ], + "streaming_elapsed_ms": [ + 15, + 14, + 56 + ], + "streaming_ttft_ms": [ + 14, + 13, + 54 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21856256, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21856256, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21856256, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21856256, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23752704, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23752704, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23752704, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23752704, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 12, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23773184, + "pending_requests": 0, + "request_statuses": { + "completed": 12 + }, + "requests": 12, + "reservations": 12, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23773184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 3201272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 18, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23814144, + "pending_requests": 0, + "request_statuses": { + "completed": 18 + }, + "requests": 18, + "reservations": 18, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23814144, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 24, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23834624, + "pending_requests": 0, + "request_statuses": { + "completed": 24 + }, + "requests": 24, + "reservations": 24, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23834624, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 29, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 45199360, + "pending_requests": 0, + "request_statuses": { + "completed": 29 + }, + "requests": 29, + "reservations": 29, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 45199360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173924352, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 29, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 45199360, + "pending_requests": 0, + "request_statuses": { + "completed": 29 + }, + "requests": 29, + "reservations": 29, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 45199360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173924352, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 29, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 45199360, + "pending_requests": 0, + "request_statuses": { + "completed": 29 + }, + "requests": 29, + "reservations": 29, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 45199360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173924352, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 31, + "cpu_percent": 0.0, + "db_bytes": 872448, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 22765568, + "pending_requests": 0, + "request_statuses": { + "completed": 31 + }, + "requests": 31, + "reservations": 31, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 22765568, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 622152, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 21, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/summary.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/summary.json new file mode 100644 index 00000000..4ec70469 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/summary.json @@ -0,0 +1,86 @@ +{ + "schema_version": "m023-active-throttle.v1", + "candidate_sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73", + "candidate_source_commit": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "measurement_tooling_commit": "83c0779ae4776a6b9d4d82571c7255bcb71246d6", + "target": { + "target_class": "ordinary-server", + "architecture": "x86_64", + "os": "Ubuntu 24.04.5 LTS", + "kernel": "6.8.0-147-generic", + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "logical_cpu_count": 16, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_storage_device_model": "Samsung SSD 980 1TB", + "virtualization_type": "none" + }, + "policy": { + "requests_per_minute_per_principal": 1, + "principals": 5, + "client_concurrency": 16, + "requests_per_run": 1000 + }, + "repetitions": 3, + "all_runs_passed": true, + "median": { + "admitted": 5, + "rate_limited": 995, + "requests_per_second": 1091.728, + "p50_ms": 13, + "p95_ms": 29, + "p99_ms": 113 + }, + "runs": [ + { + "artifact": "run-1.json", + "status": "pass", + "preflight_full_io_psi_avg10": 12.77, + "http_status_counts": { + "200": 5, + "429": 995 + }, + "admitted": 5, + "rate_limited": 995, + "requests_per_second": 1023.643, + "p50_ms": 13, + "p95_ms": 30, + "p99_ms": 113, + "maximum_ms": 132 + }, + { + "artifact": "run-2.json", + "status": "pass", + "preflight_full_io_psi_avg10": 3.18, + "http_status_counts": { + "200": 5, + "429": 995 + }, + "admitted": 5, + "rate_limited": 995, + "requests_per_second": 1204.462, + "p50_ms": 4, + "p95_ms": 9, + "p99_ms": 553, + "maximum_ms": 565 + }, + { + "artifact": "run-3.json", + "status": "pass", + "preflight_full_io_psi_avg10": 0.28, + "http_status_counts": { + "200": 5, + "429": 995 + }, + "admitted": 5, + "rate_limited": 995, + "requests_per_second": 1091.728, + "p50_ms": 13, + "p95_ms": 29, + "p99_ms": 39, + "maximum_ms": 55 + } + ], + "blocked_preflight_attempt": "run-3-blocked-attempt.json", + "interpretation": "Loopback synthetic-provider measurement; exactly the initial token per principal was admitted, all remaining requests returned the distinct short_window_rate_limited 429. Not a capacity SLA." +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/comparison.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/comparison.json new file mode 100644 index 00000000..7c3881b8 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/comparison.json @@ -0,0 +1,308 @@ +{ + "schema_version": "m023-ordinary-server-performance-comparison.v1", + "candidate": { + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "build_profile": "release", + "build_mode": "locked release candidate" + }, + "target": { + "target_class": "ordinary-server", + "architecture": "x86_64", + "kernel": "6.8.0-147-generic", + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "logical_cpu_count": 16, + "ram_bytes": 16622407680, + "system_vendor": "Gigabyte Technology Co., Ltd.", + "product_name": "Z390 AORUS MASTER", + "board_name": "Z390 AORUS MASTER-CF", + "root_filesystem": "ext4", + "root_storage_medium_class": "non-rotational", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_mount_options": "rw,relatime,stripe=32", + "virtualization_type": "none", + "container_type": "none", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt" + }, + "interpretation": "Same-host descriptive comparison against M002; no latency SLA or team capacity claim.", + "matrix": { + "conditions": 9, + "repetitions_per_condition": 3, + "requests_per_run": 1000, + "raw_summary": "summary.json", + "cells": [ + { + "principals": 1, + "concurrency": 1, + "m002": { + "p50_ms": 15, + "p95_ms": 82, + "p99_ms": 148, + "requests_per_second": 36.495 + }, + "m023_no_limit": { + "p50_ms": 15, + "p95_ms": 120, + "p99_ms": 180, + "requests_per_second": 32.164 + } + }, + { + "principals": 5, + "concurrency": 1, + "m002": { + "p50_ms": 15, + "p95_ms": 61, + "p99_ms": 89, + "requests_per_second": 42.179 + }, + "m023_no_limit": { + "p50_ms": 15, + "p95_ms": 62, + "p99_ms": 151, + "requests_per_second": 36.941 + } + }, + { + "principals": 5, + "concurrency": 2, + "m002": { + "p50_ms": 14, + "p95_ms": 61, + "p99_ms": 79, + "requests_per_second": 93.976 + }, + "m023_no_limit": { + "p50_ms": 15, + "p95_ms": 62, + "p99_ms": 82, + "requests_per_second": 92.816 + } + }, + { + "principals": 5, + "concurrency": 4, + "m002": { + "p50_ms": 14, + "p95_ms": 59, + "p99_ms": 75, + "requests_per_second": 193.683 + }, + "m023_no_limit": { + "p50_ms": 15, + "p95_ms": 60, + "p99_ms": 82, + "requests_per_second": 190.485 + } + }, + { + "principals": 5, + "concurrency": 8, + "m002": { + "p50_ms": 19, + "p95_ms": 69, + "p99_ms": 76, + "requests_per_second": 335.268 + }, + "m023_no_limit": { + "p50_ms": 22, + "p95_ms": 73, + "p99_ms": 85, + "requests_per_second": 295.215 + } + }, + { + "principals": 5, + "concurrency": 16, + "m002": { + "p50_ms": 35, + "p95_ms": 88, + "p99_ms": 106, + "requests_per_second": 341.347 + }, + "m023_no_limit": { + "p50_ms": 38, + "p95_ms": 91, + "p99_ms": 119, + "requests_per_second": 326.388 + } + }, + { + "principals": 10, + "concurrency": 1, + "m002": { + "p50_ms": 14, + "p95_ms": 61, + "p99_ms": 77, + "requests_per_second": 42.858 + }, + "m023_no_limit": { + "p50_ms": 13, + "p95_ms": 61, + "p99_ms": 109, + "requests_per_second": 41.592 + } + }, + { + "principals": 25, + "concurrency": 1, + "m002": { + "p50_ms": 16, + "p95_ms": 61, + "p99_ms": 81, + "requests_per_second": 42.495 + }, + "m023_no_limit": { + "p50_ms": 14, + "p95_ms": 61, + "p99_ms": 66, + "requests_per_second": 43.399 + } + }, + { + "principals": 49, + "concurrency": 1, + "m002": { + "p50_ms": 15, + "p95_ms": 61, + "p99_ms": 81, + "requests_per_second": 42.322 + }, + "m023_no_limit": { + "p50_ms": 12, + "p95_ms": 126, + "p99_ms": 172, + "requests_per_second": 40.76 + } + } + ] + }, + "controlled_single_concurrency_repeats": { + "u1-c1": { + "repetitions": 3, + "median": { + "p50_ms": 13, + "p95_ms": 96, + "p99_ms": 164, + "requests_per_second": 35.987 + }, + "runs": [ + { + "artifact": "u1-c1-repeat-1.json", + "preflight_full_io_psi_avg10": 0.93, + "p50_ms": 13, + "p95_ms": 96, + "p99_ms": 164, + "requests_per_second": 35.987, + "post_run_full_io_psi_avg10": 5.29 + }, + { + "artifact": "u1-c1-repeat-2.json", + "preflight_full_io_psi_avg10": 3.07, + "p50_ms": 13, + "p95_ms": 126, + "p99_ms": 171, + "requests_per_second": 33.164, + "post_run_full_io_psi_avg10": 6.3 + }, + { + "artifact": "u1-c1-repeat-3.json", + "preflight_full_io_psi_avg10": 4.45, + "p50_ms": 13, + "p95_ms": 62, + "p99_ms": 112, + "requests_per_second": 39.933, + "post_run_full_io_psi_avg10": 1.79 + } + ] + }, + "u5-c1": { + "repetitions": 3, + "median": { + "p50_ms": 15, + "p95_ms": 62, + "p99_ms": 123, + "requests_per_second": 38.753 + }, + "runs": [ + { + "artifact": "u5-c1-repeat-1.json", + "preflight_full_io_psi_avg10": 0.17, + "p50_ms": 15, + "p95_ms": 65, + "p99_ms": 123, + "requests_per_second": 38.753, + "post_run_full_io_psi_avg10": 4.32 + }, + { + "artifact": "u5-c1-repeat-2.json", + "preflight_full_io_psi_avg10": 3.6, + "p50_ms": 14, + "p95_ms": 61, + "p99_ms": 66, + "requests_per_second": 43.178, + "post_run_full_io_psi_avg10": 0.88 + }, + { + "artifact": "u5-c1-repeat-3.json", + "preflight_full_io_psi_avg10": 0.32, + "p50_ms": 15, + "p95_ms": 62, + "p99_ms": 145, + "requests_per_second": 37.399, + "post_run_full_io_psi_avg10": 3.48 + } + ] + }, + "u49-c1": { + "repetitions": 3, + "median": { + "p50_ms": 14, + "p95_ms": 61, + "p99_ms": 81, + "requests_per_second": 43.498 + }, + "runs": [ + { + "artifact": "u49-c1-repeat-1.json", + "preflight_full_io_psi_avg10": 0.02, + "p50_ms": 14, + "p95_ms": 61, + "p99_ms": 73, + "requests_per_second": 43.739, + "post_run_full_io_psi_avg10": 0.48 + }, + { + "artifact": "u49-c1-repeat-2.json", + "preflight_full_io_psi_avg10": 0.36, + "p50_ms": 15, + "p95_ms": 61, + "p99_ms": 81, + "requests_per_second": 43.498, + "post_run_full_io_psi_avg10": 0.47 + }, + { + "artifact": "u49-c1-repeat-3.json", + "preflight_full_io_psi_avg10": 0.21, + "p50_ms": 14, + "p95_ms": 79, + "p99_ms": 149, + "requests_per_second": 37.963, + "post_run_full_io_psi_avg10": 3.1 + } + ] + } + }, + "comparison_baseline_candidate": { + "sha256": "43cdf634d1ffcd6759ad14adca072869101b379f074e8001b4a28cf52f10bce3", + "repository_commit_sha": "1c3d15b8911f1f0f3e003c9e918282c0363736e3", + "build_profile": "release", + "build_mode": "locked release candidate" + }, + "measurement_policy": { + "admission_fairness": "disabled; no [admission_fairness] section in the benchmark fixture" + } +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u1-c1-repeat-1.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u1-c1-repeat-1.json new file mode 100644 index 00000000..b9df2fc7 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u1-c1-repeat-1.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "3872189098d68786b68c7967621288f2b469de4851c55e6648dcd4a4d9e7c950", + "archive_size_bytes": 4369879, + "elapsed_ms": 569, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 1, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23568384, + "baseline_rss_bytes": 23568384, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 25800704, + "peak_rss_bytes": 25800704 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 27788, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 27.78, + "cpu_ticks": 772, + "cpu_time_ms": 7720 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 5.29, + "avg300": 5.93, + "avg60": 3.01 + }, + "some": { + "avg10": 5.36, + "avg300": 6.0, + "avg60": 3.05 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.62, + "avg300": 6.07, + "avg60": 1.99 + }, + "some": { + "avg10": 0.63, + "avg300": 6.14, + "avg60": 2.02 + } + }, + "root_device_read_delta_bytes": 1785856, + "root_device_write_delta_bytes": 382615552 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 35.987, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 217, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 96, + "p95_ttft_ms": 96, + "p99_elapsed_ms": 164, + "p99_ttft_ms": 164, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 60, + "model": "q008-responses", + "p50_elapsed_ms": 60, + "p50_ttft_ms": 59, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 59, + "sample_count": 1, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 57, + "minimum_elapsed_ms": 57, + "model": "q008-responses", + "p50_elapsed_ms": 57, + "p50_ttft_ms": 56, + "p95_elapsed_ms": 57, + "p95_ttft_ms": 56, + "sample_count": 1, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 56, + "minimum_elapsed_ms": 56, + "model": "q008-messages", + "p50_elapsed_ms": 56, + "p50_ttft_ms": 53, + "p95_elapsed_ms": 56, + "p95_ttft_ms": 53, + "sample_count": 1, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 6, + "warmup_count": 5 + } + }, + "sample_count": 1 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "83c0779ae4776a6b9d4d82571c7255bcb71246d6", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-w0s16268/config.toml Server: 127.0.0.1:40963 Accounts: 1 Database: Content digest: 189ab8a72e453c211eff26db48967d9910b529e9e680a34cd7aa8c1d2dde8784", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 86, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 40, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2549366, \"ppid\": 2549342, \"uptime_seconds\": 2.791555827, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 569, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-w0s16268/backups/eggpool-backup-20261016-052604.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 56, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2549366)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 426, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-w0s16268/recovery/config.toml db: /tmp/eggpool-q008-w0s16268/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 194, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2550534)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1026, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 8, + "/responses": 1015 + }, + "loopback_provider_requests": 1026, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.93, + "avg300": 6.16, + "avg60": 2.12 + }, + "some": { + "avg10": 0.95, + "avg300": 6.23, + "avg60": 2.16 + } + }, + "root_device_read_bytes": 761894895616, + "root_device_write_bytes": 1363941667840 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 3, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21676032, + 21680128, + 23568384, + 23568384, + 23588864, + 23629824, + 23654400, + 25800704, + 25800704, + 25800704, + 23564288 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 3, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 27, + 62 + ], + "second_workload_ttft_ms": [ + 27, + 62 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 56 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21676032, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21676032, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21680128, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21680128, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23568384, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23568384, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23568384, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23568384, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 12, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23588864, + "pending_requests": 0, + "request_statuses": { + "completed": 12 + }, + "requests": 12, + "reservations": 12, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23588864, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 3201272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 18, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23629824, + "pending_requests": 0, + "request_statuses": { + "completed": 18 + }, + "requests": 18, + "reservations": 18, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23629824, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 24, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23654400, + "pending_requests": 0, + "request_statuses": { + "completed": 24 + }, + "requests": 24, + "reservations": 24, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23654400, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4341760, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 25800704, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 25800704, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173223936, + "wal_bytes": 4354872, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4341760, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 25800704, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 25800704, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173223936, + "wal_bytes": 4354872, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4341760, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 25800704, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 25800704, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173223936, + "wal_bytes": 4354872, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1026, + "cpu_percent": 0.0, + "db_bytes": 4169728, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23564288, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23564288, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 881712, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 20, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u1-c1-repeat-2.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u1-c1-repeat-2.json new file mode 100644 index 00000000..65c0018a --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u1-c1-repeat-2.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "09df9fcb9eae7b9167acecb4f471d392ba487d94b493aafa740420638d133def", + "archive_size_bytes": 4369879, + "elapsed_ms": 648, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 1, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23891968, + "baseline_rss_bytes": 23891968, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 26095616, + "peak_rss_bytes": 26095616 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 30153, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 24.91, + "cpu_ticks": 751, + "cpu_time_ms": 7510 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 6.3, + "avg300": 5.76, + "avg60": 4.01 + }, + "some": { + "avg10": 6.3, + "avg300": 5.82, + "avg60": 4.03 + } + }, + "io_pressure_before": { + "full": { + "avg10": 2.05, + "avg300": 5.72, + "avg60": 2.61 + }, + "some": { + "avg10": 2.08, + "avg300": 5.79, + "avg60": 2.65 + } + }, + "root_device_read_delta_bytes": 2621440, + "root_device_write_delta_bytes": 381472768 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 33.164, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 294, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 126, + "p95_ttft_ms": 126, + "p99_elapsed_ms": 171, + "p99_ttft_ms": 171, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 55, + "minimum_elapsed_ms": 55, + "model": "q008-responses", + "p50_elapsed_ms": 55, + "p50_ttft_ms": 54, + "p95_elapsed_ms": 55, + "p95_ttft_ms": 54, + "sample_count": 1, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 59, + "model": "q008-responses", + "p50_elapsed_ms": 59, + "p50_ttft_ms": 57, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 1, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 54, + "minimum_elapsed_ms": 54, + "model": "q008-messages", + "p50_elapsed_ms": 54, + "p50_ttft_ms": 53, + "p95_elapsed_ms": 54, + "p95_ttft_ms": 53, + "sample_count": 1, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 6, + "warmup_count": 5 + } + }, + "sample_count": 1 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "83c0779ae4776a6b9d4d82571c7255bcb71246d6", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-9w4n0dtr/config.toml Server: 127.0.0.1:35263 Accounts: 1 Database: Content digest: 46f2639e88473807b1c8ce04efcee47516d7dcf0c2819c72a0dac4ac429f0307", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 115, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 67, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2550567, \"ppid\": 2550543, \"uptime_seconds\": 2.793585579, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 13, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 648, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-9w4n0dtr/backups/eggpool-backup-20261016-052646.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 161, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2550567)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 560, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-9w4n0dtr/recovery/config.toml db: /tmp/eggpool-q008-9w4n0dtr/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 302, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2551735)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1026, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 8, + "/responses": 1015 + }, + "loopback_provider_requests": 1026, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 3.07, + "avg300": 5.8, + "avg60": 2.79 + }, + "some": { + "avg10": 3.1, + "avg300": 5.87, + "avg60": 2.83 + } + }, + "root_device_read_bytes": 761897705472, + "root_device_write_bytes": 1364374516736 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 15, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 13 + }, + { + "body_bytes": 480, + "elapsed_ms": 60, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21864448, + 21864448, + 23891968, + 23891968, + 23920640, + 23945216, + 23969792, + 26095616, + 26095616, + 26095616, + 23851008 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 14, + 57 + ], + "second_workload_ttft_ms": [ + 14, + 57 + ], + "streaming_elapsed_ms": [ + 12, + 15, + 60 + ], + "streaming_ttft_ms": [ + 12, + 13, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21864448, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21864448, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21864448, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21864448, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23891968, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23891968, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23891968, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23891968, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 12, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23920640, + "pending_requests": 0, + "request_statuses": { + "completed": 12 + }, + "requests": 12, + "reservations": 12, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23920640, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 3201272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 18, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23945216, + "pending_requests": 0, + "request_statuses": { + "completed": 18 + }, + "requests": 18, + "reservations": 18, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23945216, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 24, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23969792, + "pending_requests": 0, + "request_statuses": { + "completed": 24 + }, + "requests": 24, + "reservations": 24, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23969792, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4341760, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 26095616, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 26095616, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173101056, + "wal_bytes": 4354872, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4341760, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 26095616, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 26095616, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173101056, + "wal_bytes": 4354872, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4341760, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 26095616, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 26095616, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173101056, + "wal_bytes": 4354872, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1026, + "cpu_percent": 0.0, + "db_bytes": 4169728, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23851008, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23851008, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 894072, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 28, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u1-c1-repeat-3.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u1-c1-repeat-3.json new file mode 100644 index 00000000..9a6bb651 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u1-c1-repeat-3.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "4af9695cbdf339fb33c2ea205d5c1f81d8a0399fb35ce18c27c0aaaeab0d6768", + "archive_size_bytes": 4369879, + "elapsed_ms": 480, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 1, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23797760, + "baseline_rss_bytes": 23797760, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 26017792, + "peak_rss_bytes": 26017792 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 25042, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 30.39, + "cpu_ticks": 761, + "cpu_time_ms": 7610 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 1.79, + "avg300": 5.38, + "avg60": 3.31 + }, + "some": { + "avg10": 1.87, + "avg300": 5.44, + "avg60": 3.38 + } + }, + "io_pressure_before": { + "full": { + "avg10": 3.57, + "avg300": 5.63, + "avg60": 3.76 + }, + "some": { + "avg10": 3.57, + "avg300": 5.69, + "avg60": 3.78 + } + }, + "root_device_read_delta_bytes": 987136, + "root_device_write_delta_bytes": 378822656 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 39.933, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 230, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 62, + "p95_ttft_ms": 62, + "p99_elapsed_ms": 112, + "p99_ttft_ms": 112, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 53, + "minimum_elapsed_ms": 53, + "model": "q008-responses", + "p50_elapsed_ms": 53, + "p50_ttft_ms": 53, + "p95_elapsed_ms": 53, + "p95_ttft_ms": 53, + "sample_count": 1, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 59, + "model": "q008-responses", + "p50_elapsed_ms": 59, + "p50_ttft_ms": 57, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 1, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 59, + "model": "q008-messages", + "p50_elapsed_ms": 59, + "p50_ttft_ms": 57, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 1, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 6, + "warmup_count": 5 + } + }, + "sample_count": 1 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "83c0779ae4776a6b9d4d82571c7255bcb71246d6", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-mv4k3dbv/config.toml Server: 127.0.0.1:50471 Accounts: 1 Database: Content digest: a9224e9b4c92056880d1d2e0bde046b0ec78fbc3e8e0be3046fcdb81bc38c815", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 250, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 76, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 11, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 16, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2551790, \"ppid\": 2551764, \"uptime_seconds\": 2.801082841, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 14, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 480, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-mv4k3dbv/backups/eggpool-backup-20261016-052724.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2551790)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 94, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-mv4k3dbv/recovery/config.toml db: /tmp/eggpool-q008-mv4k3dbv/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 48, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 55, + "id": "stop", + "reason": "Stopping server (PID 2552956)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1026, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 8, + "/responses": 1015 + }, + "loopback_provider_requests": 1026, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 4.45, + "avg300": 5.69, + "avg60": 3.88 + }, + "some": { + "avg10": 4.45, + "avg300": 5.75, + "avg60": 3.9 + } + }, + "root_device_read_bytes": 761900843008, + "root_device_write_bytes": 1364807230464 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 55, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21827584, + 21827584, + 23797760, + 23797760, + 23818240, + 23846912, + 23867392, + 26017792, + 26017792, + 26017792, + 23846912 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 27, + 61 + ], + "second_workload_ttft_ms": [ + 26, + 61 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 55 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21827584, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21827584, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21827584, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21827584, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23797760, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23797760, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23797760, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23797760, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 12, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23818240, + "pending_requests": 0, + "request_statuses": { + "completed": 12 + }, + "requests": 12, + "reservations": 12, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23818240, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 3201272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 18, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23846912, + "pending_requests": 0, + "request_statuses": { + "completed": 18 + }, + "requests": 18, + "reservations": 18, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23846912, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 24, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23867392, + "pending_requests": 0, + "request_statuses": { + "completed": 24 + }, + "requests": 24, + "reservations": 24, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23867392, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4341760, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 26017792, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 26017792, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173228032, + "wal_bytes": 4354872, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4341760, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 26017792, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 26017792, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173228032, + "wal_bytes": 4354872, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4341760, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 26017792, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 26017792, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173228032, + "wal_bytes": 4354872, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1026, + "cpu_percent": 0.0, + "db_bytes": 4169728, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23846912, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23846912, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 881712, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 27, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u49-c1-repeat-1.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u49-c1-repeat-1.json new file mode 100644 index 00000000..7fce70f0 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u49-c1-repeat-1.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "84c8298f11a148d06e5ee1a4f514508c4970f2c7692dd1b3faeb4fe6c71a194b", + "archive_size_bytes": 4423127, + "elapsed_ms": 465, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 49, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23695360, + "baseline_rss_bytes": 23695360, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 47042560, + "peak_rss_bytes": 47042560 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 22863, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 34.07, + "cpu_ticks": 779, + "cpu_time_ms": 7790 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 0.48, + "avg300": 7.92, + "avg60": 4.35 + }, + "some": { + "avg10": 0.48, + "avg300": 8.02, + "avg60": 4.4 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.01, + "avg300": 8.51, + "avg60": 6.03 + }, + "some": { + "avg10": 0.01, + "avg300": 8.6, + "avg60": 6.11 + } + }, + "root_device_read_delta_bytes": 2555904, + "root_device_write_delta_bytes": 379916288 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 43.739, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 89, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 73, + "p99_ttft_ms": 73, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 56, + "minimum_elapsed_ms": 56, + "model": "q008-responses", + "p50_elapsed_ms": 56, + "p50_ttft_ms": 55, + "p95_elapsed_ms": 56, + "p95_ttft_ms": 55, + "sample_count": 1, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 58, + "minimum_elapsed_ms": 58, + "model": "q008-responses", + "p50_elapsed_ms": 58, + "p50_ttft_ms": 56, + "p95_elapsed_ms": 58, + "p95_ttft_ms": 56, + "sample_count": 1, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 55, + "minimum_elapsed_ms": 55, + "model": "q008-messages", + "p50_elapsed_ms": 55, + "p50_ttft_ms": 53, + "p95_elapsed_ms": 55, + "p95_ttft_ms": 53, + "sample_count": 1, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 6, + "warmup_count": 5 + } + }, + "sample_count": 1 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "83c0779ae4776a6b9d4d82571c7255bcb71246d6", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-m21gi7f7/config.toml Server: 127.0.0.1:50411 Accounts: 1 Database: Content digest: be9c7cef8febbd6a241f2580d722f8c45e6b312f410d6436bcdeb9a842a5d6c3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 84, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 38, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2545087, \"ppid\": 2545063, \"uptime_seconds\": 2.785245468, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 465, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-m21gi7f7/backups/eggpool-backup-20261016-052402.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2545087)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 130, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-m21gi7f7/recovery/config.toml db: /tmp/eggpool-q008-m21gi7f7/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 53, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2546508)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1026, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 8, + "/responses": 1015 + }, + "loopback_provider_requests": 1026, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.02, + "avg300": 8.68, + "avg60": 6.66 + }, + "some": { + "avg10": 0.02, + "avg300": 8.78, + "avg60": 6.75 + } + }, + "root_device_read_bytes": 761869938688, + "root_device_write_bytes": 1362628117504 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 11 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21803008, + 21803008, + 23695360, + 23695360, + 23715840, + 23752704, + 23773184, + 47042560, + 47042560, + 47042560, + 23887872 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 27, + 62 + ], + "second_workload_ttft_ms": [ + 26, + 61 + ], + "streaming_elapsed_ms": [ + 12, + 13, + 56 + ], + "streaming_ttft_ms": [ + 11, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21803008, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21803008, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21803008, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21803008, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23695360, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23695360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23695360, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23695360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 12, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23715840, + "pending_requests": 0, + "request_statuses": { + "completed": 12 + }, + "requests": 12, + "reservations": 12, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23715840, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 3201272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 18, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23752704, + "pending_requests": 0, + "request_statuses": { + "completed": 18 + }, + "requests": 18, + "reservations": 18, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23752704, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 24, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23773184, + "pending_requests": 0, + "request_statuses": { + "completed": 24 + }, + "requests": 24, + "reservations": 24, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23773184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4395008, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 47042560, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47042560, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173969408, + "wal_bytes": 4342512, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4395008, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 47042560, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47042560, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173969408, + "wal_bytes": 4342512, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4395008, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 47042560, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47042560, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173969408, + "wal_bytes": 4342512, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1026, + "cpu_percent": 0.0, + "db_bytes": 4272128, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23887872, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23887872, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 881712, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 20, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u49-c1-repeat-2.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u49-c1-repeat-2.json new file mode 100644 index 00000000..ff4ae5a9 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u49-c1-repeat-2.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "69a244fc04a67adc6441d37faf55d8a788fec11be510c1515b5d65f9559c38b7", + "archive_size_bytes": 4414935, + "elapsed_ms": 430, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 49, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23805952, + "baseline_rss_bytes": 23805952, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 47005696, + "peak_rss_bytes": 47005696 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 22990, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 34.62, + "cpu_ticks": 796, + "cpu_time_ms": 7960 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 0.47, + "avg300": 7.04, + "avg60": 2.57 + }, + "some": { + "avg10": 0.47, + "avg300": 7.12, + "avg60": 2.6 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.2, + "avg300": 7.56, + "avg60": 3.47 + }, + "some": { + "avg10": 0.2, + "avg300": 7.64, + "avg60": 3.51 + } + }, + "root_device_read_delta_bytes": 6332416, + "root_device_write_delta_bytes": 380522496 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 43.498, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 90, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 81, + "p99_ttft_ms": 80, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 56, + "minimum_elapsed_ms": 56, + "model": "q008-responses", + "p50_elapsed_ms": 56, + "p50_ttft_ms": 56, + "p95_elapsed_ms": 56, + "p95_ttft_ms": 56, + "sample_count": 1, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 59, + "model": "q008-responses", + "p50_elapsed_ms": 59, + "p50_ttft_ms": 57, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 1, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 58, + "minimum_elapsed_ms": 58, + "model": "q008-messages", + "p50_elapsed_ms": 58, + "p50_ttft_ms": 57, + "p95_elapsed_ms": 58, + "p95_ttft_ms": 57, + "sample_count": 1, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 6, + "warmup_count": 5 + } + }, + "sample_count": 1 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "83c0779ae4776a6b9d4d82571c7255bcb71246d6", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-olban3hp/config.toml Server: 127.0.0.1:51341 Accounts: 1 Database: Content digest: 161c0ecedda41d82f6404722dbc51aaadfd2b77fb42b05f1443eca5f0a01394f", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 85, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2546541, \"ppid\": 2546517, \"uptime_seconds\": 2.787386549, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 430, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-olban3hp/backups/eggpool-backup-20261016-052438.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2546541)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 132, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-olban3hp/recovery/config.toml db: /tmp/eggpool-q008-olban3hp/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 106, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 56, + "id": "stop", + "reason": "Stopping server (PID 2547882)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1026, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 8, + "/responses": 1015 + }, + "loopback_provider_requests": 1026, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.36, + "avg300": 7.71, + "avg60": 3.84 + }, + "some": { + "avg10": 0.36, + "avg300": 7.8, + "avg60": 3.88 + } + }, + "root_device_read_bytes": 761882615808, + "root_device_write_bytes": 1363066188800 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 11 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 55, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21970944, + 21970944, + 23805952, + 23805952, + 23834624, + 23855104, + 23875584, + 47005696, + 47005696, + 47005696, + 23662592 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 27, + 61 + ], + "second_workload_ttft_ms": [ + 27, + 61 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 55 + ], + "streaming_ttft_ms": [ + 11, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21970944, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21970944, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21970944, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21970944, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23805952, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23805952, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23805952, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23805952, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 12, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23834624, + "pending_requests": 0, + "request_statuses": { + "completed": 12 + }, + "requests": 12, + "reservations": 12, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23834624, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 3201272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 18, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23855104, + "pending_requests": 0, + "request_statuses": { + "completed": 18 + }, + "requests": 18, + "reservations": 18, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23855104, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 24, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23875584, + "pending_requests": 0, + "request_statuses": { + "completed": 24 + }, + "requests": 24, + "reservations": 24, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23875584, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4390912, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 47005696, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47005696, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173977600, + "wal_bytes": 4330152, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4390912, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 47005696, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47005696, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173977600, + "wal_bytes": 4330152, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4390912, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 47005696, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47005696, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173977600, + "wal_bytes": 4330152, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1026, + "cpu_percent": 0.0, + "db_bytes": 4272128, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23662592, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23662592, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 881712, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 21, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u49-c1-repeat-3.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u49-c1-repeat-3.json new file mode 100644 index 00000000..ee4f75f7 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u49-c1-repeat-3.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "f0c8c614677e553b8e735e510d6593837cd63e4e17e9cc79fbe6404d9ab77618", + "archive_size_bytes": 4423127, + "elapsed_ms": 419, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 49, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23707648, + "baseline_rss_bytes": 23707648, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 47054848, + "peak_rss_bytes": 47054848 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 26341, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 29.12, + "cpu_ticks": 767, + "cpu_time_ms": 7670 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 3.1, + "avg300": 6.42, + "avg60": 2.6 + }, + "some": { + "avg10": 3.15, + "avg300": 6.49, + "avg60": 2.64 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.11, + "avg300": 6.71, + "avg60": 2.03 + }, + "some": { + "avg10": 0.11, + "avg300": 6.79, + "avg60": 2.06 + } + }, + "root_device_read_delta_bytes": 2822144, + "root_device_write_delta_bytes": 380104704 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 37.963, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 240, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 79, + "p95_ttft_ms": 79, + "p99_elapsed_ms": 149, + "p99_ttft_ms": 149, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 60, + "model": "q008-responses", + "p50_elapsed_ms": 60, + "p50_ttft_ms": 60, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 60, + "sample_count": 1, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 56, + "minimum_elapsed_ms": 56, + "model": "q008-responses", + "p50_elapsed_ms": 56, + "p50_ttft_ms": 54, + "p95_elapsed_ms": 56, + "p95_ttft_ms": 54, + "sample_count": 1, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 59, + "model": "q008-messages", + "p50_elapsed_ms": 59, + "p50_ttft_ms": 57, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 1, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 6, + "warmup_count": 5 + } + }, + "sample_count": 1 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "83c0779ae4776a6b9d4d82571c7255bcb71246d6", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-vrxmbvlv/config.toml Server: 127.0.0.1:53797 Accounts: 1 Database: Content digest: a61525855f4d5bcef7fa0bb713c8513adbc525e593342c7e7fe90cfe6dc9ca2c", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 86, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 11, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 23, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 23, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 16, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2547918, \"ppid\": 2547894, \"uptime_seconds\": 2.857928997, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 419, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-vrxmbvlv/backups/eggpool-backup-20261016-052519.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2547918)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 137, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-vrxmbvlv/recovery/config.toml db: /tmp/eggpool-q008-vrxmbvlv/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 47, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop", + "reason": "Stopping server (PID 2549317)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1026, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 8, + "/responses": 1015 + }, + "loopback_provider_requests": 1026, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.21, + "avg300": 6.85, + "avg60": 2.25 + }, + "some": { + "avg10": 0.21, + "avg300": 6.93, + "avg60": 2.27 + } + }, + "root_device_read_bytes": 761890484224, + "root_device_write_bytes": 1363505873920 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 15, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 20, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 480, + "elapsed_ms": 60, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21889024, + 21889024, + 23707648, + 23707648, + 23748608, + 23777280, + 23789568, + 47054848, + 47054848, + 47054848, + 24104960 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 21, + 56 + ], + "second_workload_ttft_ms": [ + 21, + 56 + ], + "streaming_elapsed_ms": [ + 15, + 20, + 60 + ], + "streaming_ttft_ms": [ + 12, + 17, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21889024, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21889024, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21889024, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21889024, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23707648, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23707648, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23707648, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23707648, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 12, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23748608, + "pending_requests": 0, + "request_statuses": { + "completed": 12 + }, + "requests": 12, + "reservations": 12, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23748608, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 3201272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 18, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23777280, + "pending_requests": 0, + "request_statuses": { + "completed": 18 + }, + "requests": 18, + "reservations": 18, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23777280, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 24, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23789568, + "pending_requests": 0, + "request_statuses": { + "completed": 24 + }, + "requests": 24, + "reservations": 24, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23789568, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4399104, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 47054848, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47054848, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174104576, + "wal_bytes": 4334272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4399104, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 47054848, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47054848, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174104576, + "wal_bytes": 4334272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4399104, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 47054848, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47054848, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174104576, + "wal_bytes": 4334272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1026, + "cpu_percent": 0.0, + "db_bytes": 4272128, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 24104960, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24104960, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 922912, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 20, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u5-c1-repeat-1.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u5-c1-repeat-1.json new file mode 100644 index 00000000..f7ba85f8 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u5-c1-repeat-1.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "98c3ba0b698787978fefd63d8ce9a28dd8f39a53d1e76b10ce55b6b7946d8ab4", + "archive_size_bytes": 4337111, + "elapsed_ms": 610, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23388160, + "baseline_rss_bytes": 23388160, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 45862912, + "peak_rss_bytes": 45862912 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 25805, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 30.96, + "cpu_ticks": 799, + "cpu_time_ms": 7990 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 4.32, + "avg300": 4.68, + "avg60": 2.49 + }, + "some": { + "avg10": 4.32, + "avg300": 4.73, + "avg60": 2.51 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.11, + "avg300": 4.83, + "avg60": 2.01 + }, + "some": { + "avg10": 0.11, + "avg300": 4.88, + "avg60": 2.05 + } + }, + "root_device_read_delta_bytes": 634880, + "root_device_write_delta_bytes": 382083072 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 38.753, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 172, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 65, + "p95_ttft_ms": 65, + "p99_elapsed_ms": 123, + "p99_ttft_ms": 123, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 60, + "model": "q008-responses", + "p50_elapsed_ms": 60, + "p50_ttft_ms": 60, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 60, + "sample_count": 1, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 58, + "minimum_elapsed_ms": 58, + "model": "q008-responses", + "p50_elapsed_ms": 58, + "p50_ttft_ms": 56, + "p95_elapsed_ms": 58, + "p95_ttft_ms": 56, + "sample_count": 1, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 55, + "minimum_elapsed_ms": 55, + "model": "q008-messages", + "p50_elapsed_ms": 55, + "p50_ttft_ms": 53, + "p95_elapsed_ms": 55, + "p95_ttft_ms": 53, + "sample_count": 1, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 6, + "warmup_count": 5 + } + }, + "sample_count": 1 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "83c0779ae4776a6b9d4d82571c7255bcb71246d6", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-adnkife9/config.toml Server: 127.0.0.1:33809 Accounts: 1 Database: Content digest: 5af138312c4dabe2bbaf508263a27bf36fe346bed387da16ded3002c57797747", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 105, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 4, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 7, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 6, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2553087, \"ppid\": 2553063, \"uptime_seconds\": 2.789388881, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 610, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-adnkife9/backups/eggpool-backup-20261016-052824.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2553087)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 584, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-adnkife9/recovery/config.toml db: /tmp/eggpool-q008-adnkife9/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 248, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 105, + "id": "stop", + "reason": "Stopping server (PID 2554266)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1026, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 8, + "/responses": 1015 + }, + "loopback_provider_requests": 1026, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.17, + "avg300": 4.89, + "avg60": 2.15 + }, + "some": { + "avg10": 0.17, + "avg300": 4.95, + "avg60": 2.19 + } + }, + "root_device_read_bytes": 761942732800, + "root_device_write_bytes": 1365243794432 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 3, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 3 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 16, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 13 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 53, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 52 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21430272, + 21434368, + 23388160, + 23388160, + 23420928, + 23457792, + 23465984, + 45862912, + 45862912, + 45862912, + 23904256 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 3, + 2, + 2 + ], + "finite_ttft_ms": [ + 3, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 20, + 57 + ], + "second_workload_ttft_ms": [ + 20, + 56 + ], + "streaming_elapsed_ms": [ + 16, + 13, + 53 + ], + "streaming_ttft_ms": [ + 13, + 12, + 52 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21430272, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21430272, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21434368, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21434368, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23388160, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23388160, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23388160, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23388160, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 12, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23420928, + "pending_requests": 0, + "request_statuses": { + "completed": 12 + }, + "requests": 12, + "reservations": 12, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23420928, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 3201272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 18, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23457792, + "pending_requests": 0, + "request_statuses": { + "completed": 18 + }, + "requests": 18, + "reservations": 18, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23457792, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 24, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23465984, + "pending_requests": 0, + "request_statuses": { + "completed": 24 + }, + "requests": 24, + "reservations": 24, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23465984, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4313088, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 45862912, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 45862912, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173219840, + "wal_bytes": 4449632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4313088, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 45862912, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 45862912, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173219840, + "wal_bytes": 4449632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4313088, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 45862912, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 45862912, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173219840, + "wal_bytes": 4449632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1026, + "cpu_percent": 0.0, + "db_bytes": 4169728, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23904256, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23904256, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 914672, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 21, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u5-c1-repeat-2.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u5-c1-repeat-2.json new file mode 100644 index 00000000..14a6eb16 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u5-c1-repeat-2.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "80fd13186ae900eeebeba58d67ff37070c183cbf360fc665067d0f818a187067", + "archive_size_bytes": 4328919, + "elapsed_ms": 421, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23920640, + "baseline_rss_bytes": 23920640, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46313472, + "peak_rss_bytes": 46313472 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 23160, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 34.84, + "cpu_ticks": 807, + "cpu_time_ms": 8070 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 0.88, + "avg300": 4.3, + "avg60": 1.97 + }, + "some": { + "avg10": 0.88, + "avg300": 4.35, + "avg60": 1.98 + } + }, + "io_pressure_before": { + "full": { + "avg10": 2.1, + "avg300": 4.53, + "avg60": 2.29 + }, + "some": { + "avg10": 2.1, + "avg300": 4.58, + "avg60": 2.31 + } + }, + "root_device_read_delta_bytes": 277479424, + "root_device_write_delta_bytes": 381886464 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 43.178, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 90, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 66, + "p99_ttft_ms": 65, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 60, + "model": "q008-responses", + "p50_elapsed_ms": 60, + "p50_ttft_ms": 60, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 60, + "sample_count": 1, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 59, + "model": "q008-responses", + "p50_elapsed_ms": 59, + "p50_ttft_ms": 57, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 1, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 56, + "minimum_elapsed_ms": 56, + "model": "q008-messages", + "p50_elapsed_ms": 56, + "p50_ttft_ms": 53, + "p95_elapsed_ms": 56, + "p95_ttft_ms": 53, + "sample_count": 1, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 6, + "warmup_count": 5 + } + }, + "sample_count": 1 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "83c0779ae4776a6b9d4d82571c7255bcb71246d6", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-90bpe614/config.toml Server: 127.0.0.1:39169 Accounts: 1 Database: Content digest: f0e21634ab8a51412c20e500930ea3985fc16fa60bf6e29c6efb90550aad9645", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 134, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 52, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 2, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 5, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 5, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 1, \"native_count\": 1, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2554301, \"ppid\": 2554277, \"uptime_seconds\": 2.78836395, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, ", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 6, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 421, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-90bpe614/backups/eggpool-backup-20261016-052900.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2554301)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 95, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-90bpe614/recovery/config.toml db: /tmp/eggpool-q008-90bpe614/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 54, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 64, + "id": "stop", + "reason": "Stopping server (PID 2555448)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1026, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 8, + "/responses": 1015 + }, + "loopback_provider_requests": 1026, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 3.6, + "avg300": 4.62, + "avg60": 2.5 + }, + "some": { + "avg10": 3.6, + "avg300": 4.67, + "avg60": 2.52 + } + }, + "root_device_read_bytes": 761944260608, + "root_device_write_bytes": 1365677995008 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 53, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 52 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21889024, + 21889024, + 23920640, + 23920640, + 23941120, + 23973888, + 23990272, + 46313472, + 46313472, + 46313472, + 23990272 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 21, + 56 + ], + "second_workload_ttft_ms": [ + 21, + 55 + ], + "streaming_elapsed_ms": [ + 12, + 13, + 53 + ], + "streaming_ttft_ms": [ + 12, + 12, + 52 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21889024, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21889024, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21889024, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21889024, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23920640, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23920640, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23920640, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23920640, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 12, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23941120, + "pending_requests": 0, + "request_statuses": { + "completed": 12 + }, + "requests": 12, + "reservations": 12, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23941120, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 3201272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 18, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23973888, + "pending_requests": 0, + "request_statuses": { + "completed": 18 + }, + "requests": 18, + "reservations": 18, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23973888, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 24, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23990272, + "pending_requests": 0, + "request_statuses": { + "completed": 24 + }, + "requests": 24, + "reservations": 24, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23990272, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4308992, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46313472, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46313472, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173101056, + "wal_bytes": 4429032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4308992, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46313472, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46313472, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173101056, + "wal_bytes": 4429032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4308992, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46313472, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46313472, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173101056, + "wal_bytes": 4429032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1026, + "cpu_percent": 0.0, + "db_bytes": 4169728, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23990272, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23990272, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 894072, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 23, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u5-c1-repeat-3.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u5-c1-repeat-3.json new file mode 100644 index 00000000..499a49be --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/controls/u5-c1-repeat-3.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "56c7ea1e88cdd09e4859931c19547fd86e577a53d79e4069a66687e95ae8ee30", + "archive_size_bytes": 4328919, + "elapsed_ms": 419, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23748608, + "baseline_rss_bytes": 23748608, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46321664, + "peak_rss_bytes": 46321664 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 26738, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 30.44, + "cpu_ticks": 814, + "cpu_time_ms": 8140 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 3.48, + "avg300": 4.01, + "avg60": 2.21 + }, + "some": { + "avg10": 3.48, + "avg300": 4.05, + "avg60": 2.22 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.17, + "avg300": 4.07, + "avg60": 1.51 + }, + "some": { + "avg10": 0.17, + "avg300": 4.11, + "avg60": 1.52 + } + }, + "root_device_read_delta_bytes": 626688, + "root_device_write_delta_bytes": 380981248 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 37.399, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 255, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 62, + "p95_ttft_ms": 62, + "p99_elapsed_ms": 145, + "p99_ttft_ms": 145, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 58, + "minimum_elapsed_ms": 58, + "model": "q008-responses", + "p50_elapsed_ms": 58, + "p50_ttft_ms": 58, + "p95_elapsed_ms": 58, + "p95_ttft_ms": 58, + "sample_count": 1, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 60, + "model": "q008-responses", + "p50_elapsed_ms": 60, + "p50_ttft_ms": 57, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 1, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 60, + "model": "q008-messages", + "p50_elapsed_ms": 60, + "p50_ttft_ms": 57, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 1, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 6, + "warmup_count": 5 + } + }, + "sample_count": 1 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "83c0779ae4776a6b9d4d82571c7255bcb71246d6", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-2o6gh19n/config.toml Server: 127.0.0.1:47979 Accounts: 1 Database: Content digest: 476b671e97132678fc6e077eee3a4b4a67fb1f65239dfd0d63707a8e64815511", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 90, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 38, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 6, \"native_count\": 6, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2555489, \"ppid\": 2555465, \"uptime_seconds\": 2.792910031, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 4, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 419, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-2o6gh19n/backups/eggpool-backup-20261016-052942.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2555489)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 135, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-2o6gh19n/recovery/config.toml db: /tmp/eggpool-q008-2o6gh19n/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 50, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2556660)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1026, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 8, + "/responses": 1015 + }, + "loopback_provider_requests": 1026, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.32, + "avg300": 4.15, + "avg60": 1.67 + }, + "some": { + "avg10": 0.32, + "avg300": 4.2, + "avg60": 1.68 + } + }, + "root_device_read_bytes": 762222260224, + "root_device_write_bytes": 1366110770176 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 3, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 3 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21856256, + 21856256, + 23748608, + 23748608, + 23777280, + 23810048, + 23826432, + 46321664, + 46321664, + 46321664, + 23998464 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 3, + 2, + 2 + ], + "finite_ttft_ms": [ + 3, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 27, + 62 + ], + "second_workload_ttft_ms": [ + 27, + 61 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 56 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21856256, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21856256, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21856256, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21856256, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23748608, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23748608, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23748608, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23748608, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 12, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23777280, + "pending_requests": 0, + "request_statuses": { + "completed": 12 + }, + "requests": 12, + "reservations": 12, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23777280, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 3201272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 18, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23810048, + "pending_requests": 0, + "request_statuses": { + "completed": 18 + }, + "requests": 18, + "reservations": 18, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23810048, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 24, + "cpu_percent": 0.0, + "db_bytes": 856064, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23826432, + "pending_requests": 0, + "request_statuses": { + "completed": 24 + }, + "requests": 24, + "reservations": 24, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23826432, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4169472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4308992, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46321664, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46321664, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173252608, + "wal_bytes": 4412552, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4308992, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46321664, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46321664, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173252608, + "wal_bytes": 4412552, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1024, + "cpu_percent": 0.0, + "db_bytes": 4308992, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46321664, + "pending_requests": 0, + "request_statuses": { + "completed": 1024 + }, + "requests": 1024, + "reservations": 1024, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46321664, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173252608, + "wal_bytes": 4412552, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1026, + "cpu_percent": 0.0, + "db_bytes": 4169728, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23998464, + "pending_requests": 0, + "request_statuses": { + "completed": 1026 + }, + "requests": 1026, + "reservations": 1026, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23998464, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 902312, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 20, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/lifecycle.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/lifecycle.json new file mode 100644 index 00000000..842f9bca --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/lifecycle.json @@ -0,0 +1,502 @@ +{ + "candidate": { + "binary_size_bytes": 39042128, + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "check-config" + ], + "duration_ms": 3, + "reason": "Configuration loaded successfully from /tmp/eggpool-q009-3yi5_ah9/config.toml Server: 127.0.0.1:45817 Accounts: 3 Database: /tmp/eggpool-q009-3yi5_ah9/data-home/usage.sqlite3 Content digest: 1a1ef765acc71809d2fc6fa0ff01d23567a83140a4828ab207b0c5455e3a4377", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "migrate" + ], + "duration_ms": 144, + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "bootstrap" + ], + "duration_ms": 283, + "reason": "synthetic managed principal created", + "status": "pass" + }, + { + "command": [ + "users", + "policy", + "create", + "q009-disable-policy", + "Q009 Disable Qualification", + "allow", + "--actor", + "q009-admin" + ], + "duration_ms": 33, + "reason": "Model policy profile q009-disable-policy created.", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "add" + ], + "duration_ms": 376, + "reason": "synthetic disable principal setup", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "token" + ], + "duration_ms": 441, + "reason": "synthetic disable credential issued", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "budget", + "set", + "q009-admin", + "--window", + "day", + "--max-requests", + "100", + "--enforcement", + "hard", + "--actor", + "q009-admin" + ], + "duration_ms": 28, + "reason": "Budget rule for q009-admin saved (hard).", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "budget", + "set", + "q009-disablee", + "--window", + "day", + "--max-requests", + "100", + "--enforcement", + "hard", + "--actor", + "q009-admin" + ], + "duration_ms": 30, + "reason": "Budget rule for q009-disablee saved (hard).", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "global-budget", + "set", + "--window", + "day", + "--max-requests", + "100", + "--enforcement", + "hard", + "--actor", + "q009-admin" + ], + "duration_ms": 38, + "reason": "Instance budget ceiling saved (hard).", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "budget", + "set", + "q009-admin", + "--window", + "day", + "--max-requests", + "1", + "--enforcement", + "hard", + "--actor", + "q009-admin" + ], + "duration_ms": 5, + "reason": "Budget rule for q009-admin saved (hard).", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "budget", + "set", + "q009-admin", + "--window", + "day", + "--max-requests", + "100", + "--enforcement", + "hard", + "--actor", + "q009-admin" + ], + "duration_ms": 5, + "reason": "Budget rule for q009-admin saved (hard).", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "global-budget", + "set", + "--window", + "day", + "--max-requests", + "1", + "--enforcement", + "hard", + "--actor", + "q009-admin" + ], + "duration_ms": 6, + "reason": "Instance budget ceiling saved (hard).", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "global-budget", + "set", + "--window", + "day", + "--max-requests", + "100", + "--enforcement", + "hard", + "--actor", + "q009-admin" + ], + "duration_ms": 6, + "reason": "Instance budget ceiling saved (hard).", + "returncode": 0, + "status": "pass" + }, + { + "command": [ + "users", + "disable" + ], + "duration_ms": 6, + "reason": "User q009-disablee disabled; browser sessions and personal tokens were revoked.", + "returncode": 0, + "status": "pass" + } + ], + "configuration": { + "fault_cycles": 1, + "reload_cycles": 2, + "restart_cycles": 1, + "steady_cycles": 2, + "timeout_s": 20.0, + "warmup_cycles": 1 + }, + "durable": { + "active_reservations": 0, + "attempts": 14, + "pending_requests": 0, + "request_statuses": { + "completed": 4, + "error": 6, + "interrupted": 1 + }, + "requests": 11, + "reservations": 14 + }, + "duration_ms": 2177, + "environment": { + "architecture": "x86_64", + "network_policy": "loopback-only", + "platform": "Linux", + "system": "posix" + }, + "findings": [], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "ownership_convergence": { + "active_leases": null, + "active_reservations": 0, + "finalization_jobs": null, + "logical_leaks": false, + "pending_requests": 0, + "retiring_generations": null, + "terminal_references": 0 + }, + "phases": [ + { + "cycles": 1, + "observations": [ + { + "category": "provider", + "client_http_status": 200, + "expected_upstream_status": 408, + "model": "q009-fault-408", + "provider_requests": 2, + "response_bytes": 242 + }, + { + "category": "provider", + "client_http_status": 429, + "expected_upstream_status": 429, + "model": "q009-fault-429", + "post_fault_server_restart": "pass", + "provider_requests": 1, + "response_bytes": 37 + }, + { + "category": "provider", + "client_http_status": 200, + "expected_upstream_status": 503, + "model": "q009-fault-5xx", + "provider_requests": 2, + "response_bytes": 242 + }, + { + "category": "provider", + "client_http_status": 200, + "expected_upstream_status": "timeout", + "model": "q009-timeout", + "provider_requests": 1, + "response_bytes": 242 + }, + { + "http_status": 200, + "model": "q009-partial", + "terminal_evidence": false + }, + { + "http_status": 200, + "model": "q009-disconnect", + "terminal_evidence": false + }, + { + "model": "q009-cancel", + "status": "client-cancelled-and-write-aborted" + }, + { + "http_status": 200, + "model": "q009-wire", + "status": "alternate-wire-success" + }, + { + "http_status": 200, + "model": "q009-malformed", + "status": "malformed-incomplete" + }, + { + "http_status": 404, + "model": "q009-absent", + "status": "model-absent" + }, + { + "http_status": 503, + "model": "q009-connect/unreachable", + "status": "connect-failure" + }, + { + "category": "budget", + "denial_kind": "user_budget", + "http_status": 429, + "provider_requests": 0 + }, + { + "category": "budget", + "denial_kind": "global_budget", + "http_status": 429, + "provider_requests": 0 + } + ], + "phase": "faults", + "status": "pass" + }, + { + "client_outcome": "cancelled-or-closed", + "credential_revocation_during_load": true, + "cycles": 1, + "hard_budgets_active": true, + "identity_denial_http_status": 401, + "phase": "restart", + "status": "pass", + "unknown_inflight_replayed": false, + "user_disabled_during_load": true + } + ], + "plan": "stability qualification", + "provider": { + "request_count": 13, + "requests_by_model": { + "q009-abrupt": 1, + "q009-absent": 1, + "q009-disconnect": 1, + "q009-fault-408": 2, + "q009-fault-429": 1, + "q009-fault-5xx": 2, + "q009-malformed": 1, + "q009-partial": 1, + "q009-timeout": 1, + "q009-wire": 2 + } + }, + "resource_analysis": { + "fd_counts": [ + 14, + 14, + 14 + ], + "interpretation": "bounded characterization; warmup/high-water allocator RSS is not treated as a leak and no SLA is inferred", + "rss_bytes": [ + 21884928, + 23457792, + 21725184 + ], + "sample_count": 3, + "thread_counts": [ + 2, + 2, + 2 + ] + }, + "resource_evidence": { + "runtime_read_errors": 0, + "semantics_affected": false, + "status": "pass" + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21884928, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 50, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103780352, + "wal_bytes": 111272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 13, + "cpu_percent": 0.0, + "db_bytes": 847872, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-faults", + "open_fd_count": 14, + "pending_requests": 0, + "request_statuses": { + "completed": 4, + "error": 6 + }, + "requests": 10, + "reservations": 13, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23457792, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 50, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103899136, + "wal_bytes": 2776912, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 14, + "cpu_percent": 0.0, + "db_bytes": 872448, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-restart", + "open_fd_count": 14, + "pending_requests": 0, + "request_statuses": { + "completed": 4, + "error": 6, + "interrupted": 1 + }, + "requests": 11, + "reservations": 14, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21725184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 50, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103780352, + "wal_bytes": 3114752, + "wire_flights": null + } + ], + "schema_version": "runtime-q009.v1", + "seed": 9009, + "status": "pass", + "temporary_leftovers": 0, + "workload_policy": { + "budget_mode": "hard user and global request ceilings", + "initial_global_request_limit": 100, + "initial_user_request_limit": 100, + "principal_count": 2 + } +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/matrix.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/matrix.json new file mode 100644 index 00000000..4bf2994d --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/matrix.json @@ -0,0 +1,174 @@ +{ + "schema_version": "on-prem-ordinary-server-matrix.v1", + "plan": "on-prem-concurrency-performance M002", + "status": "complete", + "preflight_io": { + "root_device_read_bytes": 759900020736, + "root_device_write_bytes": 1359179977728, + "io_pressure": { + "some": { + "avg10": 4.26, + "avg60": 5.28, + "avg300": 3.43 + }, + "full": { + "avg10": 4.25, + "avg60": 5.24, + "avg300": 3.39 + } + } + }, + "stop_reason": null, + "candidate": { + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "build_profile": "release", + "build_mode": "locked release candidate" + }, + "qualification_tooling_commit_sha": "83c0779ae4776a6b9d4d82571c7255bcb71246d6", + "target": { + "target_class": "ordinary-server", + "architecture": "x86_64", + "kernel": "6.8.0-147-generic", + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "logical_cpu_count": 16, + "ram_bytes": 16622407680, + "system_vendor": "Gigabyte Technology Co., Ltd.", + "product_name": "Z390 AORUS MASTER", + "board_name": "Z390 AORUS MASTER-CF", + "root_filesystem": "ext4", + "root_storage_medium_class": "non-rotational", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_mount_options": "rw,relatime,stripe=32", + "virtualization_type": "none", + "container_type": "none", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt" + }, + "workload": { + "fresh_root_per_run": true, + "named_principal_counts": [ + 1, + 5, + 10, + 25, + 49 + ], + "principal_separation_condition": "concurrency=1", + "concurrent_request_counts": [ + 1, + 2, + 4, + 8, + 16 + ], + "concurrency_separation_condition": "principals=5", + "requests_per_condition": 1000, + "repetitions_per_condition": 3, + "sequential_wire_samples_per_surface": 30, + "provider_latency_profile": "mixed-0-10-50ms", + "budget_mode": "user-global hard request limits", + "surfaces": [ + "native Responses finite", + "native Responses streaming", + "translated Responses-to-Messages streaming" + ] + }, + "gates": { + "all_requests_complete": true, + "zero_http_failures": true, + "zero_pending_requests_after_stabilization": true, + "zero_active_reservations_after_stabilization": true, + "p99_minimum_completed_observations": 1000, + "three_repetitions_required": true, + "maximum_initial_io_pressure_full_avg10_percent": 25.0, + "no_capacity_sla_claim": true + }, + "conditions": [ + { + "principals": 1, + "concurrency": 1, + "run_artifacts": [ + "u1-c1-run-1.json", + "u1-c1-run-2.json", + "u1-c1-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 1, + "run_artifacts": [ + "u5-c1-run-1.json", + "u5-c1-run-2.json", + "u5-c1-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 2, + "run_artifacts": [ + "u5-c2-run-1.json", + "u5-c2-run-2.json", + "u5-c2-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 4, + "run_artifacts": [ + "u5-c4-run-1.json", + "u5-c4-run-2.json", + "u5-c4-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 8, + "run_artifacts": [ + "u5-c8-run-1.json", + "u5-c8-run-2.json", + "u5-c8-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 16, + "run_artifacts": [ + "u5-c16-run-1.json", + "u5-c16-run-2.json", + "u5-c16-run-3.json" + ] + }, + { + "principals": 10, + "concurrency": 1, + "run_artifacts": [ + "u10-c1-run-1.json", + "u10-c1-run-2.json", + "u10-c1-run-3.json" + ] + }, + { + "principals": 25, + "concurrency": 1, + "run_artifacts": [ + "u25-c1-run-1.json", + "u25-c1-run-2.json", + "u25-c1-run-3.json" + ] + }, + { + "principals": 49, + "concurrency": 1, + "run_artifacts": [ + "u49-c1-run-1.json", + "u49-c1-run-2.json", + "u49-c1-run-3.json" + ] + } + ], + "artifacts_directory": "m023-ordinary-server-no-limit", + "lifecycle_artifact": "lifecycle.json" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/summary.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/summary.json new file mode 100644 index 00000000..542700db --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/summary.json @@ -0,0 +1,522 @@ +{ + "schema_version": "on-prem-ordinary-server-summary.v1", + "plan": "on-prem-concurrency-performance M002", + "candidate": { + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "build_profile": "release", + "build_mode": "locked release candidate" + }, + "target": { + "target_class": "ordinary-server", + "architecture": "x86_64", + "kernel": "6.8.0-147-generic", + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "logical_cpu_count": 16, + "ram_bytes": 16622407680, + "system_vendor": "Gigabyte Technology Co., Ltd.", + "product_name": "Z390 AORUS MASTER", + "board_name": "Z390 AORUS MASTER-CF", + "root_filesystem": "ext4", + "root_storage_medium_class": "non-rotational", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_mount_options": "rw,relatime,stripe=32", + "virtualization_type": "none", + "container_type": "none", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt" + }, + "workload": { + "fresh_root_per_run": true, + "named_principal_counts": [ + 1, + 5, + 10, + 25, + 49 + ], + "principal_separation_condition": "concurrency=1", + "concurrent_request_counts": [ + 1, + 2, + 4, + 8, + 16 + ], + "concurrency_separation_condition": "principals=5", + "requests_per_condition": 1000, + "repetitions_per_condition": 3, + "sequential_wire_samples_per_surface": 30, + "provider_latency_profile": "mixed-0-10-50ms", + "budget_mode": "user-global hard request limits", + "surfaces": [ + "native Responses finite", + "native Responses streaming", + "translated Responses-to-Messages streaming" + ] + }, + "lifecycle": { + "artifact": "lifecycle.json", + "status": "pass", + "phases": [ + "faults", + "restart" + ], + "provider": { + "request_count": 13, + "requests_by_model": { + "q009-abrupt": 1, + "q009-absent": 1, + "q009-disconnect": 1, + "q009-fault-408": 2, + "q009-fault-429": 1, + "q009-fault-5xx": 2, + "q009-malformed": 1, + "q009-partial": 1, + "q009-timeout": 1, + "q009-wire": 2 + } + }, + "ownership_convergence": { + "active_leases": null, + "active_reservations": 0, + "finalization_jobs": null, + "logical_leaks": false, + "pending_requests": 0, + "retiring_generations": null, + "terminal_references": 0 + }, + "workload_policy": { + "budget_mode": "hard user and global request ceilings", + "initial_global_request_limit": 100, + "initial_user_request_limit": 100, + "principal_count": 2 + }, + "denial_counts": { + "provider_429": 1, + "user_budget_429": 1, + "global_budget_429": 1, + "identity_denial": 1 + }, + "identity_denial_http_status": 401 + }, + "interpretation": "Descriptive loopback capacity characterization for this host and synthetic workload; not an SLA or provider-throughput promise.", + "conditions": [ + { + "principals": 1, + "concurrency": 1, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 32.164, + "median_cpu_percent": 26.33, + "median_p50_ms": 15, + "median_p95_ms": 120, + "median_p99_ms": 180, + "maximum_ms": 628, + "maximum_peak_rss_bytes": 26112000, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 13, + "p95_elapsed_ms": 94, + "p50_ttft_ms": 13, + "p95_ttft_ms": 93 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 127, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 13, + "p95_elapsed_ms": 93, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 5.48, + "median_root_device_write_delta_bytes": 389615616, + "maximum_observed_wal_bytes": 4408432, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u1-c1-run-1.json", + "u1-c1-run-2.json", + "u1-c1-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 1, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 36.941, + "median_cpu_percent": 30.73, + "median_p50_ms": 15, + "median_p95_ms": 62, + "median_p99_ms": 151, + "maximum_ms": 254, + "maximum_peak_rss_bytes": 46530560, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 94, + "p50_ttft_ms": 12, + "p95_ttft_ms": 94 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 120, + "p50_ttft_ms": 12, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 122, + "p50_ttft_ms": 8, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 2.94, + "median_root_device_write_delta_bytes": 394305536, + "maximum_observed_wal_bytes": 4367232, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u5-c1-run-1.json", + "u5-c1-run-2.json", + "u5-c1-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 2, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 92.816, + "median_cpu_percent": 54.3, + "median_p50_ms": 15, + "median_p95_ms": 62, + "median_p99_ms": 82, + "maximum_ms": 285, + "maximum_peak_rss_bytes": 46489600, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 13, + "p95_elapsed_ms": 60, + "p50_ttft_ms": 12, + "p95_ttft_ms": 59 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 13, + "p95_elapsed_ms": 60, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 14, + "p95_elapsed_ms": 60, + "p50_ttft_ms": 8, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 1.9, + "median_root_device_write_delta_bytes": 390238208, + "maximum_observed_wal_bytes": 4375472, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u5-c2-run-1.json", + "u5-c2-run-2.json", + "u5-c2-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 4, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 190.485, + "median_cpu_percent": 62.75, + "median_p50_ms": 15, + "median_p95_ms": 60, + "median_p99_ms": 82, + "maximum_ms": 383, + "maximum_peak_rss_bytes": 46751744, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 14, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 14, + "p95_ttft_ms": 59 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 14, + "p95_elapsed_ms": 60, + "p50_ttft_ms": 8, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 2.34, + "median_root_device_write_delta_bytes": 389214208, + "maximum_observed_wal_bytes": 4412552, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u5-c4-run-1.json", + "u5-c4-run-2.json", + "u5-c4-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 8, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 295.215, + "median_cpu_percent": 66.02, + "median_p50_ms": 22, + "median_p95_ms": 73, + "median_p99_ms": 85, + "maximum_ms": 1021, + "maximum_peak_rss_bytes": 47181824, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 19, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 19, + "p95_ttft_ms": 59 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 14, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 12, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 19, + "p95_elapsed_ms": 60, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 2.04, + "median_root_device_write_delta_bytes": 389931008, + "maximum_observed_wal_bytes": 4433152, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u5-c8-run-1.json", + "u5-c8-run-2.json", + "u5-c8-run-3.json" + ] + }, + { + "principals": 5, + "concurrency": 16, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 326.388, + "median_cpu_percent": 75.72, + "median_p50_ms": 38, + "median_p95_ms": 91, + "median_p99_ms": 119, + "maximum_ms": 1119, + "maximum_peak_rss_bytes": 48058368, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 15, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 15, + "p95_ttft_ms": 59 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 14, + "p95_elapsed_ms": 60, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 60, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 2.79, + "median_root_device_write_delta_bytes": 392114176, + "maximum_observed_wal_bytes": 4470232, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u5-c16-run-1.json", + "u5-c16-run-2.json", + "u5-c16-run-3.json" + ] + }, + { + "principals": 10, + "concurrency": 1, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 41.592, + "median_cpu_percent": 32.11, + "median_p50_ms": 13, + "median_p95_ms": 61, + "median_p99_ms": 109, + "maximum_ms": 235, + "maximum_peak_rss_bytes": 46370816, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 19, + "p95_elapsed_ms": 59, + "p50_ttft_ms": 19, + "p95_ttft_ms": 59 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 14, + "p95_elapsed_ms": 60, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 16, + "p95_elapsed_ms": 60, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 2.21, + "median_root_device_write_delta_bytes": 389709824, + "maximum_observed_wal_bytes": 4457872, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u10-c1-run-1.json", + "u10-c1-run-2.json", + "u10-c1-run-3.json" + ] + }, + { + "principals": 25, + "concurrency": 1, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 43.399, + "median_cpu_percent": 34.55, + "median_p50_ms": 14, + "median_p95_ms": 61, + "median_p99_ms": 66, + "maximum_ms": 90, + "maximum_peak_rss_bytes": 46665728, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 13, + "p95_elapsed_ms": 60, + "p50_ttft_ms": 13, + "p95_ttft_ms": 59 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 13, + "p95_elapsed_ms": 60, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 13, + "p95_elapsed_ms": 60, + "p50_ttft_ms": 11, + "p95_ttft_ms": 57 + } + }, + "median_io_pressure_full_avg10_after_percent": 0.22, + "median_root_device_write_delta_bytes": 386707456, + "maximum_observed_wal_bytes": 4408432, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u25-c1-run-1.json", + "u25-c1-run-2.json", + "u25-c1-run-3.json" + ] + }, + { + "principals": 49, + "concurrency": 1, + "repetitions": 3, + "requests_per_run": 1000, + "median_requests_per_second": 40.76, + "median_cpu_percent": 8.19, + "median_p50_ms": 12, + "median_p95_ms": 126, + "median_p99_ms": 172, + "maximum_ms": 654, + "maximum_peak_rss_bytes": 47112192, + "sequential_surfaces": { + "native_responses_finite": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 52, + "p50_ttft_ms": 12, + "p95_ttft_ms": 52 + }, + "native_responses_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 6, + "p95_elapsed_ms": 52, + "p50_ttft_ms": 6, + "p95_ttft_ms": 52 + }, + "translated_responses_to_messages_streaming": { + "sample_count": 30, + "p50_elapsed_ms": 12, + "p95_elapsed_ms": 52, + "p50_ttft_ms": 6, + "p95_ttft_ms": 52 + } + }, + "median_io_pressure_full_avg10_after_percent": 6.6, + "median_root_device_write_delta_bytes": 605548544, + "maximum_observed_wal_bytes": 4420792, + "median_durable_attempts": 1113, + "settlement": "zero pending requests and active reservations in every run", + "raw_runs": [ + "u49-c1-run-1.json", + "u49-c1-run-2.json", + "u49-c1-run-3.json" + ] + } + ] +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u1-c1-run-1.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u1-c1-run-1.json new file mode 100644 index 00000000..1baa7bf0 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u1-c1-run-1.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "a7a8626c1837b5e7d69da2d33dccef257daea544f8ddfca6bde60beb3a861bea", + "archive_size_bytes": 4648407, + "elapsed_ms": 620, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 1, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23678976, + "baseline_rss_bytes": 23678976, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 25919488, + "peak_rss_bytes": 25919488 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 26509, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 31.2, + "cpu_ticks": 827, + "cpu_time_ms": 8270 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 5.48, + "avg300": 5.51, + "avg60": 2.17 + }, + "some": { + "avg10": 5.66, + "avg300": 5.58, + "avg60": 2.21 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.4, + "avg300": 5.72, + "avg60": 1.36 + }, + "some": { + "avg10": 0.4, + "avg300": 5.78, + "avg60": 1.38 + } + }, + "root_device_read_delta_bytes": 1761280, + "root_device_write_delta_bytes": 491532288 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 37.723, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 628, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 141, + "p99_ttft_ms": 141, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 81, + "minimum_elapsed_ms": 3, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 3, + "model": "q008-messages", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 10, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-k95b34mg/config.toml Server: 127.0.0.1:41725 Accounts: 1 Database: Content digest: 61d7eed33cebcdb249e9cc7322f4f381e63da6cfc15db7b001bb2ea680c9faa7", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 86, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 38, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 11, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 16, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2506194, \"ppid\": 2506172, \"uptime_seconds\": 2.815055918, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 620, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-k95b34mg/backups/eggpool-backup-20261016-050650.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 62, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2506194)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 638, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-k95b34mg/recovery/config.toml db: /tmp/eggpool-q008-k95b34mg/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 246, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 64, + "id": "stop", + "reason": "Stopping server (PID 2507461)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.29, + "avg300": 5.82, + "avg60": 1.44 + }, + "some": { + "avg10": 0.29, + "avg300": 5.89, + "avg60": 1.46 + } + }, + "root_device_read_bytes": 757962469376, + "root_device_write_bytes": 1344148677632 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 3, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21782528, + 21786624, + 23678976, + 23678976, + 23805952, + 23928832, + 24104960, + 25919488, + 25919488, + 25919488, + 24051712 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 3, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 3, + 2 + ], + "second_workload_ttft_ms": [ + 3, + 2 + ], + "streaming_elapsed_ms": [ + 12, + 13, + 56 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21782528, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21782528, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21786624, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21786624, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23678976, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23678976, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23678976, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23678976, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23805952, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23805952, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23928832, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23928832, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24104960, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24104960, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173117440, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 25919488, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 25919488, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173260800, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 25919488, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 25919488, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173260800, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 25919488, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 25919488, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173260800, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 24051712, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24051712, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 885832, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 21, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u1-c1-run-2.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u1-c1-run-2.json new file mode 100644 index 00000000..4d039495 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u1-c1-run-2.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "0b3d0ae1108e48ad2b2bd4deced489ec6f321b0d139f4f1876f5147f7592ad84", + "archive_size_bytes": 4648407, + "elapsed_ms": 628, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 1, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23855104, + "baseline_rss_bytes": 23855104, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 26112000, + "peak_rss_bytes": 26112000 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 31522, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 26.33, + "cpu_ticks": 830, + "cpu_time_ms": 8300 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 5.47, + "avg300": 5.42, + "avg60": 3.68 + }, + "some": { + "avg10": 5.49, + "avg300": 5.49, + "avg60": 3.72 + } + }, + "io_pressure_before": { + "full": { + "avg10": 3.2, + "avg300": 5.37, + "avg60": 2.25 + }, + "some": { + "avg10": 3.24, + "avg300": 5.43, + "avg60": 2.28 + } + }, + "root_device_read_delta_bytes": 1814528, + "root_device_write_delta_bytes": 387596288 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 31.724, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 304, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 16, + "p50_ttft_ms": 16, + "p95_elapsed_ms": 120, + "p95_ttft_ms": 120, + "p99_elapsed_ms": 180, + "p99_ttft_ms": 180, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 117, + "minimum_elapsed_ms": 3, + "model": "q008-responses", + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 94, + "p95_ttft_ms": 93, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 164, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 127, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 112, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 93, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-e3c82__w/config.toml Server: 127.0.0.1:45171 Accounts: 1 Database: Content digest: f64f03e0b96114eb38aade48b2b889cd8b94cd235d7ac578bc6177137815432a", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 142, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 104, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2507536, \"ppid\": 2507502, \"uptime_seconds\": 2.789334368, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 5, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 628, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-e3c82__w/backups/eggpool-backup-20261016-050734.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2507536)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 706, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-e3c82__w/recovery/config.toml db: /tmp/eggpool-q008-e3c82__w/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 270, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 60, + "id": "stop", + "reason": "Stopping server (PID 2508770)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 4.04, + "avg300": 5.44, + "avg60": 2.19 + }, + "some": { + "avg10": 4.14, + "avg300": 5.51, + "avg60": 2.22 + } + }, + "root_device_read_bytes": 757966479360, + "root_device_write_bytes": 1344720614400 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 3, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 3 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21766144, + 21766144, + 23855104, + 23855104, + 23982080, + 24137728, + 24281088, + 26112000, + 26112000, + 26112000, + 23715840 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 3, + 2, + 2 + ], + "finite_ttft_ms": [ + 3, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 3, + 2 + ], + "second_workload_ttft_ms": [ + 3, + 2 + ], + "streaming_elapsed_ms": [ + 12, + 13, + 56 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21766144, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21766144, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21766144, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21766144, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23855104, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23855104, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23855104, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23855104, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23982080, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23982080, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24137728, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24137728, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24281088, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24281088, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 26112000, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 26112000, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173228032, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 26112000, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 26112000, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173228032, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 26112000, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 26112000, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173228032, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23715840, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23715840, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 885832, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 20, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u1-c1-run-3.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u1-c1-run-3.json new file mode 100644 index 00000000..8a714ae6 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u1-c1-run-3.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "33309ff5d132a1bf68201c7885a01fa84f60b95e8e8c2ff5c6a06617bba13ef0", + "archive_size_bytes": 4648407, + "elapsed_ms": 813, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 1, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23601152, + "baseline_rss_bytes": 23601152, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 25853952, + "peak_rss_bytes": 25853952 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 31091, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 24.86, + "cpu_ticks": 773, + "cpu_time_ms": 7730 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 7.13, + "avg300": 5.53, + "avg60": 5.08 + }, + "some": { + "avg10": 7.14, + "avg300": 5.59, + "avg60": 5.12 + } + }, + "io_pressure_before": { + "full": { + "avg10": 3.75, + "avg300": 5.35, + "avg60": 3.69 + }, + "some": { + "avg10": 3.94, + "avg300": 5.42, + "avg60": 3.75 + } + }, + "root_device_read_delta_bytes": 2019328, + "root_device_write_delta_bytes": 389615616 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 32.164, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 247, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 125, + "p95_ttft_ms": 125, + "p99_elapsed_ms": 191, + "p99_ttft_ms": 191, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 119, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 102, + "p95_ttft_ms": 102, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 150, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 139, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 136, + "minimum_elapsed_ms": 6, + "model": "q008-messages", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 125, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-1ry8y023/config.toml Server: 127.0.0.1:47445 Accounts: 1 Database: Content digest: 06ca981fca8db962cfce6223ef12f5e37fd57fb02be7d045cb8e04c9fb618082", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 153, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 59, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2508802, \"ppid\": 2508780, \"uptime_seconds\": 2.790148176, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 813, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-1ry8y023/backups/eggpool-backup-20261016-050818.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2508802)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 545, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-1ry8y023/recovery/config.toml db: /tmp/eggpool-q008-1ry8y023/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 283, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2510153)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 5.0, + "avg300": 5.41, + "avg60": 3.77 + }, + "some": { + "avg10": 5.01, + "avg300": 5.47, + "avg60": 3.8 + } + }, + "root_device_read_bytes": 757969584128, + "root_device_write_bytes": 1345185383424 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 55, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21520384, + 21520384, + 23601152, + 23601152, + 23728128, + 23883776, + 24035328, + 25853952, + 25853952, + 25853952, + 24055808 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 10, + 2 + ], + "second_workload_ttft_ms": [ + 10, + 2 + ], + "streaming_elapsed_ms": [ + 12, + 13, + 55 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21520384, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21520384, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21520384, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21520384, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23601152, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23601152, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23601152, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23601152, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23728128, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23728128, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23883776, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23883776, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24035328, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24035328, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 25853952, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 25853952, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173232128, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 25853952, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 25853952, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173232128, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 25853952, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 25853952, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173232128, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 24055808, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24055808, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103645184, + "wal_bytes": 885832, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 30, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u10-c1-run-1.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u10-c1-run-1.json new file mode 100644 index 00000000..87c624f2 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u10-c1-run-1.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "08b19400fe53febce1c35e117c31b2f6296400e93a3ffc2600f374bd5ab58fc4", + "archive_size_bytes": 4599255, + "elapsed_ms": 616, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 10, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23695360, + "baseline_rss_bytes": 23695360, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46370816, + "peak_rss_bytes": 46370816 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 24043, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 31.86, + "cpu_ticks": 766, + "cpu_time_ms": 7660 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 2.82, + "avg300": 3.46, + "avg60": 2.77 + }, + "some": { + "avg10": 2.82, + "avg300": 3.5, + "avg60": 2.8 + } + }, + "io_pressure_before": { + "full": { + "avg10": 2.21, + "avg300": 3.61, + "avg60": 3.2 + }, + "some": { + "avg10": 2.21, + "avg300": 3.65, + "avg60": 3.23 + } + }, + "root_device_read_delta_bytes": 12210176, + "root_device_write_delta_bytes": 389709824 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 41.592, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 235, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 109, + "p99_ttft_ms": 109, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 116, + "minimum_elapsed_ms": 5, + "model": "q008-responses", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 19, + "p95_elapsed_ms": 100, + "p95_ttft_ms": 99, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 262, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 17, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 97, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 113, + "minimum_elapsed_ms": 9, + "model": "q008-messages", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 99, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-_ds_uopa/config.toml Server: 127.0.0.1:60565 Accounts: 1 Database: Content digest: b32b9e36fcf7fc4690242ecdf735141d629b9236cac5b98bd33297a45011f934", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 143, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 50, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 3, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 5, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 6, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 6, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 4, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2530925, \"ppid\": 2530903, \"uptime_seconds\": 2.78722747, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, ", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 616, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-_ds_uopa/backups/eggpool-backup-20261016-051523.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 63, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2530925)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 500, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-_ds_uopa/recovery/config.toml db: /tmp/eggpool-q008-_ds_uopa/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 225, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2532288)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 2.1, + "avg300": 3.65, + "avg60": 3.33 + }, + "some": { + "avg10": 2.1, + "avg300": 3.69, + "avg60": 3.37 + } + }, + "root_device_read_bytes": 758132760576, + "root_device_write_bytes": 1353151882240 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 3, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 3 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 52, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 52 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21803008, + 21803008, + 23695360, + 23695360, + 23822336, + 23973888, + 24125440, + 46370816, + 46370816, + 46370816, + 23572480 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 3, + 2, + 2 + ], + "finite_ttft_ms": [ + 3, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 4, + 3 + ], + "second_workload_ttft_ms": [ + 4, + 3 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 52 + ], + "streaming_ttft_ms": [ + 12, + 12, + 52 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21803008, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21803008, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21803008, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21803008, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23695360, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23695360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23695360, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23695360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23822336, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23822336, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23973888, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23973888, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24125440, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24125440, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4575232, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46370816, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46370816, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173219840, + "wal_bytes": 4334272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4575232, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46370816, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46370816, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173219840, + "wal_bytes": 4334272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4575232, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46370816, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46370816, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173219840, + "wal_bytes": 4334272, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23572480, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23572480, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 22, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u10-c1-run-2.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u10-c1-run-2.json new file mode 100644 index 00000000..3ce1e161 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u10-c1-run-2.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "e5c232aa3a3d823006f686824989eee7e629a38e4e056bd47907815cc66d950a", + "archive_size_bytes": 4595159, + "elapsed_ms": 594, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 10, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23818240, + "baseline_rss_bytes": 23818240, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46350336, + "peak_rss_bytes": 46350336 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 24133, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 32.11, + "cpu_ticks": 775, + "cpu_time_ms": 7750 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 2.21, + "avg300": 3.21, + "avg60": 2.14 + }, + "some": { + "avg10": 2.21, + "avg300": 3.24, + "avg60": 2.15 + } + }, + "io_pressure_before": { + "full": { + "avg10": 1.58, + "avg300": 3.37, + "avg60": 2.5 + }, + "some": { + "avg10": 1.58, + "avg300": 3.41, + "avg60": 2.52 + } + }, + "root_device_read_delta_bytes": 794624, + "root_device_write_delta_bytes": 391094272 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 41.436, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 175, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 118, + "p99_ttft_ms": 118, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 5, + "model": "q008-responses", + "p50_elapsed_ms": 18, + "p50_ttft_ms": 18, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 75, + "minimum_elapsed_ms": 6, + "model": "q008-responses", + "p50_elapsed_ms": 14, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 4, + "model": "q008-messages", + "p50_elapsed_ms": 16, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-iic6mhbg/config.toml Server: 127.0.0.1:46741 Accounts: 1 Database: Content digest: 01b031f241244707ca3ede5c4b5942eb4d3fac912e75ea853b597ceb5bcdb4bf", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 139, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 54, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 11, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 16, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2532317, \"ppid\": 2532295, \"uptime_seconds\": 2.808833493, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 594, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-iic6mhbg/backups/eggpool-backup-20261016-051600.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2532317)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 531, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-iic6mhbg/recovery/config.toml db: /tmp/eggpool-q008-iic6mhbg/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 397, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 107, + "id": "stop", + "reason": "Stopping server (PID 2533670)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 2.87, + "avg300": 3.44, + "avg60": 2.77 + }, + "some": { + "avg10": 2.87, + "avg300": 3.48, + "avg60": 2.79 + } + }, + "root_device_read_bytes": 758146314240, + "root_device_write_bytes": 1353620280320 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21725184, + 21725184, + 23818240, + 23818240, + 23941120, + 24076288, + 24248320, + 46350336, + 46350336, + 46350336, + 24084480 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 12, + 13, + 56 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21725184, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21725184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21725184, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21725184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23818240, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23818240, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23818240, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23818240, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23941120, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23941120, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24076288, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24076288, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24248320, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24248320, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4591616, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46350336, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46350336, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173228032, + "wal_bytes": 4457872, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4591616, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46350336, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46350336, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173228032, + "wal_bytes": 4457872, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4591616, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46350336, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46350336, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173228032, + "wal_bytes": 4457872, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 24084480, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24084480, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 24, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u10-c1-run-3.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u10-c1-run-3.json new file mode 100644 index 00000000..39b271ce --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u10-c1-run-3.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "02b1cda25e3769fb78764195e83184af332f0dd27a95a23c4785782b93e996b7", + "archive_size_bytes": 4595159, + "elapsed_ms": 459, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 10, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23793664, + "baseline_rss_bytes": 23793664, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46333952, + "peak_rss_bytes": 46333952 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 23298, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 34.47, + "cpu_ticks": 803, + "cpu_time_ms": 8030 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 0.77, + "avg300": 2.97, + "avg60": 1.66 + }, + "some": { + "avg10": 0.77, + "avg300": 3.0, + "avg60": 1.67 + } + }, + "io_pressure_before": { + "full": { + "avg10": 2.29, + "avg300": 3.18, + "avg60": 2.23 + }, + "some": { + "avg10": 2.29, + "avg300": 3.22, + "avg60": 2.24 + } + }, + "root_device_read_delta_bytes": 606208, + "root_device_write_delta_bytes": 384536576 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 42.921, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 90, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 82, + "p99_ttft_ms": 81, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 19, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 81, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 15, + "p50_ttft_ms": 8, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-0585q6mo/config.toml Server: 127.0.0.1:50259 Accounts: 1 Database: Content digest: 8e07ec44065868931afbcff073381f2b715cc1bae475284493a6d291ed00bd94", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 90, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 38, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2533699, \"ppid\": 2533677, \"uptime_seconds\": 2.792545904, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 459, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-0585q6mo/backups/eggpool-backup-20261016-051636.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 55, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2533699)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 123, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-0585q6mo/recovery/config.toml db: /tmp/eggpool-q008-0585q6mo/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 47, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2534973)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 2.84, + "avg300": 3.21, + "avg60": 2.26 + }, + "some": { + "avg10": 2.84, + "avg300": 3.24, + "avg60": 2.27 + } + }, + "root_device_read_bytes": 758147301376, + "root_device_write_bytes": 1354086851584 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21966848, + 21966848, + 23793664, + 23793664, + 23916544, + 24047616, + 24223744, + 46333952, + 46333952, + 46333952, + 23695360 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 12, + 13, + 56 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21966848, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21966848, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21966848, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21966848, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23793664, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23793664, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23793664, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23793664, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23916544, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23916544, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24047616, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24047616, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24223744, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24223744, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173105152, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4571136, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46333952, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46333952, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173273088, + "wal_bytes": 4433152, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4571136, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46333952, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46333952, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173273088, + "wal_bytes": 4433152, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4571136, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46333952, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46333952, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173273088, + "wal_bytes": 4433152, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23695360, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23695360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 21, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u25-c1-run-1.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u25-c1-run-1.json new file mode 100644 index 00000000..7ffdc584 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u25-c1-run-1.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "79a47cea43088ce57bcbef5459454e223f2bb3f046e45c73d01259aae292cd98", + "archive_size_bytes": 4627927, + "elapsed_ms": 447, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 25, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23535616, + "baseline_rss_bytes": 23535616, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46342144, + "peak_rss_bytes": 46342144 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 23099, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 34.55, + "cpu_ticks": 798, + "cpu_time_ms": 7980 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 0.22, + "avg300": 2.63, + "avg60": 0.96 + }, + "some": { + "avg10": 0.34, + "avg300": 2.67, + "avg60": 1.0 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.23, + "avg300": 2.85, + "avg60": 1.36 + }, + "some": { + "avg10": 0.23, + "avg300": 2.88, + "avg60": 1.36 + } + }, + "root_device_read_delta_bytes": 1015808, + "root_device_write_delta_bytes": 385323008 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 43.293, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 90, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 78, + "p99_ttft_ms": 78, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 82, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 3, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-6_2om2ev/config.toml Server: 127.0.0.1:39125 Accounts: 1 Database: Content digest: c2dac837c04518285ccedf8158b6a02b6a5acba99dfa70da30b5e7511ce95a54", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 91, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 5, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2535002, \"ppid\": 2534980, \"uptime_seconds\": 2.79196363, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, ", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 447, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-6_2om2ev/backups/eggpool-backup-20261016-051711.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 55, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2535002)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 89, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-6_2om2ev/recovery/config.toml db: /tmp/eggpool-q008-6_2om2ev/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 47, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 63, + "id": "stop", + "reason": "Stopping server (PID 2536360)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.42, + "avg300": 2.91, + "avg60": 1.5 + }, + "some": { + "avg10": 0.42, + "avg300": 2.94, + "avg60": 1.51 + } + }, + "root_device_read_bytes": 758148308992, + "root_device_write_bytes": 1354552673280 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21581824, + 21585920, + 23535616, + 23535616, + 23666688, + 23785472, + 23973888, + 46342144, + 46342144, + 46342144, + 23781376 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 5, + 4 + ], + "second_workload_ttft_ms": [ + 5, + 4 + ], + "streaming_elapsed_ms": [ + 12, + 13, + 56 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21581824, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21581824, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21585920, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21585920, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23535616, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23535616, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23535616, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23535616, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23666688, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23666688, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 4.99, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23785472, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23785472, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23973888, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23973888, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173109248, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4603904, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46342144, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46342144, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173522944, + "wal_bytes": 4371352, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4603904, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46342144, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46342144, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173522944, + "wal_bytes": 4371352, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4603904, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46342144, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46342144, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173522944, + "wal_bytes": 4371352, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4497408, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23781376, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23781376, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 19, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u25-c1-run-2.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u25-c1-run-2.json new file mode 100644 index 00000000..97a313e1 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u25-c1-run-2.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "012d952acc185f6c2046d1580aaa05368465dff1b154bd461bdcc7945e7fc93a", + "archive_size_bytes": 4627927, + "elapsed_ms": 444, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 25, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23724032, + "baseline_rss_bytes": 23724032, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46665728, + "peak_rss_bytes": 46665728 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 23042, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 34.68, + "cpu_ticks": 799, + "cpu_time_ms": 7990 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 0.11, + "avg300": 2.36, + "avg60": 0.62 + }, + "some": { + "avg10": 0.12, + "avg300": 2.39, + "avg60": 0.64 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.16, + "avg300": 2.53, + "avg60": 0.82 + }, + "some": { + "avg10": 0.2, + "avg300": 2.56, + "avg60": 0.85 + } + }, + "root_device_read_delta_bytes": 499712, + "root_device_write_delta_bytes": 386707456 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 43.399, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 89, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 65, + "p99_ttft_ms": 65, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 81, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 3, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 7, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-wsa4llgr/config.toml Server: 127.0.0.1:37141 Accounts: 1 Database: Content digest: 642859d45f3c5cb6b43432f32989300c3e42d3df714162f0739152b1c1315a43", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 84, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 40, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2536393, \"ppid\": 2536370, \"uptime_seconds\": 2.792648409, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 444, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-wsa4llgr/backups/eggpool-backup-20261016-051745.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2536393)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 88, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-wsa4llgr/recovery/config.toml db: /tmp/eggpool-q008-wsa4llgr/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 52, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 64, + "id": "stop", + "reason": "Stopping server (PID 2537705)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.15, + "avg300": 2.6, + "avg60": 0.9 + }, + "some": { + "avg10": 0.23, + "avg300": 2.63, + "avg60": 0.93 + } + }, + "root_device_read_bytes": 758150995968, + "root_device_write_bytes": 1355020641280 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 3, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21893120, + 21897216, + 23724032, + 23724032, + 23842816, + 23969792, + 24154112, + 46665728, + 46665728, + 46665728, + 24072192 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 3, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 3, + 2 + ], + "second_workload_ttft_ms": [ + 3, + 2 + ], + "streaming_elapsed_ms": [ + 12, + 13, + 56 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21893120, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21893120, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21897216, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21897216, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23724032, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23724032, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23724032, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23724032, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23842816, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23842816, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23969792, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23969792, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173092864, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24154112, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24154112, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173092864, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4608000, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46665728, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46665728, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173522944, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4608000, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46665728, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46665728, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173522944, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4608000, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46665728, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46665728, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173522944, + "wal_bytes": 4408432, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4497408, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 24072192, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24072192, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 21, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u25-c1-run-3.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u25-c1-run-3.json new file mode 100644 index 00000000..74fada2c --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u25-c1-run-3.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "edf4747810c5dea4b3a51bca24e48a6d0eeefc05d9bdc398d6656e9816986cf6", + "archive_size_bytes": 4627927, + "elapsed_ms": 439, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 25, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23535616, + "baseline_rss_bytes": 23535616, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46329856, + "peak_rss_bytes": 46329856 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 19276, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 24.38, + "cpu_ticks": 470, + "cpu_time_ms": 4700 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 0.55, + "avg300": 2.14, + "avg60": 0.51 + }, + "some": { + "avg10": 0.55, + "avg300": 2.17, + "avg60": 0.52 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.03, + "avg300": 2.26, + "avg60": 0.5 + }, + "some": { + "avg10": 0.03, + "avg300": 2.29, + "avg60": 0.52 + } + }, + "root_device_read_delta_bytes": 32276480, + "root_device_write_delta_bytes": 391954432 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 51.877, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 89, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 66, + "p99_ttft_ms": 65, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 3, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 58, + "p95_ttft_ms": 58, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 79, + "minimum_elapsed_ms": 3, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 9, + "model": "q008-messages", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-_m_z7b65/config.toml Server: 127.0.0.1:58069 Accounts: 1 Database: Content digest: 0dea231425f30f2b1f86b5412b49bcd59a8a2976ac3ceb087c779616cc6774de", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 88, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 38, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 5, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 10, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 10, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 10, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 5, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2537787, \"ppid\": 2537748, \"uptime_seconds\": 2.849933277, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 439, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-_m_z7b65/backups/eggpool-backup-20261016-051817.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2537787)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 115, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-_m_z7b65/recovery/config.toml db: /tmp/eggpool-q008-_m_z7b65/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 52, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2539243)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.07, + "avg300": 2.32, + "avg60": 0.58 + }, + "some": { + "avg10": 0.08, + "avg300": 2.36, + "avg60": 0.59 + } + }, + "root_device_read_bytes": 758152265728, + "root_device_write_bytes": 1355486036992 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 11, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 11 + }, + { + "body_bytes": 309, + "elapsed_ms": 9, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 213, + "elapsed_ms": 9, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 145, + "elapsed_ms": 20, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 270, + "elapsed_ms": 20, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 480, + "elapsed_ms": 58, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21577728, + 21581824, + 23535616, + 23535616, + 23662592, + 23785472, + 23961600, + 46329856, + 46329856, + 46329856, + 23953408 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 11, + 9, + 9 + ], + "finite_ttft_ms": [ + 11, + 9, + 9 + ], + "second_workload_elapsed_ms": [ + 3, + 2 + ], + "second_workload_ttft_ms": [ + 3, + 2 + ], + "streaming_elapsed_ms": [ + 20, + 20, + 58 + ], + "streaming_ttft_ms": [ + 17, + 17, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21577728, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21577728, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21581824, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21581824, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23535616, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23535616, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23535616, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23535616, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23662592, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23662592, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 4.99, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23785472, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23785472, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23961600, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23961600, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4608000, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46329856, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46329856, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173502464, + "wal_bytes": 4354872, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4608000, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46329856, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46329856, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173502464, + "wal_bytes": 4354872, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4608000, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46329856, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46329856, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173502464, + "wal_bytes": 4354872, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4497408, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23953408, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23953408, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 20, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u49-c1-run-1.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u49-c1-run-1.json new file mode 100644 index 00000000..2b1ef335 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u49-c1-run-1.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "f5de427e334807825c54c0221376663b53c1afb6dfbba284775500c62948af3d", + "archive_size_bytes": 4701655, + "elapsed_ms": 412, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 49, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23785472, + "baseline_rss_bytes": 23785472, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 47005696, + "peak_rss_bytes": 47005696 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 15549, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 12.03, + "cpu_ticks": 187, + "cpu_time_ms": 1870 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 1.34, + "avg300": 2.15, + "avg60": 1.06 + }, + "some": { + "avg10": 1.34, + "avg300": 2.18, + "avg60": 1.07 + } + }, + "io_pressure_before": { + "full": { + "avg10": 1.53, + "avg300": 2.18, + "avg60": 0.94 + }, + "some": { + "avg10": 1.53, + "avg300": 2.21, + "avg60": 0.95 + } + }, + "root_device_read_delta_bytes": 139894784, + "root_device_write_delta_bytes": 605548544 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 64.314, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 74, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 53, + "p95_ttft_ms": 53, + "p99_elapsed_ms": 66, + "p99_ttft_ms": 66, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 53, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 52, + "p95_ttft_ms": 52, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 73, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 6, + "p50_ttft_ms": 6, + "p95_elapsed_ms": 52, + "p95_ttft_ms": 52, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 52, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 6, + "p95_elapsed_ms": 52, + "p95_ttft_ms": 52, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-9mdgn7u8/config.toml Server: 127.0.0.1:54563 Accounts: 1 Database: Content digest: 24975033e537845f585ff493f98a23a11e6bcda21a2f9855171455f3a2fd911c", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 92, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 40, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 2, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 5, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2539272, \"ppid\": 2539250, \"uptime_seconds\": 2.762838254, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 412, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-9mdgn7u8/backups/eggpool-backup-20261016-051844.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2539272)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 93, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-9mdgn7u8/recovery/config.toml db: /tmp/eggpool-q008-9mdgn7u8/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 56, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2540657)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 3.13, + "avg300": 2.23, + "avg60": 1.05 + }, + "some": { + "avg10": 3.13, + "avg300": 2.26, + "avg60": 1.06 + } + }, + "root_device_read_bytes": 759136333824, + "root_device_write_bytes": 1356003886080 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 3, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 3 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 52, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 52 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21827584, + 21831680, + 23785472, + 23785472, + 23908352, + 24031232, + 24211456, + 47005696, + 47005696, + 47005696, + 23982080 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 3, + 2, + 2 + ], + "finite_ttft_ms": [ + 3, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 4, + 2 + ], + "second_workload_ttft_ms": [ + 3, + 2 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 52 + ], + "streaming_ttft_ms": [ + 12, + 12, + 52 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21827584, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21827584, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21831680, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21831680, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23785472, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23785472, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172965888, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23785472, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23785472, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172965888, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23908352, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23908352, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172965888, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24031232, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24031232, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173101056, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24211456, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24211456, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173101056, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4698112, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 47005696, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47005696, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174014464, + "wal_bytes": 4346632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4698112, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 47005696, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47005696, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174014464, + "wal_bytes": 4346632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4698112, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 47005696, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47005696, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174014464, + "wal_bytes": 4346632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4567040, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23982080, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23982080, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 885832, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 20, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u49-c1-run-2.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u49-c1-run-2.json new file mode 100644 index 00000000..31555bf4 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u49-c1-run-2.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "5181ccc5164fe1843dab283e7c8d4528a5df5223cc9ea8237d08f0ac7f54934b", + "archive_size_bytes": 4697559, + "elapsed_ms": 990, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 49, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23871488, + "baseline_rss_bytes": 23871488, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 47112192, + "peak_rss_bytes": 47112192 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 24534, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 8.19, + "cpu_ticks": 201, + "cpu_time_ms": 2010 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 6.6, + "avg300": 2.55, + "avg60": 3.26 + }, + "some": { + "avg10": 6.6, + "avg300": 2.57, + "avg60": 3.26 + } + }, + "io_pressure_before": { + "full": { + "avg10": 1.37, + "avg300": 2.1, + "avg60": 1.06 + }, + "some": { + "avg10": 1.37, + "avg300": 2.13, + "avg60": 1.07 + } + }, + "root_device_read_delta_bytes": 64331776, + "root_device_write_delta_bytes": 412794880 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 40.76, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 351, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 128, + "p95_ttft_ms": 128, + "p99_elapsed_ms": 172, + "p99_ttft_ms": 172, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 53, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 52, + "p95_ttft_ms": 52, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 64, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 6, + "p50_ttft_ms": 6, + "p95_elapsed_ms": 52, + "p95_ttft_ms": 52, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 117, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 6, + "p95_elapsed_ms": 53, + "p95_ttft_ms": 52, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-b9yffq_x/config.toml Server: 127.0.0.1:45875 Accounts: 1 Database: Content digest: 79162564e8f456352bb6e4f3b3ccd8997444b365b0461f74d3d240d1e1acc405", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 85, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 40, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 2, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 5, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2540686, \"ppid\": 2540664, \"uptime_seconds\": 2.767594247, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 15, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 990, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-b9yffq_x/backups/eggpool-backup-20261016-051921.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 520, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2540686)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 1937, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-b9yffq_x/recovery/config.toml db: /tmp/eggpool-q008-b9yffq_x/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 834, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2542090)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.86, + "avg300": 2.11, + "avg60": 0.99 + }, + "some": { + "avg10": 0.86, + "avg300": 2.14, + "avg60": 1.0 + } + }, + "root_device_read_bytes": 759311384576, + "root_device_write_bytes": 1356706051072 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 53, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 52 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21770240, + 21770240, + 23871488, + 23871488, + 23982080, + 24109056, + 24285184, + 47112192, + 47112192, + 47112192, + 23867392 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 5, + 3 + ], + "second_workload_ttft_ms": [ + 5, + 3 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 53 + ], + "streaming_ttft_ms": [ + 12, + 12, + 52 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21770240, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21770240, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21770240, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21770240, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23871488, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23871488, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23871488, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23871488, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23982080, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23982080, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24109056, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24109056, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24285184, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24285184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173113344, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4694016, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 47112192, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47112192, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174022656, + "wal_bytes": 4400192, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4694016, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 47112192, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47112192, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174022656, + "wal_bytes": 4400192, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4694016, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 47112192, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47112192, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174022656, + "wal_bytes": 4400192, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4567040, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23867392, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23867392, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 894072, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 20, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u49-c1-run-3.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u49-c1-run-3.json new file mode 100644 index 00000000..5ffa7fad --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u49-c1-run-3.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "9a9aa6eb1ba4d25b0d2d64e77aa636371b7292db546dd70fa9461644e525234a", + "archive_size_bytes": 4697559, + "elapsed_ms": 617, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 49, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23879680, + "baseline_rss_bytes": 23879680, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 47112192, + "peak_rss_bytes": 47112192 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 36039, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 7.99, + "cpu_ticks": 288, + "cpu_time_ms": 2880 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 32.76, + "avg300": 6.48, + "avg60": 16.57 + }, + "some": { + "avg10": 33.06, + "avg300": 6.54, + "avg60": 16.71 + } + }, + "io_pressure_before": { + "full": { + "avg10": 3.04, + "avg300": 3.36, + "avg60": 4.84 + }, + "some": { + "avg10": 3.04, + "avg300": 3.39, + "avg60": 4.87 + } + }, + "root_device_read_delta_bytes": 59129856, + "root_device_write_delta_bytes": 1405227008 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 27.748, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 654, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 126, + "p95_ttft_ms": 126, + "p99_elapsed_ms": 510, + "p99_ttft_ms": 510, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 53, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 52, + "p95_ttft_ms": 52, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 64, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 6, + "p50_ttft_ms": 6, + "p95_elapsed_ms": 52, + "p95_ttft_ms": 52, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 52, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 5, + "p95_elapsed_ms": 52, + "p95_ttft_ms": 52, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-q5swq2dm/config.toml Server: 127.0.0.1:44505 Accounts: 1 Database: Content digest: 2a2abbc5b45465d55d7892ed8e19ded45419c2588f608076d5cb60b8fa38a261", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 87, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 40, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 2, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 5, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2542278, \"ppid\": 2542256, \"uptime_seconds\": 2.768131839, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 617, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-q5swq2dm/backups/eggpool-backup-20261016-052056.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 105, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2542278)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 493, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-q5swq2dm/recovery/config.toml db: /tmp/eggpool-q008-q5swq2dm/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 261, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2543735)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 3.84, + "avg300": 3.39, + "avg60": 5.14 + }, + "some": { + "avg10": 3.85, + "avg300": 3.42, + "avg60": 5.17 + } + }, + "root_device_read_bytes": 759902724096, + "root_device_write_bytes": 1359184331776 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 3, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 3 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 52, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 52 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21987328, + 21991424, + 23879680, + 23879680, + 24006656, + 24133632, + 24363008, + 47112192, + 47112192, + 47112192, + 24010752 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 3, + 2, + 2 + ], + "finite_ttft_ms": [ + 3, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 4, + 3 + ], + "second_workload_ttft_ms": [ + 4, + 3 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 52 + ], + "streaming_ttft_ms": [ + 12, + 12, + 52 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21987328, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21987328, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21991424, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21991424, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23879680, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23879680, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23879680, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23879680, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 24006656, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24006656, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24133632, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24133632, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24363008, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24363008, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4681728, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 47112192, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47112192, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174010368, + "wal_bytes": 4420792, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4681728, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 47112192, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47112192, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174010368, + "wal_bytes": 4420792, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4681728, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 47112192, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47112192, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174010368, + "wal_bytes": 4420792, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4567040, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 24010752, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24010752, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 20, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c1-run-1.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c1-run-1.json new file mode 100644 index 00000000..076ec1ad --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c1-run-1.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "7ed6e6fdd548302e923c9640164228a5ba8ee7e6e5de3d2e93094333a2f5306e", + "archive_size_bytes": 4607447, + "elapsed_ms": 629, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23859200, + "baseline_rss_bytes": 23859200, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46530560, + "peak_rss_bytes": 46530560 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 30230, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 26.03, + "cpu_ticks": 787, + "cpu_time_ms": 7870 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 5.69, + "avg300": 5.47, + "avg60": 5.15 + }, + "some": { + "avg10": 5.69, + "avg300": 5.52, + "avg60": 5.17 + } + }, + "io_pressure_before": { + "full": { + "avg10": 4.24, + "avg300": 5.43, + "avg60": 4.73 + }, + "some": { + "avg10": 4.24, + "avg300": 5.49, + "avg60": 4.76 + } + }, + "root_device_read_delta_bytes": 1986560, + "root_device_write_delta_bytes": 394305536 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 33.079, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 254, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 120, + "p95_ttft_ms": 120, + "p99_elapsed_ms": 172, + "p99_ttft_ms": 172, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 118, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 17, + "p50_ttft_ms": 17, + "p95_elapsed_ms": 94, + "p95_ttft_ms": 94, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 135, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 129, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 149, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 7, + "p95_elapsed_ms": 122, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-mrnj9zwa/config.toml Server: 127.0.0.1:49795 Accounts: 1 Database: Content digest: d78ae308c2fcf85231bb9cb913f268a72d69319987f5c6f2812e6004f503542a", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 165, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 85, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 5, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2510182, \"ppid\": 2510160, \"uptime_seconds\": 2.800488889, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 629, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-mrnj9zwa/backups/eggpool-backup-20261016-050901.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 58, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2510182)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 515, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-mrnj9zwa/recovery/config.toml db: /tmp/eggpool-q008-mrnj9zwa/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 217, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 55, + "id": "stop", + "reason": "Stopping server (PID 2511522)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 5.06, + "avg300": 5.47, + "avg60": 4.85 + }, + "some": { + "avg10": 5.07, + "avg300": 5.53, + "avg60": 4.89 + } + }, + "root_device_read_bytes": 757972017152, + "root_device_write_bytes": 1345649742848 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 11 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 22036480, + 22036480, + 23859200, + 23859200, + 23994368, + 24137728, + 24358912, + 46530560, + 46530560, + 46530560, + 23736320 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 56 + ], + "streaming_ttft_ms": [ + 11, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 22036480, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 22036480, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 22036480, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 22036480, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23859200, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23859200, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23859200, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23859200, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23994368, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23994368, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24137728, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24137728, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24358912, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24358912, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4603904, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46530560, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46530560, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173219840, + "wal_bytes": 4358992, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4603904, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46530560, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46530560, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173219840, + "wal_bytes": 4358992, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4603904, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46530560, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46530560, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173219840, + "wal_bytes": 4358992, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23736320, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23736320, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 43, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c1-run-2.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c1-run-2.json new file mode 100644 index 00000000..17b5d9b5 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c1-run-2.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "8d041d2f78705db545d6ee8245c7be0c4d602757fd3376af559cfd8c319cd892", + "archive_size_bytes": 4607447, + "elapsed_ms": 460, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23793664, + "baseline_rss_bytes": 23793664, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46309376, + "peak_rss_bytes": 46309376 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 25674, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 31.86, + "cpu_ticks": 818, + "cpu_time_ms": 8180 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 1.6, + "avg300": 5.11, + "avg60": 3.87 + }, + "some": { + "avg10": 1.63, + "avg300": 5.16, + "avg60": 3.9 + } + }, + "io_pressure_before": { + "full": { + "avg10": 3.17, + "avg300": 5.33, + "avg60": 4.61 + }, + "some": { + "avg10": 3.17, + "avg300": 5.38, + "avg60": 4.62 + } + }, + "root_device_read_delta_bytes": 6107136, + "root_device_write_delta_bytes": 387338240 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 38.949, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 204, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 151, + "p99_ttft_ms": 151, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 120, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 95, + "p95_ttft_ms": 95, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 139, + "minimum_elapsed_ms": 7, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 120, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 131, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 129, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-5t5wocey/config.toml Server: 127.0.0.1:45971 Accounts: 1 Database: Content digest: 16e9572a52892401b8c63d105a985e5a72956911fa9f64b3e1590730b1f16f0c", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 127, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 58, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2511551, \"ppid\": 2511529, \"uptime_seconds\": 2.790748448, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 7, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 460, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-5t5wocey/backups/eggpool-backup-20261016-050939.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2511551)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 112, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-5t5wocey/recovery/config.toml db: /tmp/eggpool-q008-5t5wocey/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 49, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 64, + "id": "stop", + "reason": "Stopping server (PID 2512843)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 4.57, + "avg300": 5.42, + "avg60": 4.95 + }, + "some": { + "avg10": 4.57, + "avg300": 5.47, + "avg60": 4.97 + } + }, + "root_device_read_bytes": 757974593536, + "root_device_write_bytes": 1346118677504 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 55, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21966848, + 21966848, + 23793664, + 23793664, + 23916544, + 24059904, + 24219648, + 46309376, + 46309376, + 46309376, + 23957504 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 10, + 2 + ], + "second_workload_ttft_ms": [ + 10, + 2 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 55 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21966848, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21966848, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21966848, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21966848, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23793664, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23793664, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23793664, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23793664, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23916544, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23916544, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24059904, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24059904, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173092864, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24219648, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24219648, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173092864, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4583424, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46309376, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46309376, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173215744, + "wal_bytes": 4367232, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4583424, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46309376, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46309376, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173215744, + "wal_bytes": 4367232, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4583424, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46309376, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46309376, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173215744, + "wal_bytes": 4367232, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23957504, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23957504, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 21, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c1-run-3.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c1-run-3.json new file mode 100644 index 00000000..3b00be8d --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c1-run-3.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "edf7853d4d66ab9cc61d8a53c1332ad1d81e7d88efbc9a25d4d43f37ff55233a", + "archive_size_bytes": 4607447, + "elapsed_ms": 460, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 1, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23588864, + "baseline_rss_bytes": 23588864, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46186496, + "peak_rss_bytes": 46186496 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 27070, + "client_concurrency": 1, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 30.73, + "cpu_ticks": 832, + "cpu_time_ms": 8320 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 2.94, + "avg300": 4.79, + "avg60": 3.3 + }, + "some": { + "avg10": 2.94, + "avg300": 4.83, + "avg60": 3.32 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.48, + "avg300": 4.9, + "avg60": 3.16 + }, + "some": { + "avg10": 0.49, + "avg300": 4.95, + "avg60": 3.19 + } + }, + "root_device_read_delta_bytes": 5697536, + "root_device_write_delta_bytes": 396316672 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 36.941, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 183, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 16, + "p50_ttft_ms": 16, + "p95_elapsed_ms": 62, + "p95_ttft_ms": 62, + "p99_elapsed_ms": 137, + "p99_ttft_ms": 137, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 80, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 9, + "p50_ttft_ms": 7, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 8, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-rnlm24ec/config.toml Server: 127.0.0.1:55143 Accounts: 1 Database: Content digest: 7fad4641d4d9cde3a09e51adf167e47a7eb5a3a1a3fa65a2b00796040bc242c3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 88, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2512872, \"ppid\": 2512850, \"uptime_seconds\": 2.786559193, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 460, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-rnlm24ec/backups/eggpool-backup-20261016-051018.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2512872)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 93, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-rnlm24ec/recovery/config.toml db: /tmp/eggpool-q008-rnlm24ec/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 49, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop", + "reason": "Stopping server (PID 2514240)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.87, + "avg300": 5.0, + "avg60": 3.5 + }, + "some": { + "avg10": 0.89, + "avg300": 5.06, + "avg60": 3.53 + } + }, + "root_device_read_bytes": 757983637504, + "root_device_write_bytes": 1346585555968 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 3, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21630976, + 21630976, + 23588864, + 23588864, + 23719936, + 23838720, + 24014848, + 46186496, + 46186496, + 46186496, + 23838720 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 3, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 10, + 2 + ], + "second_workload_ttft_ms": [ + 10, + 2 + ], + "streaming_elapsed_ms": [ + 12, + 13, + 56 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21630976, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21630976, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21630976, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21630976, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23588864, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23588864, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23588864, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23588864, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23719936, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23719936, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23838720, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23838720, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173101056, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24014848, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24014848, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173101056, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4603904, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46186496, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46186496, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173228032, + "wal_bytes": 4350752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4603904, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46186496, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46186496, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173228032, + "wal_bytes": 4350752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4603904, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46186496, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46186496, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173228032, + "wal_bytes": 4350752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23838720, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23838720, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 20, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c16-run-1.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c16-run-1.json new file mode 100644 index 00000000..5dfe757e --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c16-run-1.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "8771f09b0922d1547c8b647ddf498ca31023ecc607d3e78ed0479490c3d227fb", + "archive_size_bytes": 4709847, + "elapsed_ms": 447, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 16, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23703552, + "baseline_rss_bytes": 23703552, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 47927296, + "peak_rss_bytes": 47927296 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 2595, + "client_concurrency": 16, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 65.12, + "cpu_ticks": 169, + "cpu_time_ms": 1690 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 5.48, + "avg300": 3.89, + "avg60": 4.54 + }, + "some": { + "avg10": 5.53, + "avg300": 3.94, + "avg60": 4.61 + } + }, + "io_pressure_before": { + "full": { + "avg10": 5.14, + "avg300": 3.87, + "avg60": 4.46 + }, + "some": { + "avg10": 5.2, + "avg300": 3.92, + "avg60": 4.53 + } + }, + "root_device_read_delta_bytes": 53248, + "root_device_write_delta_bytes": 390266880 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 385.329, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 1078, + "minimum_elapsed_ms": 5, + "p50_elapsed_ms": 31, + "p50_ttft_ms": 31, + "p95_elapsed_ms": 84, + "p95_ttft_ms": 84, + "p99_elapsed_ms": 93, + "p99_ttft_ms": 93, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 7, + "model": "q008-responses", + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 81, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 10, + "p50_ttft_ms": 7, + "p95_elapsed_ms": 57, + "p95_ttft_ms": 56, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 4, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-g6sx_ckk/config.toml Server: 127.0.0.1:55521 Accounts: 1 Database: Content digest: d0cc75eca741f69f05e8fc6dff78b12442988911fadd159f231b8fb76c9a55c3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 129, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 50, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 5, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2527242, \"ppid\": 2527220, \"uptime_seconds\": 2.81388076, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, ", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 447, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-g6sx_ckk/backups/eggpool-backup-20261016-051417.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 56, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2527242)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 98, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-g6sx_ckk/recovery/config.toml db: /tmp/eggpool-q008-g6sx_ckk/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 51, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 56, + "id": "stop", + "reason": "Stopping server (PID 2528440)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 9.45, + "avg300": 3.91, + "avg60": 4.79 + }, + "some": { + "avg10": 9.59, + "avg300": 3.96, + "avg60": 4.88 + } + }, + "root_device_read_bytes": 758130028544, + "root_device_write_bytes": 1351741920256 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 4, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 4 + }, + { + "body_bytes": 309, + "elapsed_ms": 4, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 4 + }, + { + "body_bytes": 213, + "elapsed_ms": 5, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 5 + }, + { + "body_bytes": 145, + "elapsed_ms": 17, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 15 + }, + { + "body_bytes": 270, + "elapsed_ms": 19, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 480, + "elapsed_ms": 60, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21811200, + 21811200, + 23703552, + 23703552, + 23822336, + 23949312, + 24125440, + 47927296, + 47927296, + 47927296, + 23711744 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 4, + 4, + 5 + ], + "finite_ttft_ms": [ + 4, + 4, + 5 + ], + "second_workload_elapsed_ms": [ + 9, + 2 + ], + "second_workload_ttft_ms": [ + 9, + 2 + ], + "streaming_elapsed_ms": [ + 17, + 19, + 60 + ], + "streaming_ttft_ms": [ + 15, + 17, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21811200, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21811200, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21811200, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21811200, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23703552, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23703552, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23703552, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23703552, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23822336, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23822336, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23949312, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23949312, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173101056, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24125440, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24125440, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173101056, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4702208, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 47927296, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47927296, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 175034368, + "wal_bytes": 4470232, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4702208, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 47927296, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47927296, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 175034368, + "wal_bytes": 4470232, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4702208, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 47927296, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47927296, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 175034368, + "wal_bytes": 4470232, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23711744, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23711744, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 25, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c16-run-2.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c16-run-2.json new file mode 100644 index 00000000..b5f4f60b --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c16-run-2.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "b0a4ee0fc267bdebcb55fc1275274abf621f2b8556f493d9050103a1c4243a09", + "archive_size_bytes": 4709847, + "elapsed_ms": 456, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 16, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23715840, + "baseline_rss_bytes": 23715840, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 48005120, + "peak_rss_bytes": 48005120 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 3143, + "client_concurrency": 16, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 77.64, + "cpu_ticks": 244, + "cpu_time_ms": 2440 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 2.79, + "avg300": 3.79, + "avg60": 3.96 + }, + "some": { + "avg10": 2.8, + "avg300": 3.83, + "avg60": 4.01 + } + }, + "io_pressure_before": { + "full": { + "avg10": 2.08, + "avg300": 3.77, + "avg60": 3.89 + }, + "some": { + "avg10": 2.1, + "avg300": 3.82, + "avg60": 3.95 + } + }, + "root_device_read_delta_bytes": 90112, + "root_device_write_delta_bytes": 392114176 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 318.21, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 1119, + "minimum_elapsed_ms": 5, + "p50_elapsed_ms": 40, + "p50_ttft_ms": 40, + "p95_elapsed_ms": 91, + "p95_ttft_ms": 91, + "p99_elapsed_ms": 119, + "p99_ttft_ms": 119, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 4, + "model": "q008-responses", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 19, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 85, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 14, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 3, + "model": "q008-messages", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-8v2w4uwj/config.toml Server: 127.0.0.1:37569 Accounts: 1 Database: Content digest: f3ef0ed49dc7c265b88ef1ab6e48bf45271ce31e748a978bdb4bc7274201edac", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 84, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 38, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2528469, \"ppid\": 2528447, \"uptime_seconds\": 2.780740459, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 456, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-8v2w4uwj/backups/eggpool-backup-20261016-051432.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2528469)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 101, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-8v2w4uwj/recovery/config.toml db: /tmp/eggpool-q008-8v2w4uwj/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 55, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop", + "reason": "Stopping server (PID 2529667)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 4.41, + "avg300": 3.87, + "avg60": 4.41 + }, + "some": { + "avg10": 4.45, + "avg300": 3.92, + "avg60": 4.47 + } + }, + "root_device_read_bytes": 758130724864, + "root_device_write_bytes": 1352212046848 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 13, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21823488, + 21823488, + 23715840, + 23715840, + 23834624, + 23961600, + 24141824, + 48005120, + 48005120, + 48005120, + 23982080 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 12, + 13, + 56 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21823488, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21823488, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21823488, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21823488, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23715840, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23715840, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23715840, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23715840, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23834624, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23834624, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23961600, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23961600, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24141824, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24141824, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4706304, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 48005120, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 48005120, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174895104, + "wal_bytes": 4387832, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4706304, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 48005120, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 48005120, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174895104, + "wal_bytes": 4387832, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4706304, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 48005120, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 48005120, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174895104, + "wal_bytes": 4387832, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23982080, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23982080, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 21, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c16-run-3.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c16-run-3.json new file mode 100644 index 00000000..d22664f2 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c16-run-3.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "bc7d2529a4eaf8e9555f3acc0d7a1e2e6d45fff2714e0ad6090c9728bf576615", + "archive_size_bytes": 4718039, + "elapsed_ms": 627, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 16, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23769088, + "baseline_rss_bytes": 23769088, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 48058368, + "peak_rss_bytes": 48058368 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 3064, + "client_concurrency": 16, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 75.72, + "cpu_ticks": 232, + "cpu_time_ms": 2320 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 1.84, + "avg300": 3.67, + "avg60": 3.41 + }, + "some": { + "avg10": 1.84, + "avg300": 3.71, + "avg60": 3.45 + } + }, + "io_pressure_before": { + "full": { + "avg10": 1.14, + "avg300": 3.66, + "avg60": 3.35 + }, + "some": { + "avg10": 1.14, + "avg300": 3.7, + "avg60": 3.4 + } + }, + "root_device_read_delta_bytes": 16384, + "root_device_write_delta_bytes": 393334784 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 326.388, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 1115, + "minimum_elapsed_ms": 5, + "p50_elapsed_ms": 38, + "p50_ttft_ms": 38, + "p95_elapsed_ms": 91, + "p95_ttft_ms": 91, + "p99_elapsed_ms": 119, + "p99_ttft_ms": 118, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 82, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 14, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 7, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-hs7oyq9k/config.toml Server: 127.0.0.1:52047 Accounts: 1 Database: Content digest: 54f98dc027e0b91e00ca966fa6049983614b86fb30778943379fcd99ac24d0c8", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 87, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 2, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 5, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 5, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2529697, \"ppid\": 2529674, \"uptime_seconds\": 2.832350943, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 627, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-hs7oyq9k/backups/eggpool-backup-20261016-051446.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2529697)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 490, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-hs7oyq9k/recovery/config.toml db: /tmp/eggpool-q008-hs7oyq9k/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 268, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 55, + "id": "stop", + "reason": "Stopping server (PID 2530896)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 2.07, + "avg300": 3.73, + "avg60": 3.7 + }, + "some": { + "avg10": 2.08, + "avg300": 3.78, + "avg60": 3.76 + } + }, + "root_device_read_bytes": 758132506624, + "root_device_write_bytes": 1352682619904 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 11, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 11 + }, + { + "body_bytes": 309, + "elapsed_ms": 10, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 213, + "elapsed_ms": 9, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 145, + "elapsed_ms": 20, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 270, + "elapsed_ms": 20, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 480, + "elapsed_ms": 57, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 56 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21880832, + 21880832, + 23769088, + 23769088, + 23896064, + 24031232, + 24207360, + 48058368, + 48058368, + 48058368, + 24096768 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 11, + 10, + 9 + ], + "finite_ttft_ms": [ + 11, + 9, + 9 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 20, + 20, + 57 + ], + "streaming_ttft_ms": [ + 17, + 17, + 56 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21880832, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21880832, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21880832, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21880832, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23769088, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23769088, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23769088, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23769088, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23896064, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23896064, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24031232, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24031232, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173092864, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24207360, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24207360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173092864, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4710400, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 48058368, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 48058368, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 175054848, + "wal_bytes": 4346632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4710400, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 48058368, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 48058368, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 175054848, + "wal_bytes": 4346632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4710400, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 48058368, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 48058368, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 175054848, + "wal_bytes": 4346632, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 24096768, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24096768, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 885832, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 20, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c2-run-1.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c2-run-1.json new file mode 100644 index 00000000..c927b5f2 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c2-run-1.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "517acc80b1b83aa6a61472753c29b12934cd6edeff8bf354cfd59ea6a5ece246", + "archive_size_bytes": 4632023, + "elapsed_ms": 628, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 2, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23867392, + "baseline_rss_bytes": 23867392, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46489600, + "peak_rss_bytes": 46489600 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 14335, + "client_concurrency": 2, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 50.51, + "cpu_ticks": 724, + "cpu_time_ms": 7240 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 4.64, + "avg300": 4.6, + "avg60": 3.18 + }, + "some": { + "avg10": 4.64, + "avg300": 4.65, + "avg60": 3.19 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.95, + "avg300": 4.6, + "avg60": 2.73 + }, + "some": { + "avg10": 0.95, + "avg300": 4.64, + "avg60": 2.74 + } + }, + "root_device_read_delta_bytes": 634880, + "root_device_write_delta_bytes": 386879488 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 69.759, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 285, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 112, + "p95_ttft_ms": 112, + "p99_elapsed_ms": 167, + "p99_ttft_ms": 167, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 75, + "minimum_elapsed_ms": 3, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 14, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-s72_47zp/config.toml Server: 127.0.0.1:57267 Accounts: 1 Database: Content digest: 7476eb7a62d479beafc86fa6a791a6d13f88f622329bd62fb086fd83d86888bc", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 84, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 38, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 11, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 16, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2514269, \"ppid\": 2514247, \"uptime_seconds\": 2.800984745, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 628, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-s72_47zp/backups/eggpool-backup-20261016-051044.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2514269)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 476, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-s72_47zp/recovery/config.toml db: /tmp/eggpool-q008-s72_47zp/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 210, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop", + "reason": "Stopping server (PID 2515472)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 1.73, + "avg300": 4.69, + "avg60": 3.01 + }, + "some": { + "avg10": 1.73, + "avg300": 4.74, + "avg60": 3.03 + } + }, + "root_device_read_bytes": 757989953536, + "root_device_write_bytes": 1347057292288 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 15, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 13 + }, + { + "body_bytes": 480, + "elapsed_ms": 60, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21905408, + 21909504, + 23867392, + 23867392, + 23982080, + 24117248, + 24358912, + 46489600, + 46489600, + 46489600, + 23789568 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 12, + 15, + 60 + ], + "streaming_ttft_ms": [ + 12, + 13, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21905408, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21905408, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21909504, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21909504, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23867392, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23867392, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23867392, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23867392, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23982080, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23982080, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24117248, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24117248, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24358912, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24358912, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173109248, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4628480, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46489600, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46489600, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173375488, + "wal_bytes": 4367232, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4628480, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46489600, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46489600, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173375488, + "wal_bytes": 4367232, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4628480, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46489600, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46489600, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173375488, + "wal_bytes": 4367232, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23789568, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23789568, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 20, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c2-run-2.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c2-run-2.json new file mode 100644 index 00000000..ed23649d --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c2-run-2.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "2c5f4d6001c528a497f6d23aff3e5fcf0372ad8830529e48174573ffdf4ed925", + "archive_size_bytes": 4644311, + "elapsed_ms": 453, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 2, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23420928, + "baseline_rss_bytes": 23420928, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46112768, + "peak_rss_bytes": 46112768 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 10774, + "client_concurrency": 2, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 54.3, + "cpu_ticks": 585, + "cpu_time_ms": 5850 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 1.9, + "avg300": 4.41, + "avg60": 2.83 + }, + "some": { + "avg10": 1.9, + "avg300": 4.45, + "avg60": 2.83 + } + }, + "io_pressure_before": { + "full": { + "avg10": 3.41, + "avg300": 4.55, + "avg60": 3.19 + }, + "some": { + "avg10": 3.41, + "avg300": 4.59, + "avg60": 3.2 + } + }, + "root_device_read_delta_bytes": 819200, + "root_device_write_delta_bytes": 392257536 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 92.816, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 93, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 62, + "p95_ttft_ms": 62, + "p99_elapsed_ms": 82, + "p99_ttft_ms": 82, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 126, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 96, + "p95_ttft_ms": 96, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 172, + "minimum_elapsed_ms": 6, + "model": "q008-responses", + "p50_elapsed_ms": 14, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 139, + "p95_ttft_ms": 58, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 3, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 8, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-pj8koh54/config.toml Server: 127.0.0.1:45401 Accounts: 1 Database: Content digest: 7a9132ba7c8df4d283134f965bfbc89b1288cb74a5601e3e8a3e312814b1d7db", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 136, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 59, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 3, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 7, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 9, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 13, + "id": "transcoding-stats", + "reason": "{ \"total\": 5, \"native_count\": 5, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 12, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2515501, \"ppid\": 2515479, \"uptime_seconds\": 2.809981561, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 453, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-pj8koh54/backups/eggpool-backup-20261016-051107.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 60, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2515501)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 125, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-pj8koh54/recovery/config.toml db: /tmp/eggpool-q008-pj8koh54/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 47, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2516893)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 4.78, + "avg300": 4.61, + "avg60": 3.35 + }, + "some": { + "avg10": 4.78, + "avg300": 4.65, + "avg60": 3.36 + } + }, + "root_device_read_bytes": 757990805504, + "root_device_write_bytes": 1347523875840 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 53, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 52 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21450752, + 21450752, + 23420928, + 23420928, + 23531520, + 23674880, + 23834624, + 46112768, + 46112768, + 46112768, + 23965696 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 53 + ], + "streaming_ttft_ms": [ + 12, + 12, + 52 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21450752, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21450752, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21450752, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21450752, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23420928, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23420928, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23420928, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23420928, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23531520, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23531520, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23674880, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23674880, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23834624, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23834624, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173105152, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4632576, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46112768, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46112768, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173363200, + "wal_bytes": 4375472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4632576, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46112768, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46112768, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173363200, + "wal_bytes": 4375472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4632576, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46112768, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46112768, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173363200, + "wal_bytes": 4375472, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23965696, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23965696, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 885832, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 26, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c2-run-3.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c2-run-3.json new file mode 100644 index 00000000..ee3f632e --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c2-run-3.json @@ -0,0 +1,1080 @@ +{ + "backup": { + "archive_sha256": "ab726f2710a0a70e9e315ccd5acb0f901082705cd63bfe58e1f659b374b1c638", + "archive_size_bytes": 4636119, + "elapsed_ms": 489, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 2, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23826432, + "baseline_rss_bytes": 23826432, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46448640, + "peak_rss_bytes": 46448640 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 10385, + "client_concurrency": 2, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 56.52, + "cpu_ticks": 587, + "cpu_time_ms": 5870 + }, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 0.45, + "avg300": 4.11, + "avg60": 2.04 + }, + "some": { + "avg10": 0.45, + "avg300": 4.15, + "avg60": 2.05 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.67, + "avg300": 4.24, + "avg60": 2.34 + }, + "some": { + "avg10": 0.67, + "avg300": 4.28, + "avg60": 2.35 + } + }, + "root_device_read_delta_bytes": 733184, + "root_device_write_delta_bytes": 390238208 + }, + "http_status_counts": { + "200": 1000 + }, + "request_count": 1000, + "requests_per_second": 96.292, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 84, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 61, + "p95_ttft_ms": 61, + "p99_elapsed_ms": 73, + "p99_ttft_ms": 73, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 83, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 4, + "model": "q008-messages", + "p50_elapsed_ms": 18, + "p50_ttft_ms": 7, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-be0xtmrp/config.toml Server: 127.0.0.1:52681 Accounts: 1 Database: Content digest: abae8cda3ff4feb39d4a9fb2747a909ab32ba0664d7300ca85132f2808e1b2aa", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 87, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2516922, \"ppid\": 2516900, \"uptime_seconds\": 2.78958184, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, ", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 489, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-be0xtmrp/backups/eggpool-backup-20261016-051129.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 64, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2516922)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 126, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-be0xtmrp/recovery/config.toml db: /tmp/eggpool-q008-be0xtmrp/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 54, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 55, + "id": "stop", + "reason": "Stopping server (PID 2518124)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 1.27, + "avg300": 4.35, + "avg60": 2.64 + }, + "some": { + "avg10": 1.27, + "avg300": 4.39, + "avg60": 2.65 + } + }, + "root_device_read_bytes": 757992267776, + "root_device_write_bytes": 1347994768384 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21868544, + 21868544, + 23826432, + 23826432, + 23957504, + 24080384, + 24326144, + 46448640, + 46448640, + 46448640, + 24068096 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 56 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21868544, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21868544, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21868544, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21868544, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23826432, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23826432, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23826432, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23826432, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23957504, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23957504, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172957696, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24080384, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24080384, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173092864, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24326144, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24326144, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173092864, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4628480, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46448640, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46448640, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173350912, + "wal_bytes": 4371352, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4628480, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46448640, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46448640, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173350912, + "wal_bytes": 4371352, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4628480, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46448640, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46448640, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173350912, + "wal_bytes": 4371352, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 24068096, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24068096, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103645184, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 19, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c4-run-1.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c4-run-1.json new file mode 100644 index 00000000..0f689081 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c4-run-1.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "de483e9f65a2bc7a33b9ba864ee042fe0ddf84247fb13d43396bbbeeeac56fdd", + "archive_size_bytes": 4656599, + "elapsed_ms": 445, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 4, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23830528, + "baseline_rss_bytes": 23830528, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46735360, + "peak_rss_bytes": 46735360 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 5084, + "client_concurrency": 4, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 62.75, + "cpu_ticks": 319, + "cpu_time_ms": 3190 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 1.36, + "avg300": 3.94, + "avg60": 1.82 + }, + "some": { + "avg10": 1.36, + "avg300": 3.98, + "avg60": 1.82 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.28, + "avg300": 3.98, + "avg60": 1.76 + }, + "some": { + "avg10": 0.28, + "avg300": 4.01, + "avg60": 1.76 + } + }, + "root_device_read_delta_bytes": 97591296, + "root_device_write_delta_bytes": 392986624 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 196.71, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 104, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "p99_elapsed_ms": 82, + "p99_ttft_ms": 82, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 61, + "minimum_elapsed_ms": 4, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 82, + "minimum_elapsed_ms": 8, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 61, + "minimum_elapsed_ms": 3, + "model": "q008-messages", + "p50_elapsed_ms": 14, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 5, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 4, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 6, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-q5ha5stt/config.toml Server: 127.0.0.1:36943 Accounts: 1 Database: Content digest: e296418ee9f663834b49bc8efdcdddba6ed3f459c323c2e147b1e22a27c79298", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 96, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 41, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 11, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 16, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2518153, \"ppid\": 2518131, \"uptime_seconds\": 2.8199773219999997, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\"", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 445, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-q5ha5stt/backups/eggpool-backup-20261016-051145.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2518153)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 100, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-q5ha5stt/recovery/config.toml db: /tmp/eggpool-q008-q5ha5stt/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 59, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2519552)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.3, + "avg300": 4.05, + "avg60": 1.91 + }, + "some": { + "avg10": 0.3, + "avg300": 4.09, + "avg60": 1.91 + } + }, + "root_device_read_bytes": 757993295872, + "root_device_write_bytes": 1348462076928 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 4, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 4 + }, + { + "body_bytes": 309, + "elapsed_ms": 3, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 3 + }, + { + "body_bytes": 213, + "elapsed_ms": 4, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 4 + }, + { + "body_bytes": 145, + "elapsed_ms": 15, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 13 + }, + { + "body_bytes": 270, + "elapsed_ms": 19, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 16 + }, + { + "body_bytes": 480, + "elapsed_ms": 60, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 22003712, + 22003712, + 23830528, + 23830528, + 23957504, + 24080384, + 24330240, + 46735360, + 46735360, + 46735360, + 23986176 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 4, + 3, + 4 + ], + "finite_ttft_ms": [ + 4, + 3, + 4 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 15, + 19, + 60 + ], + "streaming_ttft_ms": [ + 13, + 16, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 22003712, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 22003712, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 22003712, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 22003712, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23830528, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23830528, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23830528, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23830528, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23957504, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23957504, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172961792, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24080384, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24080384, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24330240, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24330240, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173096960, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4636672, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46735360, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46735360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173658112, + "wal_bytes": 4412552, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4636672, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46735360, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46735360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173658112, + "wal_bytes": 4412552, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4636672, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46735360, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46735360, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173658112, + "wal_bytes": 4412552, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23986176, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23986176, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103653376, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 21, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c4-run-2.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c4-run-2.json new file mode 100644 index 00000000..ced35370 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c4-run-2.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "e0f065dce3c260a78509567af1a011d3be39cc389c88b3042ff123814bd48212", + "archive_size_bytes": 4668887, + "elapsed_ms": 446, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 4, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23703552, + "baseline_rss_bytes": 23703552, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46641152, + "peak_rss_bytes": 46641152 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 5250, + "client_concurrency": 4, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 66.1, + "cpu_ticks": 347, + "cpu_time_ms": 3470 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 2.34, + "avg300": 3.86, + "avg60": 1.98 + }, + "some": { + "avg10": 2.34, + "avg300": 3.9, + "avg60": 1.98 + } + }, + "io_pressure_before": { + "full": { + "avg10": 1.97, + "avg300": 3.88, + "avg60": 1.91 + }, + "some": { + "avg10": 1.97, + "avg300": 3.91, + "avg60": 1.91 + } + }, + "root_device_read_delta_bytes": 90112, + "root_device_write_delta_bytes": 387727360 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 190.485, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 103, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 15, + "p50_ttft_ms": 15, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 60, + "p99_elapsed_ms": 75, + "p99_ttft_ms": 75, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 7, + "model": "q008-responses", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 19, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 90, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 13, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 14, + "p50_ttft_ms": 7, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-rf25uoba/config.toml Server: 127.0.0.1:58151 Accounts: 1 Database: Content digest: 2601433867d63ae6edccff6460d5e8c9e313d59e8ee9dae2807915840ff991d4", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 85, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 10, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 9, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 6, \"native_count\": 6, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2519582, \"ppid\": 2519559, \"uptime_seconds\": 2.80036872, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, ", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 7, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 446, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-rf25uoba/backups/eggpool-backup-20261016-051202.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 55, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2519582)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 99, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-rf25uoba/recovery/config.toml db: /tmp/eggpool-q008-rf25uoba/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 48, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2520753)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 1.2, + "avg300": 3.9, + "avg60": 1.76 + }, + "some": { + "avg10": 1.2, + "avg300": 3.94, + "avg60": 1.77 + } + }, + "root_device_read_bytes": 758091710464, + "root_device_write_bytes": 1348935328768 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 15, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 13 + }, + { + "body_bytes": 270, + "elapsed_ms": 17, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 16 + }, + { + "body_bytes": 480, + "elapsed_ms": 54, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 52 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21876736, + 21876736, + 23703552, + 23703552, + 23822336, + 23945216, + 24137728, + 46641152, + 46641152, + 46641152, + 23945216 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 16, + 12 + ], + "second_workload_ttft_ms": [ + 16, + 12 + ], + "streaming_elapsed_ms": [ + 15, + 17, + 54 + ], + "streaming_ttft_ms": [ + 13, + 16, + 52 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21876736, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21876736, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21876736, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21876736, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23703552, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23703552, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23703552, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23703552, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23822336, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23822336, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 4.99, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23945216, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23945216, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24137728, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24137728, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173113344, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4653056, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46641152, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46641152, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173670400, + "wal_bytes": 4326032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4653056, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46641152, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46641152, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173670400, + "wal_bytes": 4326032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4653056, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46641152, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46641152, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173670400, + "wal_bytes": 4326032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23945216, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23945216, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 20, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c4-run-3.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c4-run-3.json new file mode 100644 index 00000000..d3ae4e20 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c4-run-3.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "bd63feba8a955d5bd10f5fa9e72be719da1d009faa656d3ac52c78e73ad6dcf3", + "archive_size_bytes": 4664791, + "elapsed_ms": 463, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 4, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23711744, + "baseline_rss_bytes": 23711744, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 46751744, + "peak_rss_bytes": 46751744 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 9319, + "client_concurrency": 4, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 43.24, + "cpu_ticks": 403, + "cpu_time_ms": 4030 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 8.1, + "avg300": 4.04, + "avg60": 3.42 + }, + "some": { + "avg10": 8.25, + "avg300": 4.08, + "avg60": 3.48 + } + }, + "io_pressure_before": { + "full": { + "avg10": 1.45, + "avg300": 3.75, + "avg60": 1.8 + }, + "some": { + "avg10": 1.63, + "avg300": 3.79, + "avg60": 1.84 + } + }, + "root_device_read_delta_bytes": 495616, + "root_device_write_delta_bytes": 389214208 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 107.308, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 383, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 16, + "p50_ttft_ms": 16, + "p95_elapsed_ms": 144, + "p95_ttft_ms": 144, + "p99_elapsed_ms": 193, + "p99_ttft_ms": 193, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 106, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 14, + "p50_ttft_ms": 14, + "p95_elapsed_ms": 94, + "p95_ttft_ms": 94, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 168, + "minimum_elapsed_ms": 5, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 138, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 112, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 8, + "p95_elapsed_ms": 89, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-9bmian41/config.toml Server: 127.0.0.1:42123 Accounts: 1 Database: Content digest: b433fcd59d907d774f645a5656b0e5a4bdf267deefab6b1201d567aa42024102", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 87, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 38, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 5, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2520784, \"ppid\": 2520761, \"uptime_seconds\": 2.793116595, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 463, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-9bmian41/backups/eggpool-backup-20261016-051223.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2520784)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 131, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-9bmian41/recovery/config.toml db: /tmp/eggpool-q008-9bmian41/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 52, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 54, + "id": "stop", + "reason": "Stopping server (PID 2522024)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 1.65, + "avg300": 3.8, + "avg60": 1.88 + }, + "some": { + "avg10": 1.65, + "avg300": 3.84, + "avg60": 1.89 + } + }, + "root_device_read_bytes": 758092091392, + "root_device_write_bytes": 1349406036992 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 56, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 53 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21819392, + 21819392, + 23711744, + 23711744, + 23842816, + 23986176, + 24141824, + 46620672, + 46751744, + 46751744, + 24092672 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 17, + 4 + ], + "second_workload_ttft_ms": [ + 17, + 4 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 56 + ], + "streaming_ttft_ms": [ + 12, + 12, + 53 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21819392, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21819392, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21819392, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21819392, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23711744, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23711744, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172978176, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23711744, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23711744, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172978176, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23842816, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23842816, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172978176, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 23986176, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23986176, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172978176, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24141824, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24141824, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173113344, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 46620672, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46620672, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173694976, + "wal_bytes": 4412552, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 46751744, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46751744, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173682688, + "wal_bytes": 4412552, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4644864, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 46751744, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 46751744, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 173682688, + "wal_bytes": 4412552, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 24092672, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24092672, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 21, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c8-run-1.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c8-run-1.json new file mode 100644 index 00000000..e955dc16 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c8-run-1.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "a42510754aefaac69b0be1015728a4fd47c4a8ab6aecc4180b44a6e37e38cb05", + "archive_size_bytes": 4705751, + "elapsed_ms": 445, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 8, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23879680, + "baseline_rss_bytes": 23879680, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 47079424, + "peak_rss_bytes": 47079424 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 3014, + "client_concurrency": 8, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 66.02, + "cpu_ticks": 199, + "cpu_time_ms": 1990 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 1.32, + "avg300": 3.43, + "avg60": 1.73 + }, + "some": { + "avg10": 1.32, + "avg300": 3.46, + "avg60": 1.75 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.06, + "avg300": 3.4, + "avg60": 1.55 + }, + "some": { + "avg10": 0.06, + "avg300": 3.44, + "avg60": 1.57 + } + }, + "root_device_read_delta_bytes": 36864, + "root_device_write_delta_bytes": 389931008 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 331.769, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 86, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 19, + "p50_ttft_ms": 19, + "p95_elapsed_ms": 69, + "p95_ttft_ms": 69, + "p99_elapsed_ms": 76, + "p99_ttft_ms": 76, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 59, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 19, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 82, + "minimum_elapsed_ms": 6, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 2, + "model": "q008-messages", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-p5_z2s5k/config.toml Server: 127.0.0.1:49113 Accounts: 1 Database: Content digest: 66034df5956dba765fe1ed5f0e1b81d5f75cac40d95d4702e211b0d8fdd5062c", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 97, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 2, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 4, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 2, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2523481, \"ppid\": 2523459, \"uptime_seconds\": 2.776561356, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 3, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 445, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-p5_z2s5k/backups/eggpool-backup-20261016-051321.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 58, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2523481)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 92, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-p5_z2s5k/recovery/config.toml db: /tmp/eggpool-q008-p5_z2s5k/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 48, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 61, + "id": "stop", + "reason": "Stopping server (PID 2524704)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 0.15, + "avg300": 3.5, + "avg60": 1.77 + }, + "some": { + "avg10": 0.15, + "avg300": 3.54, + "avg60": 1.8 + } + }, + "root_device_read_bytes": 758101467136, + "root_device_write_bytes": 1350332003328 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 12, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 480, + "elapsed_ms": 52, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 52 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21860352, + 21860352, + 23879680, + 23879680, + 24006656, + 24137728, + 24317952, + 47079424, + 47079424, + 47079424, + 24055808 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 10, + 2 + ], + "second_workload_ttft_ms": [ + 10, + 2 + ], + "streaming_elapsed_ms": [ + 12, + 12, + 52 + ], + "streaming_ttft_ms": [ + 12, + 12, + 52 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21860352, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21860352, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21860352, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21860352, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23879680, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23879680, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23879680, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23879680, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 24006656, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24006656, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24137728, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24137728, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24317952, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24317952, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173109248, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4689920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 47079424, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47079424, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174084096, + "wal_bytes": 4371352, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4689920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 47079424, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47079424, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174084096, + "wal_bytes": 4371352, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4689920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 47079424, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47079424, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174084096, + "wal_bytes": 4371352, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 24055808, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24055808, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 20, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c8-run-2.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c8-run-2.json new file mode 100644 index 00000000..03d2ec40 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c8-run-2.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "61cfef2c3c952d9471399efc0a7dfa7208038603daaa018e83bad0f0403fd1ad", + "archive_size_bytes": 4697559, + "elapsed_ms": 661, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 8, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23818240, + "baseline_rss_bytes": 23818240, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 47181824, + "peak_rss_bytes": 47181824 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 3387, + "client_concurrency": 8, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 75.28, + "cpu_ticks": 255, + "cpu_time_ms": 2550 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 2.04, + "avg300": 3.34, + "avg60": 1.75 + }, + "some": { + "avg10": 2.04, + "avg300": 3.37, + "avg60": 1.76 + } + }, + "io_pressure_before": { + "full": { + "avg10": 0.76, + "avg300": 3.32, + "avg60": 1.55 + }, + "some": { + "avg10": 0.76, + "avg300": 3.36, + "avg60": 1.57 + } + }, + "root_device_read_delta_bytes": 98304, + "root_device_write_delta_bytes": 391954432 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 295.215, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 1021, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 22, + "p50_ttft_ms": 22, + "p95_elapsed_ms": 73, + "p95_ttft_ms": 73, + "p99_elapsed_ms": 85, + "p99_ttft_ms": 85, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 19, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 59, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 78, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 14, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 59, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 60, + "minimum_elapsed_ms": 9, + "model": "q008-messages", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 11, + "p95_elapsed_ms": 60, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-vh3nhw54/config.toml Server: 127.0.0.1:50997 Accounts: 1 Database: Content digest: 30150c120712ababa768bd953d553d304a48657b2b2d05947ac7643dcb6c4fe8", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 100, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 39, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2524733, \"ppid\": 2524711, \"uptime_seconds\": 2.840059037, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 7, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 661, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-vh3nhw54/backups/eggpool-backup-20261016-051336.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 59, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2524733)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 765, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-vh3nhw54/recovery/config.toml db: /tmp/eggpool-q008-vh3nhw54/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 211, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 111, + "id": "stop", + "reason": "Stopping server (PID 2525921)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 1.39, + "avg300": 3.39, + "avg60": 1.72 + }, + "some": { + "avg10": 1.39, + "avg300": 3.43, + "avg60": 1.74 + } + }, + "root_device_read_bytes": 758102040576, + "root_device_write_bytes": 1350804214784 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 11, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 11 + }, + { + "body_bytes": 309, + "elapsed_ms": 10, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 9 + }, + { + "body_bytes": 213, + "elapsed_ms": 10, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 10 + }, + { + "body_bytes": 145, + "elapsed_ms": 20, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 270, + "elapsed_ms": 20, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 17 + }, + { + "body_bytes": 480, + "elapsed_ms": 60, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21913600, + 21913600, + 23818240, + 23818240, + 23928832, + 24051712, + 24231936, + 47181824, + 47181824, + 47181824, + 23588864 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 3, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 11, + 10, + 10 + ], + "finite_ttft_ms": [ + 11, + 9, + 10 + ], + "second_workload_elapsed_ms": [ + 5, + 2 + ], + "second_workload_ttft_ms": [ + 5, + 2 + ], + "streaming_elapsed_ms": [ + 20, + 20, + 60 + ], + "streaming_ttft_ms": [ + 17, + 17, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21913600, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21913600, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21913600, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21913600, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103616512, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23818240, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23818240, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23818240, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23818240, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23928832, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23928832, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24051712, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24051712, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172969984, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24231936, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24231936, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173105152, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4694016, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 47181824, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47181824, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174075904, + "wal_bytes": 4433152, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4694016, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 47181824, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47181824, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174075904, + "wal_bytes": 4433152, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4694016, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 47181824, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47181824, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 174075904, + "wal_bytes": 4433152, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23588864, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23588864, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 906432, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 21, + "status": "pass" +} diff --git a/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c8-run-3.json b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c8-run-3.json new file mode 100644 index 00000000..77aac191 --- /dev/null +++ b/artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/u5-c8-run-3.json @@ -0,0 +1,1083 @@ +{ + "backup": { + "archive_sha256": "a08ec510d9062c12b725c641cde85994682125921bc97b51ff99b5c3b572d85f", + "archive_size_bytes": 4677079, + "elapsed_ms": 634, + "member_count": 3 + }, + "benchmark": { + "budget_mode": "user-global", + "cadence": { + "backup_enabled": "false", + "backup_interval_s": "86400", + "database_busy_timeout_ms": "10000", + "database_synchronous": "\"NORMAL\"", + "database_wal": "true", + "database_worker_threads": "1", + "metrics_aggregate_only": "true", + "metrics_cleanup_interval_s": "86400", + "metrics_event_loop_lag_enabled": "false", + "metrics_flush_interval_s": "120", + "metrics_max_buffered_events": "250", + "metrics_write_mode": "\"low_wear\"", + "model_info_enabled": "false", + "model_info_startup_refresh": "false", + "models_refresh_interval_s": "300", + "models_startup_refresh": "true", + "routing_trace_mode": "\"off\"", + "routing_trace_sample_rate": "0.0", + "server_access_log": "false", + "server_threads": "1" + }, + "client_concurrency": 8, + "concurrency_batch_size": 1000, + "config_fixture": "sbc-benchmark.toml", + "database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "interpretation": "descriptive loopback characterization; no performance threshold applied", + "principal_count": 5, + "provider_latency_profile": "mixed-0-10-50ms", + "resource_summary": { + "baseline_peak_rss_bytes": 23875584, + "baseline_rss_bytes": 23875584, + "final_active_reservations": 0, + "final_finalization_jobs": null, + "final_pending_requests": 0, + "final_rss_bytes": 47144960, + "peak_rss_bytes": 47144960 + }, + "runs": { + "concurrent_native_responses_finite": { + "batch_elapsed_ms": 12749, + "client_concurrency": 8, + "competing_io": { + "bytes_written": 0, + "configured_mib_per_second": 0, + "elapsed_ms": 0, + "enabled": false, + "maximum_mib": 0, + "storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + } + }, + "completed_count": 1000, + "cpu": { + "cpu_percent": 42.12, + "cpu_ticks": 537, + "cpu_time_ms": 5370 + }, + "expected_rate_limited_count": null, + "expected_throttle_pass": false, + "failed_count": 0, + "host_io": { + "io_pressure_after": { + "full": { + "avg10": 13.03, + "avg300": 3.87, + "avg60": 4.68 + }, + "some": { + "avg10": 13.27, + "avg300": 3.92, + "avg60": 4.78 + } + }, + "io_pressure_before": { + "full": { + "avg10": 2.99, + "avg300": 3.34, + "avg60": 2.04 + }, + "some": { + "avg10": 3.09, + "avg300": 3.38, + "avg60": 2.08 + } + }, + "root_device_read_delta_bytes": 323584, + "root_device_write_delta_bytes": 387227648 + }, + "http_status_counts": { + "200": 1000 + }, + "rate_limited_count": 0, + "request_count": 1000, + "requests_per_second": 78.44, + "status": "measured", + "timings": { + "maximum_elapsed_ms": 436, + "minimum_elapsed_ms": 2, + "p50_elapsed_ms": 119, + "p50_ttft_ms": 119, + "p95_elapsed_ms": 198, + "p95_ttft_ms": 198, + "p99_elapsed_ms": 300, + "p99_ttft_ms": 299, + "sample_count": 1000 + } + }, + "native_responses_finite": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 117, + "minimum_elapsed_ms": 2, + "model": "q008-responses", + "p50_elapsed_ms": 12, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 91, + "p95_ttft_ms": 91, + "sample_count": 30, + "status": "measured", + "streaming": false, + "warmup_count": 5 + }, + "native_responses_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "maximum_elapsed_ms": 162, + "minimum_elapsed_ms": 9, + "model": "q008-responses", + "p50_elapsed_ms": 14, + "p50_ttft_ms": 12, + "p95_elapsed_ms": 136, + "p95_ttft_ms": 57, + "sample_count": 30, + "status": "measured", + "streaming": true, + "warmup_count": 5 + }, + "translated_responses_to_messages_streaming": { + "cpu": { + "cpu_percent": null, + "cpu_ticks": null, + "cpu_time_ms": null + }, + "expected_upstream_path": "/messages", + "maximum_elapsed_ms": 117, + "minimum_elapsed_ms": 3, + "model": "q008-messages", + "p50_elapsed_ms": 19, + "p50_ttft_ms": 13, + "p95_elapsed_ms": 95, + "p95_ttft_ms": 58, + "sample_count": 30, + "status": "measured", + "streaming": true, + "upstream_proof": "fixture Messages path observed", + "upstream_request_delta": 35, + "warmup_count": 5 + } + }, + "sample_count": 30 + }, + "candidate": { + "binary_size_bytes": 39042128, + "build_mode": "on-device-release-build", + "build_profile": "release", + "repository_commit_sha": "f08a760e2cd995ac9b3416b571d092e3315b6524", + "rust_toolchain": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "sha256": "d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73" + }, + "commands": [ + { + "command": [ + "", + "--config", + "/config.toml", + "version" + ], + "duration_ms": 2, + "id": "version", + "reason": "0.8.2", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "--help" + ], + "duration_ms": 2, + "id": "help", + "reason": "EggPool - aggregate OpenCode Go subscriptions. Usage: eggpool [OPTIONS] [COMMAND] Commands: serve connect logout check-config edit getkey newkey configsetup configremote deploy accounts dashboard db models modelinfo stats users onboard croncheck ensure-running migrate stop restart init-config help recover uninstall update set rehash runtime-status status backup version Options: --config Path to the TOML configuration file. Falls back to $EGGPOOL_CONFIG, then ~/.config/eggpool/config.toml, then ./config.toml. -h, --help Print help -V, --version Print version", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "check-config" + ], + "duration_ms": 2, + "id": "check-config", + "reason": "Configuration loaded successfully from /tmp/eggpool-q008-msd1fx7z/config.toml Server: 127.0.0.1:47051 Accounts: 1 Database: Content digest: bd782022cdbd622c710073f50d39dcabbbafda077c67ef4350f260e05650908b", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "migrate" + ], + "duration_ms": 137, + "id": "migrate", + "reason": "Migrations completed successfully Applied migrations: 63 Current schema: 63", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "users", + "bootstrap" + ], + "duration_ms": 55, + "id": "bootstrap-admin", + "reason": "synthetic administrator bootstrap complete", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "accounts", + "list" + ], + "duration_ms": 7, + "id": "accounts-list", + "reason": "Configured accounts: fixture/q008-account Total: 1 accounts", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "modelinfo", + "list" + ], + "duration_ms": 4, + "id": "modelinfo-list", + "reason": "No model-info rows found.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stats", + "explain-dashboard", + "--json" + ], + "duration_ms": 4, + "id": "operator-stats", + "reason": "{ \"period\": \"24h\", \"bucket\": \"hour\", \"group_by\": \"provider_model\", \"queries\": [ { \"name\": \"fetch_timeseries\", \"plan\": [ \"SEARCH r USING COVERING INDEX idx_requests_started (started_at>? AND started_at? AND started_at? AND started_at? AND started_at", + "--config", + "/config.toml", + "stats", + "transcoding", + "--json" + ], + "duration_ms": 5, + "id": "transcoding-stats", + "reason": "{ \"total\": 0, \"native_count\": 0, \"transcoded_count\": 0, \"per_direction\": {}, \"top_loss_warnings\": [], \"period\": \"24h\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "runtime-status", + "--json" + ], + "duration_ms": 3, + "id": "runtime-status", + "reason": "{ \"status\": \"ok\", \"degraded\": false, \"server\": { \"pid\": 2525968, \"ppid\": 2525946, \"uptime_seconds\": 2.796688937, \"configured_server_threads\": 1, \"python_version\": null, \"rust_version\": \"0.8.2\" }, \"memory\": { \"rss_bytes\": null, \"vms_bytes\": null, \"open_fd_count\": null, \"thread_count\": 1 }, \"processes\": { \"eggpool_process_count\": 1, \"expected_worker_process_count\": 1, \"process_count_warning\": false }, \"background_tasks\": [ { \"name\": \"catalog_refresh\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null, \"in_tick\": false }, { \"name\": \"checkpoint\", \"running\": true, \"enabled\": true, \"tick_count\": 1, \"last_outcome\": \"success\", \"in_tick\": false }, { \"name\": \"metrics_flush\", \"running\": true, \"enabled\": true, \"tick_count\": 0, \"last_outcome\": null,", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "rehash", + "--json" + ], + "duration_ms": 5, + "id": "rehash", + "reason": "{ \"ok\": true, \"stage\": \"commit\", \"exit_code\": 0, \"generation\": 1, \"changed_sections\": [], \"warnings\": [], \"restart_required\": [], \"retirement_pending\": false, \"message\": \"no changes\" }", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "backup", + "--output-dir", + "/backups" + ], + "duration_ms": 634, + "id": "backup", + "reason": "Wrote backup: /tmp/eggpool-q008-msd1fx7z/backups/eggpool-backup-20261016-051402.zip included: config.toml included: usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 163, + "id": "stop-before-recovery", + "reason": "Stopping server (PID 2525968)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/recovery/config.toml", + "recover", + "/recovery/recovered.zip" + ], + "duration_ms": 592, + "id": "recover", + "reason": "Overwrite current configuration and database? [y/N] Restore complete. Restart the server to load the new config. config: /tmp/eggpool-q008-msd1fx7z/recovery/config.toml db: /tmp/eggpool-q008-msd1fx7z/recovery/usage.sqlite3", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "db", + "vacuum" + ], + "duration_ms": 236, + "id": "vacuum", + "reason": "Database vacuum completed successfully", + "returncode": 0, + "status": "pass", + "timed_out": false + }, + { + "command": [ + "", + "--config", + "/config.toml", + "stop", + "--timeout", + "30" + ], + "duration_ms": 57, + "id": "stop", + "reason": "Stopping server (PID 2527211)... Server stopped.", + "returncode": 0, + "status": "pass", + "timed_out": false + } + ], + "durable": { + "active_reservations": 0, + "attempts": 1113, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113 + }, + "environment": { + "architecture": "x86_64", + "attestation": "Linux host metadata from procfs, sysfs, DMI and systemd-detect-virt", + "board_name": "Z390 AORUS MASTER-CF", + "container_type": "none", + "cpu_frequency_policy_end": null, + "cpu_governor_end": null, + "cpu_model": "Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz", + "kernel": "6.8.0-147-generic", + "logical_cpu_count": 16, + "loopback_provider_path_counts": { + "/chat/completions": 3, + "/messages": 37, + "/responses": 1073 + }, + "loopback_provider_requests": 1113, + "os": "Ubuntu 24.04.5 LTS", + "os_version_id": "24.04", + "power_thermal_mode_celsius_end": null, + "preflight_io": { + "io_pressure": { + "full": { + "avg10": 1.55, + "avg300": 3.3, + "avg60": 1.66 + }, + "some": { + "avg10": 1.55, + "avg300": 3.33, + "avg60": 1.68 + } + }, + "root_device_read_bytes": 758128746496, + "root_device_write_bytes": 1351273645056 + }, + "product_name": "Z390 AORUS MASTER", + "qualification_database_storage": { + "device_class": "nvme", + "filesystem": "ext4", + "storage_medium_class": "non-rotational" + }, + "ram_bytes": 16622407680, + "root_filesystem": "ext4", + "root_mount_options": "rw,relatime,stripe=32", + "root_storage_device_class": "nvme", + "root_storage_device_model": "Samsung SSD 980 1TB", + "root_storage_medium_class": "non-rotational", + "system_vendor": "Gigabyte Technology Co., Ltd.", + "target_class": "ordinary-server", + "virtualization_type": "none" + }, + "findings": [], + "functional": [ + { + "body_bytes": 15, + "http_status": 200, + "id": "startup-readyz", + "status": "pass" + }, + { + "body_bytes": 272, + "http_status": 200, + "id": "model-listing", + "status": "pass" + }, + { + "body_bytes": 225, + "elapsed_ms": 2, + "http_status": 200, + "id": "chat_completions-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 309, + "elapsed_ms": 2, + "http_status": 200, + "id": "responses-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 213, + "elapsed_ms": 2, + "http_status": 200, + "id": "messages-finite", + "status": "pass", + "ttft_ms": 2 + }, + { + "body_bytes": 145, + "elapsed_ms": 12, + "http_status": 200, + "id": "chat_completions-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 12 + }, + { + "body_bytes": 270, + "elapsed_ms": 15, + "http_status": 200, + "id": "responses-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 13 + }, + { + "body_bytes": 480, + "elapsed_ms": 60, + "http_status": 200, + "id": "messages-stream", + "status": "pass", + "terminal_evidence": true, + "ttft_ms": 57 + }, + { + "id": "accounts-list", + "status": "pass" + }, + { + "id": "modelinfo-list", + "status": "pass" + }, + { + "id": "operator-stats", + "status": "pass" + }, + { + "id": "transcoding-stats", + "status": "pass" + }, + { + "id": "runtime-status", + "status": "pass" + }, + { + "body_bytes": 29, + "http_status": 401, + "id": "dashboard-page", + "note": "managed-fencing: dashboard shell requires browser session", + "status": "pass" + }, + { + "body_bytes": 56004, + "http_status": 200, + "id": "dashboard-static-dashboard.css", + "status": "pass" + }, + { + "body_bytes": 49083, + "http_status": 200, + "id": "dashboard-static-dashboard.js", + "status": "pass" + }, + { + "body_bytes": 131072, + "http_status": 200, + "id": "dashboard-static-chart.js", + "status": "pass" + }, + { + "body_bytes": 1203, + "http_status": 200, + "id": "dashboard-static-favicon.svg", + "status": "pass" + }, + { + "body_bytes": 1264, + "http_status": 200, + "id": "dashboard-static-theme.css?theme=Nord", + "status": "pass" + }, + { + "id": "rehash-runtime-status", + "status": "pass" + }, + { + "id": "graceful-stop-before-recovery", + "status": "pass" + }, + { + "id": "backup-recover-isolated", + "recovered_database": true, + "status": "pass" + }, + { + "id": "bounded-maintenance", + "status": "pass" + }, + { + "id": "restart-reconcile", + "status": "pass" + }, + { + "id": "graceful-shutdown", + "status": "pass" + } + ], + "isolated_temporary_root": true, + "manifest": "runtime-q001.v1", + "plan": "ordinary-server qualification", + "python_reference": { + "reason": "optional same-board comparison was not supplied", + "status": "not-run" + }, + "repeated_run_stability": { + "fd_counts": [ + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14, + 14 + ], + "interpretation": "bounded samples for release characterization; no fixed performance threshold applied", + "logical_leaks": false, + "rss_bytes": [ + 21839872, + 21839872, + 23875584, + 23875584, + 23977984, + 24133632, + 24285184, + 47144960, + 47144960, + 47144960, + 24055808 + ], + "samples": 11, + "status": "pass", + "thread_counts": [ + 2, + 2, + 3, + 3, + 3, + 3, + 3, + 2, + 2, + 2, + 2 + ] + }, + "request_workload_timings": { + "elapsed_ms": { + "finite_elapsed_ms": [ + 2, + 2, + 2 + ], + "finite_ttft_ms": [ + 2, + 2, + 2 + ], + "second_workload_elapsed_ms": [ + 10, + 2 + ], + "second_workload_ttft_ms": [ + 10, + 2 + ], + "streaming_elapsed_ms": [ + 12, + 15, + 60 + ], + "streaming_ttft_ms": [ + 12, + 13, + 57 + ] + }, + "interpretation": "bounded client-observed timings; first-byte values are diagnostic characterization, not an SLA", + "sample_count": 16 + }, + "resource_samples": [ + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "before-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21839872, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21839872, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 0, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-warmup", + "open_fd_count": 14, + "peak_rss_bytes": 21839872, + "pending_requests": 0, + "request_statuses": {}, + "requests": 0, + "reservations": 0, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 21839872, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103620608, + "wal_bytes": 103032, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-first-workload", + "open_fd_count": 14, + "peak_rss_bytes": 23875584, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23875584, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 6, + "cpu_percent": 0.0, + "db_bytes": 831488, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "benchmark-baseline", + "open_fd_count": 14, + "peak_rss_bytes": 23875584, + "pending_requests": 0, + "request_statuses": { + "completed": 6 + }, + "requests": 6, + "reservations": 6, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23875584, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 1672752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 41, + "cpu_percent": 0.0, + "db_bytes": 884736, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-finite", + "open_fd_count": 14, + "peak_rss_bytes": 23977984, + "pending_requests": 0, + "request_statuses": { + "completed": 41 + }, + "requests": 41, + "reservations": 41, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 23977984, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4144752, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 76, + "cpu_percent": 0.0, + "db_bytes": 958464, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-native-responses-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24133632, + "pending_requests": 0, + "request_statuses": { + "completed": 76 + }, + "requests": 76, + "reservations": 76, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24133632, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 172974080, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 111, + "cpu_percent": 0.0, + "db_bytes": 1105920, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-translated-streaming", + "open_fd_count": 14, + "peak_rss_bytes": 24285184, + "pending_requests": 0, + "request_statuses": { + "completed": 111 + }, + "requests": 111, + "reservations": 111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24285184, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 3, + "virtual_bytes": 173113344, + "wal_bytes": 4185952, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4653056, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-concurrency-4", + "open_fd_count": 14, + "peak_rss_bytes": 47144960, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47144960, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174075904, + "wal_bytes": 4383712, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4653056, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-benchmark-stabilization", + "open_fd_count": 14, + "peak_rss_bytes": 47144960, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47144960, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 3000, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174075904, + "wal_bytes": 4383712, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1111, + "cpu_percent": 0.0, + "db_bytes": 4653056, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-rehash-backup", + "open_fd_count": 14, + "peak_rss_bytes": 47144960, + "pending_requests": 0, + "request_statuses": { + "completed": 1111 + }, + "requests": 1111, + "reservations": 1111, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 47144960, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 174075904, + "wal_bytes": 4383712, + "wire_flights": null + }, + { + "active_generation_id": null, + "active_leases": null, + "active_reservations": 0, + "attempts": 1113, + "cpu_percent": 0.0, + "db_bytes": 4464640, + "finalization_capacity": null, + "finalization_jobs": null, + "generation_count": null, + "label": "after-second-workload", + "open_fd_count": 14, + "peak_rss_bytes": 24055808, + "pending_requests": 0, + "request_statuses": { + "completed": 1113 + }, + "requests": 1113, + "reservations": 1113, + "retiring_generations": null, + "retiring_leases": 0, + "rss_bytes": 24055808, + "running_tasks": 0, + "runtime_read_error": false, + "runtime_tasks": 0, + "sample_window_ms": 200, + "terminal_references": 0, + "thread_count": 2, + "virtual_bytes": 103649280, + "wal_bytes": 885832, + "wire_flights": null + } + ], + "schema_version": "runtime-q008.v2", + "startup_to_ready_ms": 21, + "status": "pass" +} diff --git a/plans/closure/on-prem-user-access/023-performance-qualification.md b/plans/closure/on-prem-user-access/023-performance-qualification.md new file mode 100644 index 00000000..2df7333d --- /dev/null +++ b/plans/closure/on-prem-user-access/023-performance-qualification.md @@ -0,0 +1,134 @@ +# On-Prem User Access M023 — Ordinary Linux Performance Qualification + +Status: closed + +Source implementation plan: + +- `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` + +Source subsystem roadmap: + +- `plans/subsystems/on-prem-user-access-roadmap.md#m023--short-window-team-fairness-controls` + +Previous implementation closure: + +- `plans/closure/on-prem-user-access/023-status.md` (conditionally closed; retained as the implementation and local-verification record) + +Repository baseline reviewed: `83c0779ae4776a6b9d4d82571c7255bcb71246d6`. + +Runtime candidate: release binary built from `f08a760e2cd995ac9b3416b571d092e3315b6524`; SHA-256 `d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73`. The later `83c0779a` commit changes qualification tooling only; the runtime source and candidate binary are unchanged. + +Implementation and qualification commits: + +- `84482e91b84fe7424144dcf916441479c5b492d1` — bounded short-window admission fairness. +- `f08a760e2cd995ac9b3416b571d092e3315b6524` — avoid legacy shared-key creation after administrator initialization. +- `83c0779ae4776a6b9d4d82571c7255bcb71246d6` — add measured M023 throttle mode to the ordinary-server qualification runner. + +## 1. Executive finding + +The remaining M023 operational condition is satisfied. The modified release binary was measured on the same non-virtualized Ubuntu 24.04.5 / ext4 / Samsung SSD 980 NVMe ordinary-server target used for M002. The no-rule matrix completed all 27 requested runs. Three additional single-concurrency controls repeat the 1-, 5-, and 49-principal cases. Three active-policy runs each sent 1,000 requests at concurrency 16 with five fresh principals and a 1-request/minute rule: each admitted exactly five requests and returned the distinct short-window 429 for the other 995. + +The no-rule p50 and p95 distributions remain close to the M002 comparison in the ordinary 5-principal concurrency cells. The controlled 49-principal case matches M002's p95/p99 (61/81 ms). Single-concurrency p99 varied more between repetitions; the raw reports and host I/O pressure are retained. No universal team capacity or latency guarantee is inferred. The medium finding recorded in the prior M023 closure is resolved by the named post-change evidence; no unresolved M023 finding remains. + +## 2. Requirement-to-evidence matrix + +| Requirement | Evidence | Result | Notes | +|---|---|---|---| +| Measure the modified binary with the rate limiter disabled | `artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit/matrix.json`, `summary.json`, and `comparison.json` | pass | 9 principal/concurrency cells × 3 repetitions; 1,000 requests per run; all 27 final reports pass and share the candidate SHA above. The fixture omits `[admission_fairness]`. | +| Compare the no-rule behavior to the pre-M023 M002 reference | `comparison.json`; M002 `plans/closure/on-prem-concurrency-performance/002-status.md` and its ordinary-server corpus | pass | Same host class, storage, provider profile, request count, and cell definitions. M002 candidate SHA is recorded in the comparison source summary. | +| Measure active throttling, admitted/rejected counts, and response latency | `artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/summary.json` plus three raw reports | pass | Three passing runs each record HTTP 200: 5, HTTP 429: 995, with `short_window_rate_limited`; median 1,091.728 requests/s, p50 13 ms, p95 29 ms, p99 113 ms. | +| Bound the request path and keep denials before upstream dispatch | Existing `server_transport::short_window_fairness_rejects_before_dispatch_and_reports_retry_scope`; active-throttle reports | pass | Integration test verifies no body collection/upstream dispatch for a denied request. Measured runs account for exactly five initially admitted requests per five fresh principal buckets. | +| Retain honest host and pressure evidence | Raw Q008 reports and qualification manifests | pass with environment note | Host: Intel i9-9900K, 16 logical CPUs, 15.5 GiB RAM, Ubuntu 24.04.5, non-virtualized, ext4 on Samsung SSD 980 NVMe. Two initial attempts were stopped by harness gates (one shutdown observation; one full I/O PSI avg10 at 28.33%); their cells were retried after the host settled. The blocked active attempt is retained as `run-3-blocked-attempt.json`. | + +No new capacity claim is made. The 49 named-principal result is metadata/concurrency characterization at the declared request counts, not a 49-simultaneous-inference promise. + +## 3. Production implementation evidence + +The production boundary remains `rust/src/request/admission_fairness.rs` and `rust/src/server/middleware.rs`. An absent policy returns before limiter locking or state allocation. Configured policy uses a process-local bounded principal table, a monotonic token bucket, pre-body denial, and a distinct 429 with `Retry-After`. The active measurement changes the live policy with `rehash` immediately before the measured burst so setup and warm-up traffic do not consume the fresh principal buckets. + +The onboarding corrective in `rust/src/runtime.rs` checks the database identity state before generating a legacy shared server key. Existing `admin_initialized` and `managed` installations no longer receive a new key during onboarding; uninitialized and legacy installations retain their compatibility path. + +M023 adds no database migration. Schema migration 0063 belongs to M021 and remains an additive nullable default-member-profile field with its checksum and runtime-manifest entry. + +## 4. Verification executed + +### Commands run + +```bash +rtk proxy cargo build --manifest-path rust/Cargo.toml --locked --release +rtk proxy cargo test --manifest-path rust/Cargo.toml --lib runtime::tests::onboarding_does_not_create_a_legacy_key_after_admin_initialization -- --test-threads=1 +rtk proxy cargo fmt --manifest-path rust/Cargo.toml --all -- --check +rtk proxy cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings +rtk proxy cargo check --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features +rtk proxy cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --no-default-features -- -D warnings +rtk proxy uv run ruff format --check scripts/ tests/tooling/ +rtk proxy uv run ruff check scripts/ tests/tooling/ +rtk proxy uv run pyright scripts/ +rtk proxy uv run pytest tests/tooling/test_qualification_sbc.py -q --tb=short --maxfail=1 +rtk proxy python scripts/qualification_ordinary_server.py --binary rust/target/release/eggpool --output-dir artifacts/qualification/on-prem-user-access/m023-ordinary-server-no-limit +rtk proxy python scripts/qualification_sbc.py --binary rust/target/release/eggpool --target-class ordinary-server --config-fixture tests/tooling/fixtures/qualification/sbc-benchmark.toml --benchmark-samples 1 --concurrency-workers 16 --concurrency-requests 1000 --benchmark-principals 5 --benchmark-budget-mode user-global --provider-latency-profile mixed-0-10-50ms --candidate-origin on-device-release-build --admission-fairness-rate-per-minute-per-principal 1 --output artifacts/qualification/on-prem-user-access/m023-ordinary-server-active-throttle/run-1.json +``` + +The active-throttle command was repeated for `run-2.json` and `run-3.json` with the same arguments. The no-limit controlled runs used the same Q008 arguments with concurrency 1 and principal counts 1, 5, and 49; three reports per cell are under `m023-ordinary-server-no-limit/controls/`. + +### Results + +- Locked release build passed. Candidate SHA-256: `d7bec9bf508e9b9ccfd1c9ac53ae25b1ef37a5af266f2d4b0cf048c2816f7b73`. +- Onboarding regression test: 1 passed. +- Default strict Clippy, no-default `check`, and no-default strict Clippy passed. +- Ruff format/check passed; Pyright reported 0 errors, warnings, or informations; focused tooling suite: 44 passed. +- The local full default workspace suite was stopped after six minutes when system I/O PSI exceeded 90% and the filesystem reached 99% used (13 GiB free). The pushed exact runtime/tooling head `83c0779a` has green hosted `check`, Linux installer, and macOS installer workflows. The hosted check is the full-suite gate for this candidate. +- Ordinary-server no-limit corpus: 9 cells, 3 passing runs per cell, 1,000 requests per run; every final run completed and settled durable request/reservation state. +- Active throttling: 3 passing runs; each completed all 1,000 client observations with five 200 responses and 995 correctly coded 429 responses. No expected denial was classified as a provider or budget failure. + +Selected same-host medians (M002 → modified binary, limiter disabled): + +| Principals / concurrency | p50 ms | p95 ms | p99 ms | requests/s | +|---|---:|---:|---:|---:| +| 1 / 1, controlled repeat | 15 → 13 | 82 → 96 | 148 → 164 | 36.495 → 35.987 | +| 5 / 1, controlled repeat | 15 → 15 | 61 → 62 | 89 → 123 | 42.179 → 38.753 | +| 49 / 1, controlled repeat | 15 → 14 | 61 → 61 | 81 → 81 | 42.322 → 43.498 | +| 5 / 2 | 14 → 15 | 61 → 62 | 79 → 82 | 93.976 → 92.816 | +| 5 / 4 | 14 → 15 | 59 → 60 | 75 → 82 | 193.683 → 190.485 | +| 5 / 8 | 19 → 22 | 69 → 73 | 76 → 85 | 335.268 → 295.215 | +| 5 / 16 | 35 → 38 | 88 → 91 | 106 → 119 | 341.347 → 326.388 | + +The p50/p95 shifts are small in absolute terms on these loopback synthetic-provider runs. Single-concurrency p99 is variable: the comparison artifact preserves each repetition and its pre/post I/O PSI; one 49-principal matrix run was blocked at preflight and retried after pressure fell. These descriptive measurements do not establish an SLA. + +## 5. Invariant review + +- No-rule requests allocate no bucket state and do not acquire the limiter mutex. +- Configured buckets remain principal-scoped, bounded, and process-local. User tokens, request bodies, prompts, IP addresses, and provider results are not retained. +- The throttle decision remains before body collection, durable budget admission, provider routing, and upstream submission. +- A denied request consumes neither another principal's token nor a durable request reservation. Existing daily/monthly user and instance budgets remain authoritative. +- The performance work changes no SQLite ownership, WAL/NORMAL settings, writer count, routing policy, generation lease, or streaming handoff. + +## 6. Failure and recovery review + +The existing functional tests cover simultaneous final-slot admission, retry timing, principal-table capacity, and pre-dispatch denial. The measured active batch confirms fail-closed throttling remains prompt under 16 concurrent clients. Ephemeral state still resets on restart; durable budget state is unaffected. Harness-blocked attempts collected no load observations and are not counted as passing runs. + +## 7. Migration and compatibility review + +M023 has no schema change. Missing `[admission_fairness]` retains disabled behavior. Rates are enabled through the existing live-reload policy; the active measurement installed the rule with a live `rehash`. Migration 0063 remains M021-owned and is checked by the existing schema inventory, checksum, and compatibility tests. + +## 8. Security review + +The limiter key remains the authenticated database-owned user ID, not a token, header, model, IP, or body value. The denial response reveals only its bounded scope and retry time. No new secret-bearing field or diagnostic was introduced. The onboarding correction does not mint a shared key for an installation whose administrator is already initialized. + +## 9. Documentation and operations + +Operator configuration and restart/reset semantics remain documented in the M023 implementation closure and deployment/model-limit docs. Raw run reports, comparison summary, and active-throttle summary are retained under `artifacts/qualification/on-prem-user-access/`. The qualification tooling describes the synthetic workload and rejects runs when Linux/SSD-or-NVMe or the predeclared I/O-pressure gate is not satisfied. + +## 10. Unresolved findings + +| Severity | Finding | Impact | Required action | +|---|---|---|---| +| — | No unresolved M023 performance or correctness finding | The post-change no-rule and active-throttle behavior is measured on the required ordinary Linux/NVMe class | None. Do not infer a universal latency or concurrency SLA from this host-specific synthetic workload. | + +## 11. Roadmap disposition + +M023 is closed. This follow-up resolves the sole medium finding in the earlier conditional implementation closure. All registered M017–M023 milestones are complete; the user-access roadmap has no remaining successor. Historical conditional findings are preserved in their original records and do not authorize broader capacity claims. No future plan was made ready by this closure because no registered plan depends on M023. + +## 12. Registry updates + +The roadmap and registry now point M023 at this performance-qualification closure and mark M023 complete. The earlier `023-status.md` remains unchanged as the implementation closure that recorded the missing operational evidence. The dependency audit found no M023 successor or downstream dependency; no blocked row is promoted and no other plan status changes. diff --git a/plans/registry.md b/plans/registry.md index 85b3fe5c..269a19be 100644 --- a/plans/registry.md +++ b/plans/registry.md @@ -58,7 +58,7 @@ No new runtime, schema or performance claim lands with these plans. | Persistence | active | `plans/subsystems/persistence-roadmap.md` | M011 ready — post-M010 roadmap/documentation reconciliation | M009/M010 closures accepted. Documentation/tooling-only corrective; production remains WAL/NORMAL. No technical persistence successor is registered. | | Deployment and packaging | closed | `plans/subsystems/deployment-packaging-roadmap.md` | M005 closed — active PATH integration detection corrective | M001–M005 closed; no successor registered. | | Dashboard | closed | `plans/subsystems/dashboard-roadmap.md` | M012 closed — lifecycle closure and documentation polish | M001–M012 closed; no registered successor; future dashboard work requires a new bounded plan. | -| On-prem user access | active | `plans/subsystems/on-prem-user-access-roadmap.md` | M023 conditionally closed — short-window fairness controls | M017–M022 closed; M023 production and local evidence complete. Post-change ordinary-Linux latency comparison remains the named condition. No successor is registered. | +| On-prem user access | closed | `plans/subsystems/on-prem-user-access-roadmap.md` | M023 closed — short-window fairness controls | M017–M023 complete. Modified-binary ordinary-Linux/NVMe no-rule and active-throttle qualification is recorded; no successor is registered. | | On-prem concurrency performance | active | `plans/subsystems/on-prem-concurrency-performance-roadmap.md` | M002 closed — ordinary-server synthetic baseline recorded | M001's Pi/microSD solo tail attribution stays conditionally closed; M002 authorizes no WAL/SQLite topology change. | ## Dependency-ready implementation plans @@ -108,7 +108,7 @@ Historical M007 blocker assessment at baseline `7e241ad` (`plans/closure/persist | Subsystem / plan | Disposition | Evidence | |---|---|---| -| On-prem user access M023 — short-window team fairness | conditionally closed — opt-in per-principal/global token buckets, pre-body 429/Retry-After, bounded process-local state, default/no-default suites and tooling pass; post-change ordinary-Linux p50/p95/p99 comparison remains outstanding; no registered successor | `plans/closure/on-prem-user-access/023-status.md`, implementation `84482e9` | +| On-prem user access M023 — short-window team fairness | closed — opt-in bounded token buckets, pre-body 429/Retry-After, modified-binary ordinary-Linux/NVMe no-rule matrix, and three active-throttle runs; no registered successor or capacity SLA | `plans/closure/on-prem-user-access/023-performance-qualification.md`; implementation `84482e9`, qualification tooling `83c0779` | | On-prem concurrency performance M002 — ordinary-server team baseline | closed — 27 passing Q008 runs across nine cells; Q009 hard-budget, provider-fault, user-disable and restart evidence; bounded competing-I/O and SQLite gate diagnostics; measured host-specific sizing guidance; M023 promoted to ready | `plans/closure/on-prem-concurrency-performance/002-status.md`, artifacts under `artifacts/qualification/on-prem-concurrency-performance/ordinary-server/` | | On-prem user access M022 — hard-budget agent interoperability | closed — Codex CLI 0.162.1 omitted a Responses output cap; OpenCode 1.18.35 sent configured `max_output_tokens`; stable proof-failure and exhausted-budget codes added; default/no-default verification passes; M023 is now ready after M002 operational evidence | `plans/closure/on-prem-user-access/022-status.md`, implementation `dee65ae` | | On-prem user access M021 — default member policy and provisioning | closed — transactional future-only default profile selection, CLI/browser creation, explicit no-profile override, additive schema 63 migration, policy visibility and budget-separation behavior pass; M022 subsequently closed; M023 remains blocked on ordinary-server evidence | `plans/closure/on-prem-user-access/021-status.md`, implementation `b76d359`, CLI journey `ee563be`, race test `3f1ca54` | @@ -571,3 +571,5 @@ registered successor or downstream plan, so no future work is newly unblocked and no status promotion is required. Its ordinary-Linux post-change performance comparison remains a closure condition, not a dependency for any registered plan. No capacity claim is introduced. + +M023 performance closure audit (`plans/closure/on-prem-user-access/023-performance-qualification.md`, baseline `83c0779a`): the modified release binary passed its 27-run no-rule ordinary-server matrix and three active-throttle runs on the attested Ubuntu/NVMe host. Reviewed the on-prem implementation plans, subsystem dependency graph, and dependency-ready, active, and blocked registry tables. M023 has no registered successor or downstream plan; no future work is newly unblocked, no blocked row is promoted, and no other plan status changes. The on-prem user-access roadmap is closed with its historical closure records retained. No capacity SLA is introduced. diff --git a/plans/subsystems/on-prem-user-access-roadmap.md b/plans/subsystems/on-prem-user-access-roadmap.md index 8202210c..e8fa572e 100644 --- a/plans/subsystems/on-prem-user-access-roadmap.md +++ b/plans/subsystems/on-prem-user-access-roadmap.md @@ -1,6 +1,6 @@ # On-Prem User Access Roadmap -Status: active +Status: closed Long-term references: @@ -346,21 +346,21 @@ behavior and adding actionable hard-budget diagnostics. No output cap is injected; M008/M015 proof-or-reject guarantees remain in force. Source: `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md`. -#### M023 — Short-window team fairness controls (conditionally closed) +#### M023 — Short-window team fairness controls (closed) -Class: capability. Hard M022 dependency is closed, and the operational -ordinary-server Linux/SSD-or-NVMe baseline gate is satisfied by concurrency -M002. Production implementation and local verification are complete. The -post-change ordinary-Linux p50/p95/p99 comparison remains outstanding, so no -new capacity claim is made. M002 provides a host-specific synthetic reference, -not a universal capacity guarantee. +Class: capability. Hard M022 dependency and the operational ordinary-server +Linux/SSD-or-NVMe baseline gate are closed. Production implementation, +functional verification, the post-change no-rule comparison, and active +throttle measurement are complete. The qualification is host-specific and +synthetic; it establishes no universal capacity or latency guarantee. Source: `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md`. +Final performance evidence: `plans/closure/on-prem-user-access/023-performance-qualification.md`. ### Post-merge dependency and hardware policy -- M017–M022 are closed; M023 is conditionally closed on its post-change - ordinary-Linux comparison. M022 did not depend on a Pi/Le Potato run; it became - ready after M017 closed. +- M017–M023 are closed. M022 did not depend on a Pi/Le Potato run; it became + ready after M017 closed. The separate M023 host comparison is recorded in its + final closure pass. - Hard: M017 + M019 -> M020 -> M021; M017 -> M022 -> M023. - Interface: M018's privileged local transport must not broaden HTTP rights; any new auth authority stops for ADR review. M022 hard-cost proof contract @@ -462,4 +462,4 @@ A ready plan is not an implemented capability. | M020 | closed | `plans/implementation/on-prem-user-access/020-managed-transition-headless-admin.md` | `plans/closure/on-prem-user-access/020-status.md` | Headless member lifecycle and first-member transition are atomic and audited; M021 unblocked. | | M021 | closed | `plans/implementation/on-prem-user-access/021-default-member-policy-provisioning.md` | `plans/closure/on-prem-user-access/021-status.md` | Transactional future-only profile default; CLI/browser and no-profile journeys, migration, policy and budget boundaries verified. | | M022 | closed | `plans/implementation/on-prem-user-access/022-hard-budget-agent-compatibility.md` | `plans/closure/on-prem-user-access/022-status.md` | Captured Codex/OpenCode output-cap behavior; preserved hard proof-or-reject semantics and added actionable error codes. | -| M023 | conditionally closed | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | `plans/closure/on-prem-user-access/023-status.md` | Production and local verification pass; post-change ordinary-Linux p50/p95/p99 comparison remains the named condition. No successor is registered. | +| M023 | closed | `plans/implementation/on-prem-user-access/023-short-window-fairness-controls.md` | `plans/closure/on-prem-user-access/023-performance-qualification.md` | Modified-binary no-rule matrix and active-throttle measurements pass on ordinary Linux/NVMe. No successor is registered; no universal capacity claim. |