Repository navigation
feat: serve the web UI from sqb server and ship it in the wheels - #1329
Merged
Merged
Conversation
Port the prototype pages, components and styles into ui/ as a static single-page app, with neutral example owners and data.
Wire the lineage hover handlers to the current event names, declare the run gantt element refs as state and add Node types for the generated tsconfig.
Pages and the shell read through an ApiClient interface shaped like the server API, with changed-signal subscriptions for live refetches. A mock client provides neutral example data, a coalescing warehouse sync and per-target trigger state.
The sidebar follows the new navigation with capability-gated admin items and live counts. The top bar shows location / target, connection state, snapshot age and warehouse sync freshness with a coalescing refresh, above a platform-alert banner. Pages without content yet render a not-built state.
Overview, Runs, Jobs, Triggers, Checks and Connections load from the client and refetch on changed signals. Run statuses use the orchestration vocabulary, trigger state shows the code default, and gated actions are disabled without the matching capability.
Runs list with the design's statuses and kinds, trigger and tick links, requested-by, node counts and nullable cost, with group parents expanding to their child runs. A Queue tab shows why each queued run waits (write claims, full concurrency groups, priority, upstream jobs) and its overlap outcome. The timeline projects schedule and calendar ticks only and marks covered and skipped ticks. Run detail keeps the Gantt, status rail and tabs, and now shows nodes with skipped-because-of across jobs (kept in the graph), Recall events and scheduler facts, output, per-statement SQL, liveness, cancel progress, provenance and invocations. Cancel, Rerun and Rerun from failure are capability-gated. The mock client gains an operate store that derives node outcomes, events and run status from run records, and a small scheduler loop so claims, cancels and launches progress like they do on the server.
The panel's verbs and flags are generated from the CLI's command table by scripts/extract-command-table.mjs (npm run gen:command-table; --check fails when the generated file is stale, for CI). Selectors autocomplete from the snapshot's nodes. Before launch the server check shows the resolved selection, the write set, run.adhoc coverage (refused as a whole, listing out-of-scope nodes), whether run.rewrite_history is needed, and the claims the run would queue behind. The target is locked to the target switcher, and launching opens the queued run.
One list for jobs and groups with a kind filter and no create button. Detail shows the read-only declaration and its source file, the resolved nodes, settings, overlap, on_failure, concurrency groups with current holders, priority, targets and owner; for groups, the members in their lineage-derived order with after edges and critical nodes. Attached triggers can be paused or enabled (an audited state fact), recent runs are listed, and Launch takes an optional cursor window or full refresh gated by run.rewrite_history.
The list covers cron and calendar schedules, built-in job, node and platform-event sensors and custom @sensors, with running or paused state against the code default, reset to code default, and bulk pause or enable recorded as audited facts with an optional reason. Detail shows the tick log (requested, skipped, covered by run X, failed, deferred, late and catch-up), next due times, the calendar's stored horizon with its low-horizon warning, and for sensors the condition, evaluations (requested, completed, failed, timed_out, abandoned), the cursor with a reset gated by sensor.reset_cursor, run requests by run key, a code-changed-since-last-evaluation notice and the override history.
A child run whose group run hasn't been claimed now says so, instead of reading as waiting for an upstream job. The README documents regenerating the run panel's command-table extract.
Runs now carry kind (job, group, adhoc) and origin (manual, schedule, sensor, rerun). Group children are job runs that inherit the parent's origin. The run list gains an origin filter and labels show both.
Tick outcomes are requested, skipped, covered and failed. The tick that could not be evaluated while the code location was unavailable is now a failed tick.
The tick_overdue event named activity_microbatch, which ticks normally. A new hourly segments_hourly schedule has its last tick 72 minutes ago and no evaluation since, so its next due time is in the past and the triggers list shows it overdue with a watchdog warning. Platform fixtures that still named the removed nightly_build job now point at existing jobs and nodes.
Users admin gains Disable (revokes the account's sessions, confirmed first) and New service account (name and description; it can then be issued API tokens). Both need user.manage and are audited by the mock. The mock refuses disabling yourself and removing your own last admin assignment, recording the refusal as a denied audit entry, and the roles page flags that assignment with a stronger warning.
The reworked Connections page reads the connections overview, so the unused listConnections endpoint, the old Connection type and its fixtures go. Mock principals are imported from principals.ts directly.
Add the BUILD block to the API client (overview, catalog nodes, column lineage, checks, evidence and the snapshot change feed) with wire types in types/build.ts and a mock that derives build state from the run history, builds outside the scheduler and the change feed. The overview now shows typed attention items (failed and partial runs, stale nodes split into changed and behind with a gated Rebuild, late sources, platform events, unresponsive runs, blocked queue entries, low calendar horizons and pending role approvals), metrics with partial runs counted separately and 24-hour cost against a rolling baseline where the adapter reports cost, build status by group, and recent snapshots with their git author.
Checks list the latest audit result per check with severity, run scope phase, evaluation mode, outcome and violations or measured value, plus a strip of recent outcomes. Each check has a page with its history, the executed SQL (view.sql) and stored evidence rows (view.evidence, audited). Run checks opens the run panel on sqb audit over the visible checks' nodes.
Lineage and Catalog now read catalog nodes from the client instead of inline prototype data. Last outcome and stale reasons (changed, changed upstream, behind) are shown separately, build age is labelled Last built, and drift and SQL compare against the definitions stored with each fingerprint in this target; builds outside the scheduler show their fingerprint run and time. Nodes carry an owner list that can be filtered. The catalog page gains Drift, Runs, Checks, Warehouse (object, declared against actual columns, row statistics and microbatch coverage) and History (renames, former names and old-name views) tabs. Graph selections and the toolbar filters feed the run panel, and the large-graph canvas is kept.
Changes lists each published snapshot with its generation, git revision, author and publisher: nodes added, removed, changed or renamed with their column changes, jobs, triggers and sensors, roles with the nodes they now cover, and environment fingerprint changes. Entries link to the affected nodes and to the runs that followed, and show per-run cost changes by job where the adapter reports cost.
…nnels Alerting is done by sensors, and where their messages go is an ordinary provider used by the sensor's code, so the separate channel concept is gone: no Channels panels, types, client method or mock data. A chat_webhook provider shows the pattern, with the platform-event sensor as its user. The Alerts page becomes Health under Platform. It shows the scheduler lease, server replicas, the heartbeat and the read-only server-side platform-event sink, followed by the platform event history. Each event links to the trigger, job, run or location it affects and to the sensors that reacted to it, with how each handled it. The banner, overview attention items and nav badge now point at /health.
Actions read in plain words with the code on hover. Denied rows show their copyable decision ID and a Check this link to the access checker.
…ntrol API A check's SQL and execution error come from a separate audited view.sql read, loaded on request, because driver errors quote SQL. Evidence cells can be booleans and columns are the rows' sorted keys. A move discovered from the warehouse has no origin model, so pages show its origin relation. Microbatch coverage bounds can be integer or timestamp text, the frontier can be missing, and the panel says coverage is tracked only for concurrent microbatch models, with known gaps meaning holes between completed partitions. The warehouse sync runs on the location's schedule, so the shell shows when it last synced and the on-demand refresh and its permission are gone.
npm ci refused the lockfile because two optional build dependencies were missing from it. Regenerated with npm install; the locked @lucide/svelte stays at 1.17.0, which type-checks cleanly.
The control server has neither: roles defined in code apply when their snapshot is published, and people sign in with single sign-on, a password or sqb login. The approvals tab, its banner, attention item, nav badge and change-feed tag go, along with the service-account screens, API tokens, the token.manage permission and the service-account principal kind. Mock CI and reporting actors become ordinary people.
Checkpoint, not ready for review. The grant decision is now a pure function shared by authorize_person (which still audits denials) and an unaudited capability read. GET /api/control/bootstrap lists the locations a person can view; the person change feed needs View on every node of the target, rechecks the session and grants before each notice, and drops per-run notices. Validated: targeted orchestration tests on SQLite and PostgreSQL, clippy, fensu, and the TypeScript client check and test. No UI changes yet.
A refused location listing named the nodes the person could not view and read differently for a location that does not exist. Listing refusals now carry one fixed message with the decision ID, identical for unassigned and unknown locations. The audit event keeps the missing nodes.
Checkpoint, not ready for review. The UI talks to the server unless ?mock or VITE_SQB_MOCK=1 selects the mock: password sign-in with return to the requested page, sign-out, the bootstrap's locations in the switcher, the selected location's change feed with live, reconnecting and offline states, and navigation entries hidden when the server has no endpoint for them. Validated: svelte-check, and a browser walk-through against sqb server with a SQLite store: login rejection and success, location switch, server outage and recovery, sign-out, and a person with no roles. No UI unit tests yet.
Every refusal from person authorization now shows fixed text per action and its decision ID; the audit event keeps the missing nodes. Unknown jobs and triggers are refused only after the caller is authorized for the target, so an unknown object reads the same as a forbidden one.
…direct The sign-in redirect now resolves next against a fixed origin, so backslash and control-character variants of //host are refused too.
…at/ui-live-shell # Conflicts: # crates/sqlbuild-orchestration/src/authorization/main/authorize_person.rs
Before a retry the feed now asks the feed URL itself and stops on 401 or 403. The bootstrap lists a location when any node is viewable, so a partial viewer retried the refused feed forever, auditing each refusal. The sign-in redirect also refuses paths that resolve to //host.
# Conflicts: # clients/typescript/src/schema.d.ts # crates/sqlbuild-orchestration/src/api/models.rs
Checkpoint, not ready for review. With the ui cargo feature, a build script embeds ui/build (or SQLBUILD_UI_DIST) and sqb server answers GET and HEAD outside /api and /auth with the matching file, or the index for client routes. Hashed assets are immutable; the page carries a hash-based CSP meta policy and the server adds frame-ancestors. The theme script moved to a static file so no inline script needs a hash. Validated: a fixture-table contract test of routing, caching and CSP headers, with a mutation check; clippy with all features; fensu; svelte-check and vitest; and a real sqb server built with the feature behind HTTPS, where a deep link redirected to sign-in, sign-in returned to the page, and the shell rendered its empty state. Not yet: wheel builds, a populated location, make check-ci.
Checkpoint, not ready for review. A UI path containing a double slash, a backslash, a dot or dot-dot segment, or a percent-encoded dot, slash or backslash is refused instead of reaching an asset or the shell. Every wheel job builds ui/ with Node and builds sqb with the ui feature; the linux-x86_64 smoke starts the packaged sqb server and requires the shell and an asset. Changes under ui/ now select the wheel builds. Validated: a raw-request contract test of 22 path cases (dot segments, encoded traversal, slashes, backslashes, /api versus /apix, query strings and a fragment) with a mutation check; clippy with all features; fensu; the smoke commands against a locally built wheel, which fail on a wheel built without the feature; the scope classifier on a ui-only change. Not yet: make check-ci, CI wheel builds on every platform, review.
Wheel jobs set SQLBUILD_UI_REQUIRED, which turns the missing-UI build warning into a build failure on every platform, not only the one the smoke step covers. Local builds keep the warning. The theme script now follows the page's CSP meta tag so the policy governs it.
… it unreachable A server started without password, OIDC or trusted-proxy sign-in has no bootstrap endpoint, so the shell reported a 404 as an unreachable server with a Retry button that cannot help. It now says the server has no person sign-in configured and links to the server documentation.
# Conflicts: # crates/sqlbuild-orchestration/src/authorization/main/authorize_person.rs # crates/sqlbuild-orchestration/tests/api/test_denials.rs
# Conflicts: # .github/workflows/verify.yml # ui/src/app.html # ui/src/lib/state/app.svelte.ts # ui/src/routes/+layout.svelte # ui/svelte.config.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The orchestration UI only ran on the Vite dev server. A released
sqb servershould serve it directly, so installing the wheel is enough to use it. Pages without server endpoints stay hidden, so a partial UI ships honestly.Changes
uicargo feature (sqlbuild-orchestration, re-exported by sqlbuild-cli). A build script embedsui/build(orSQLBUILD_UI_DIST); without the feature the server is unchanged.sqb serveranswers unmatched GET and HEAD requests outside/apiand/authwith the exact embedded file, or the index for client routes. Missing files, writes, and non-canonical paths (//, backslashes, dot segments, encoded.,/or\) get 404 and never reach an API route.frame-ancestors 'none',nosniffandno-referrer. The inline theme script moved to a static file.sqbwith the feature;SQLBUILD_UI_REQUIREDfails the build if the UI is missing. The linux-x86_64 smoke starts the packaged server and requires the shell and an asset. ui/ changes select the wheel builds.Verification
/apivs/apix, query and fragment). A mutation check fails them.make check-ci.sqbwith the feature behind HTTPS: deep link to sign-in and back, no CSP violations.