Repository navigation
Description
Summary
Cortex already holds the richest record of what a coding agent did — 30 days of events on
disk, with timings, token counts, hosts and failures that exist nowhere else. Today the only
ways to read it are the session API (one session at a time, for a TUI) and a bespoke Go
program against archive.Page/Earlier. There is no supported way to get a month of events
out of the archive and into a data store for analysis.
This proposes a content-free per-event export: a projection of SessionEvent to a flat
record, an NDJSON sink, and the ability to replay it out of the archive so the existing 30
days can be backfilled rather than only tapped live.
Why a projection is the feature, not an optimisation
core/sessionapi/summary.go already solved a near-identical problem and measured it:
a single inference request carries the whole conversation... 99.5% of that is the
inferencefield alone. Measured on real traffic, dropping the payloads is a 203x
reduction.
The same ratio governs an export. An analysis of a local 30-day archive (66 sessions,
29,606 events) kept ~30 scalars per event and produced a 29,606-row NDJSON file; the same
corpus at full fidelity is multi-gigabyte (BytesUp alone totals 6.08 GB). Exporting raw
events would be unshippable and would carry every prompt off-box. The projection is what
makes this safe and affordable at once.
summarizeEvent also documents the trap worth copying: its drop rule fails silently —
"a search for completion text simply stops matching." An export schema needs the same
discipline: an explicit rule for what may be dropped, and a test that pins the contract so
it cannot drift.
Tier 1 — the core record, always exported, content-free by construction
Roughly 30 scalars per event. This is the set that an offline analysis of this archive
actually used; each group is here because something could not be answered without it.
| Group | Fields |
|---|---|
| Identity | sessionId, sessionIncarnation, seq, requestId, at |
| Routing | host, requestedHost, httpMethod, httpPath |
| Outcome | phase, statusCode, errorKind, errorCode |
| Timing | durationMs (and ttfbMs once #951 lands) |
| Size | bytesUp, bytesDown |
| Attribution | client, agentRole |
| Inference | model, requestedModel, stream, streamedResponse, finishReason |
| Tokens | promptTokens, completionTokens, cacheReadTokens, cacheWriteTokens |
| Counts | messageCount, toolCount, toolCallCount, newToolResults, toolResultErrors |
| Tunnel | tunnel, tunnelReason |
Notes on the non-obvious ones:
- All four token counts separately.
ledger.Rowalready makes this argument: cost is
derived from the cache-read/write split rather than the total, so a partial set silently
changes the figure. requestedHostandrequestedModelare how a consumer sees that a plugin redirected
or re-targeted a request. Without them a redirect is invisible.streamedResponsealongsidestream— asked-for versus arrived. On this archive 295
requests asked to stream and did not; that is currently invisible (feature: three archive fields for offline session analysis — distinct tool-result counts, interleaved-completion marker, stream mismatch #1740, item 3).newToolResults/toolResultErrorsare feature: three archive fields for offline session analysis — distinct tool-result counts, interleaved-completion marker, stream mismatch #1740, items 1 and 2. Without them, any
consumer countingToolResultsrows overcounts by ~272x on this corpus, because every
request re-sends the whole conversation. This is the single easiest metric in the export
to get wrong.tunnel/tunnelReason— 287 tunnels on this archive failed withdial-failed,
program-refusedoros-trust-only, i.e. traffic every plugin was blind to. Nothing
aggregates it today.
No message content, no completion, no tool arguments, no tool results, no MCP params.
That is what makes a Tier 1 record shippable to an external collector without a per-
destination privacy review.
Tier 2 — opt-in structured detail
toolCallNames: []string— the tool names a response asked for. This is what enables
per-tool latency (below). Names are low-risk; arguments are not — aBashcommand
line is effectively a secret and belongs in Tier 3.invocations: [{plugin, action, reason}]— per-plugin outcomes.summarizeEventkeeps
these out of the drop set for good reason: the ACTION column and theplugin:<name>
filter both read them.identity: {subject, clientId, agentId}— the authenticated identity, as distinct
fromclient.
Tier 3 — content, off by default
Messages, completions, tool results, MCP params. Proposed position: an export path should
refuse this by default rather than offer it as a flag someone flips under time pressure.
The replay use case for full fidelity is already specified in #1494; this issue is not that.
What this unlocks: tool time vs human time, from Cortex alone
The motivating question was "where did the agent's wall-clock go", with no agent-harness
access. Tier 1 answers it. Classifying each response -> next request gap by whether
proxied traffic landed inside it:
| n | p50 | total | share of gap time | |
|---|---|---|---|---|
Proxied tool (measured from durationMs) |
1,910 | 1,045 ms | 11.6 h | 18.8% |
| Local tool (inferred — no proxied traffic) | 5,653 | 158 ms | 5.3 h | 8.6% |
| User wait (0 tool calls) | 1,442 | 32,358 ms | 44.8 h | 72.6% |
With toolCallNames (Tier 2), single-tool-call turns yield per-tool latency directly:
Bash p50 201 ms (n=3,754), Edit 89 ms (n=814), Read 82 ms (n=338), and AskUserQuestion
62,602 ms (n=68) — the last being useful validation, since the classifier never saw the
tool's name yet separated the one human-blocking tool from the machine ones by three orders
of magnitude.
Honest limits: the user-wait total is very sensitive to the gap ceiling (24.8 h at 300s,
44.8 h at 1800s, 56.8 h at 3600s, 539 h uncapped), so any consumer publishing it must
publish the cap too.
Three schema decisions worth settling in review
-
Export
durationMs; never expect a consumer to subtract timestamps. The fast-tool
figures above sit near wall-clock resolution, andSeq's own doc notes same-instant
events are common on a fast local proxy. A consumer pairing byrequestIdand
subtractingatgets noise instead of latency. -
sessionIncarnationis the one genuinely new field here. A session id is reused: the
iddefaultrecurs across the entire 2026-10-15 -> 2026-11-06 range of this archive, in
one directory, spanning many unrelated working periods. Without an incarnation key, every
per-session aggregate downstream silently averages unrelated work. The archive can derive
it from segment boundaries (one of these sessions has 34 segments across three weeks).
Raised independently on feature: agentop observe — group sessions and cost by working directory #1739, where it also affects per-directory cost totals. -
httpPathcan carry secrets and needs a decision before it ships off-box. Per its
own documentation: "A secret embedded in a path SEGMENT does survive, because nothing can
tell it from a resource id — a bot token or a webhook path lands here verbatim." Query
strings are already stripped; path segments are not. Either redact, or make this field
opt-out, but do not ship it off-box silently.
Same caveat applies to client, which is derived from a request header: a cost-attribution
and display key, never an authorization subject. Worth restating in the export docs,
since an external collector is exactly where someone will be tempted to treat it as identity.
Format and transport
- NDJSON, to a file or an HTTP endpoint. This is what post-processing wants: the
analysis behind this issue wasjqand Python over NDJSON, and a full 30-day pass took
~10s single-threaded. - Archive-replayable, not only a live tap. The archive already holds 30 days;
backfill is most of the value. Note for whoever implements it:ReplaySources/
ReplaySinceexcludes tunnel rows by design (correct for cost replay — no tokens),
which is 10,493 of 29,606 events here and includes every non-LLM latency figure.
An export walk wantsPage/Earlier, which do not filter. - A documented drop rule plus a test pinning the Tier 1 contract, in the spirit of
TestSummarizeEvent_KeepsEverythingTheFilterSearches.
OTLP deliberately left as a follow-up. The metrics subset maps cleanly onto the GenAI
semantic conventions (gen_ai.* already standardises model, token counts and finish reason)
and belongs with #505, which is about emitting spans for Cortex's own pipeline. Bundling
both into one issue makes it unreviewable.
Relationship to existing issues
- feature: Report statistics to a central collector #898 (report statistics to a central collector) wants aggregates — token count by
hour — shipped to a remote endpoint. This issue is the per-event layer those aggregates
could be derived from. Worth deciding whether feature: Report statistics to a central collector #898 becomes a consumer of this or stays
separate, rather than building two exporters. - feat(authbridge): first-class OpenTelemetry for the proxy #505 (first-class OpenTelemetry) instruments Cortex itself. Different axis: that is
the proxy's own behaviour, this is the traffic it observed. - feature: three archive fields for offline session analysis — distinct tool-result counts, interleaved-completion marker, stream mismatch #1740 adds three of the Tier 1 count fields. This export is the main consumer for them.
- feature: lossless capture in the session archive, so past sessions can be replayed to test a change before turning it on #1494 (lossless capture and replay) is the Tier 3 / full-fidelity case, already specified.
- No long-term view of inference server performance to compare against current performance #1466 wants a long-term performance view; it notes the archive's raw material exists
but is never aggregated beyond six hours. An export is one way to get that aggregation
outside the proxy.
Scope of the measurements above
One developer's laptop archive, heavily Claude Code (18,785 of 29,606 events carry
claude-code/2.1.270). The field list generalises — it follows the event schema, not
this traffic. The ratios do not: #1494 measured 59% of replies batching tool calls under
explicit prompting against 7.3% here unprompted, so those numbers move with working style
and prompting.
Assisted-By: Claude (Anthropic AI) noreply@anthropic.com
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsNew/ToDo
Activity
mrsabath commentedon Nov 11, 2026
Scope decision on the open question in the "Relationship to existing issues" section:
#898 consumes this layer. The boundary:
for backfill. Getting the data out of the proxy in a usable, content-free shape.
hour", per-plugin usage, savings from the cost-saving plugins, plus the delivery semantics
and the organisational questions about what may leave a machine.
So this issue does not need to grow an aggregate mode, a remote endpoint, retry or
batching. Those stay on #898, which now reads this export rather than collecting
independently.
The practical consequence for Tier 1: it must stay sufficient for #898's aggregates, since
they will be derived from it rather than from the events. The fields that issue needs —
model,host, the four token counts,durationMs,client,invocationsfor per-pluginattribution — are already in Tier 1 and Tier 2, so no additions are implied. Worth keeping
in mind if anything is dropped from the Tier 1 set during review: a field removed here is a
field #898 cannot aggregate.
Noted on #898 as well.
Fix: Address review on the content-free export record
Fix: Keep the User-Agent's version out of the export too
Fix: Close the version-shape bypass and bound every wire string
Fix: Close the build-tag door and the agentRole passthrough
Feat: Project session events to a content-free export record (#1812)
fix: Answer a status when the export walk dies before its first record
fix: Refuse the silent widenings in agentop export
mrsabath commentedon Nov 13, 2026
Spec drafted — #1854
This issue, #898, #505, #1740 and #1466 all touch the export, and #1812 shipped part of it,
but nothing said in one place what a record is or which of the fields above actually landed.
Drafted that as a spec: #1854, at
docs/superpowers/specs/2026-11-13-session-event-export-design.md.53 numbered requirements, a normative field reference for the 36 fields in
core/sessionexporttoday, and an explicit list of what this issue proposed that did not ship.
What shipped, against this issue's Tier 1
inputTokens/cacheRead/cacheWrite/outputTokens/reasoningTokens+presentKinds, not the four this issue listedhttpPathsessionIncarnationnewToolResults,toolResultErrorsTwo notes on the token change, since this issue specified the other four.
promptTokensturnedout to be the legacy aggregate (
Fillsets it to input + cacheRead + cacheWrite), soexporting it beside
cacheReadTokensdouble-counted every cached token — 196,096 of them on asingle response of this archive. And
reasoningTokensis a subset ofoutputTokens, not asibling. The spec states both as rules (R18, R20) so the next consumer does not have to
rediscover them.
One field this issue predates
SessionEvent.Projectlanded in #1815, after this issue was written — the git-repo-root labelof the agent process. It is the natural key for per-project cost and latency and is not in
the record. It is also the same question
httpPathraises: a repo path under$HOMEnamesdirectories a user may not want off-box. Flagged Open rather than quietly added.
The five decisions the spec asks for, not answers
httpPath— redact, opt-out, or never? (this issue's decision 3, still open)project— is a repo path content?sessionIncarnation— derive in the archive, or in the export walk? (this issue'sdecision 2)
Recordtype? Worth settling before twotransports exist.
This issue's decision 1 — export
durationMs, never expect a consumer to subtract timestamps— is settled and shipped, as R22/R23 plus
TestRecord_DurationsAreMilliseconds.Review welcome on #1854, particularly §8.1 (#898's relationship) and §7.2 (Tier 2 breaks the
flat-scalars rule and needs its own schema decision).
fix: Bound the unknown-session refusal in bytes, not in ids
fix: Leave a typo somewhere to go when the refusal does not fit
docs: State what the export collects, and point timings at the timeline
10 remaining items
docs: Point timings at the timeline, and document the seq and limit t…
docs: Fix the seq citations and complete the test plan
docs: Address review on #1854 - privacy claims, arithmetic, and two r…
docs: Spec the session event export (#1854)
feat(scripts): Add cortex-timings.sh for per-tool and per-call latency (
Feat: agentop export writes the archive's record export to a file (#1831
docs: Add a walkthrough for exporting and analysing the archive
docs: Rewrite the export and dump docs in the spirit of ASD-STE100
docs: Say where a session id comes from, and what the timeline cannot…
Feat: Walk the archive into content-free records over GET /v1/export (#…
mrsabath commentedon Nov 19, 2026
The Tier 1 field table above is now behind what shipped in #1812 / #1830 / #1831. Noting the differences so nobody builds against a field that does not exist, or re-litigates one that was dropped on purpose.
Dropped deliberately, and they should come off the table:
promptTokens/completionTokens.InferenceExtension.PromptTokensis alreadyInput + CacheRead + CacheWrite. Exporting it besidecacheReadTokensships a total next to its own components, and a consumer pricing the input rate on one and the cache rate on the other counts every cached token twice — 196k of them on a single response on the development archive. The record carries the four disjoint kinds only, which is what this issue's own "all four token counts separately" note asks for.httpPath. Decision three above asked for this to be settled before it shipped off-box. It was settled by leaving it out: a path segment can hold a bot token or a webhook secret, and nothing can tell that from a resource id.Invocation.Pathis a back-filled copy of the same string, so it is out too.httpMethodalone carries the request's shape.newToolResults/toolResultErrors. Still feature: three archive fields for offline session analysis — distinct tool-result counts, interleaved-completion marker, stream mismatch #1740, still unbuilt, so the record carries no tool-result figure at all rather than one that overcounts by ~272x.toolCallCountships and is unaffected.Shipped, not on the table:
ttfbMs— feature: latency metrics — time to first token, total response time, session percentiles #951 landed, so this is no longer conditional.servedModel— what the response named, when it differs from what was asked for.requestedModelalone does not show it.reasoningTokens— a subset ofoutputTokens, never a fifth kind. Adding the two double-counts.presentKinds— the one field that escapes the record's zero-means-absent rule. A set bit with a zero count is "the provider reported zero"; an unset bit is "the provider does not expose this kind". Without it those two are the same absent key.direction,phase—phasewas in the Outcome group;directionwas not there at all.schema— the record format version, on every record.Also unshipped: tool-call names (Tier 2) and
sessionIncarnation(Tier 1), the latter now tracked as #1997.Net: the shipped record is 36 fields. Decisions two and three of "Three schema decisions worth settling in review" are both now settled — two as #1997, three by omission.
Assisted-By: Claude (Anthropic AI) noreply@anthropic.com
Docs: Correct four measurement claims and redefine the gap
Docs: Spec the toolCall export record, and correct the per-tool laten…