Skip to content

feat(secrets): record which secrets each run resolves, and surface it per secret - #6823

Merged
icecrasher321 merged 20 commits into
stagingfrom
feat/secret-usage-audit-trail
Aug 19, 2026
Merged

feat(secrets): record which secrets each run resolves, and surface it per secret#6823
icecrasher321 merged 20 commits into
stagingfrom
feat/secret-usage-audit-trail

Conversation

@icecrasher321

Copy link
Copy Markdown
Collaborator

Why

A workspace secret is usable by credential Members but readable only by credential Admins. That boundary is not enforceable inside a Function block: a few lines of code can print a resolved secret one character at a time, and no value-matching redaction can stop it (MIN_SUBSTITUTABLE_LITERAL_LENGTH is 8, so per-character output never matches). Deciding whether arbitrary code will eventually reveal a value is undecidable in general — ShellCheck says the same about indirect references.

So this is the other half of the posture: detection and attribution, not prevention. Every run records which configured secrets it actually resolved, under whose identity, so an admin can spot misuse and rotate with confidence.

This cannot be derived from execution logs. They persist the whole available encrypted environment rather than what a run referenced, they evidence a secret only where redaction happened to fire, and they expire under logRetentionHours — while "who has touched this key" outlives any single run.

The data already existed in ResolvedSecretTraceRegistry.addActiveEntry, the universal funnel for every named secret that goes live in a run. It was persisted only for paused runs; this persists it for every terminal path.

Schema

secret_usage — a per-UTC-day rollup, not one row per run. A workflow on a one-minute schedule touching three secrets would otherwise write ~4,300 rows/day, which is also why this is not audit_log: that table is a human-scale compliance surface and machine-scale rows would drown it.

Two decisions worth reviewing closely:

  • workflow_id / actor_user_id use '' sentinels, not null, and are not FKs. Both sit inside the unique key, and Postgres treats nulls as distinct — two Copilot rows would never collide, so the upsert would insert forever instead of incrementing. NULLS NOT DISTINCT fixes that but needs Postgres 15, and this is self-hosted software that must not raise its database floor for one table. They are not FKs because an onDelete: 'set null' would rewrite a key column, so two rows differing only by the deleted id would collide and an ordinary workflow deletion would fail on this constraint.
  • secret_owner_user_id is part of the key. Two people can hold a personal secret under one name, and a personal secret shared with the workspace resolves for a caller who does not own it — so name and scope alone do not identify a secret. It is not the actor: a scheduled run resolves the workflow owner's personal slice under the workspace's execution actor, so filing by actor would both leak across users and hide a row from the person who can actually rotate the key.

Coverage

Surface How
Workflow runs logging-session.completeExecutionWithFinalization — the one funnel all terminal paths reach. Skips paused so a resumed run is not double-counted.
Sim agent code function-execute handler, from the mounted registry
Sim agent integration tools tool-executorresolveCopilotEnvReferences substitutes {{SECRET}} into user-only params, which is a real use of the key
MCP server config resolve-config, which resolves outside any run

Direct environment reads are now detected, so a secret read as environmentVariables['K'] or $K enters the run's provenance instead of going unredacted — this fixes a pre-existing redaction gap, not just the trail. JavaScript uses the TypeScript AST; Python is regex-matched then tokenizer-checked; shell is scanned for quoting and heredoc bodies and fails closed (absence of a quote context means the scanner skipped the region, so nothing is recorded).

Read path

See usage is gated on the same predicate that reveals the value — workspace admin or credential admin for that key, i.e. what maskWorkspaceEnvForViewer already applies; personal is the owner's own. A member sees a disabled chip with a tooltip rather than no chip, so the capability is discoverable and the denial is stated.

The trail renders through the shared ActivityLog (same component as the EE audit log and Forks activity), with a new optional trailing slot for the View log action. The column only appears when a caller supplies one, so the other two consumers render identically.

Performance

Measured, not assumed:

  • Write is fire-and-forget and never awaited, so it adds no run latency: one INSERT … ON CONFLICT per run regardless of how many secrets it touched.
  • Hot-row contention: 4,000 upserts from 16 concurrent writers, all contending on a single bucket row, completed in 1.37s (~2,900/s) with use_count = 4000 — no lost updates.
  • Compile cost: each detector prescans for names that are actually configured secrets before paying for a lex or quote-frame pass. Shell went 4.3 µs → 11.4 µs on a 3.4 KB script (a naive ordering cost 97 µs); Python is unchanged at ~6 µs unless the code really reads a secret directly; the JavaScript per-node kind check is below the TypeScript parser's own run-to-run variance (±1 ms).
  • Read is a single index-covered, limit-bounded query on secret_usage_secret_recent_idx.

Known limits (named, not hidden)

  • A hard worker kill records nothing — the same gap the execution log row already has.
  • Shell recall stops where shell stops being static: eval, ${!indirect}, printenv, and sourced files are invisible. What is reported is exact; what is missed keeps the pre-existing behavior rather than degrading it.
  • A value derived from a secret (hash, signature, re-encoding) is out of scope by construction.

Verification

bunx turbo run type-check (24/24) · bun run lint clean · 4,375 tests across the affected suites · bun run check:audits (29/29) · bun run check:migrations origin/staging backward-compatible · migration applied and the upsert/owner-separation behavior proven against a live Postgres 14.

Blog post secret-provenance updated with a short section on why redaction alone is insufficient here.

🤖 Generated with Claude Code

icecrasher321 and others added 3 commits August 18, 2026 14:34
…per secret

Redaction stops a value at a boundary but cannot stop code that never emits it —
a Function block can print a key one character at a time and nothing ever matches
the secret. That is undecidable in general, so this adds the other half of the
posture: attribution.

Every run now records which configured secrets it actually resolved, under whose
identity, through which surface (workflow, Sim agent, MCP). The data already
existed in ResolvedSecretTraceRegistry.addActiveEntry and was persisted only for
paused runs; this persists it for every terminal path.

Execution logs cannot answer this. They store the whole available encrypted
environment rather than what a run referenced, they evidence a secret only where
value-matching redaction happened to fire, and they expire under
logRetentionHours — while "who has touched this key" outlives any single run.

- secret_usage: per-UTC-day rollup keyed by workspace, secret, scope, owner,
  source, workflow, actor. A one-minute schedule touching three secrets would
  otherwise write thousands of rows a day, which is also why this is not
  audit_log. workflow_id/actor_user_id use '' sentinels rather than null so the
  unique key works on Postgres 14 without NULLS NOT DISTINCT, and are not FKs:
  they are historical facts, and an onDelete would rewrite a key column.
- secret_owner_user_id is part of the key. Two people can hold a personal secret
  under one name and a shared personal secret resolves for a caller who does not
  own it, so name and scope alone do not identify a secret. It is NOT the actor:
  a scheduled run resolves the workflow owner's personal slice under the
  workspace's execution actor.
- Direct environment reads are now detected in JS (TypeScript AST), Python
  (tokenizer-checked) and shell (quote/heredoc-scanned), so a secret read as
  environmentVariables['K'] or $K enters the run's provenance instead of going
  unredacted. Each detector prescans for names that are actually configured
  secrets before paying for a lex or quote-frame pass.
- Copilot integration tool calls are covered: resolveCopilotEnvReferences
  substitutes {{SECRET}} into user-only params, which is a real use.
- See usage lives behind a credential-admin gate, using the same predicate that
  reveals the value; members get a disabled chip explaining why.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…audit-trail

# Conflicts:
#	apps/sim/app/workspace/[workspaceId]/settings/secrets/[credentialId]/secret-detail.tsx
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Aug 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Aug 19, 2026 1:17am

Request Review

@cursor

cursor Bot commented Aug 18, 2026

Copy link
Copy Markdown

PR Summary

High Risk
Touches secret provenance, execution-log masking, and new durable attribution data across workflows, Copilot, and MCP—mis-attribution or missed reads would affect security visibility and admin rotation decisions.

Overview
Adds persistent secret usage attribution alongside log redaction: each terminal run (and Copilot/MCP surfaces) records which configured secrets were actually resolved, under which scope and owner, into a new secret_usage table with UTC-day rollups so scheduled workflows do not flood one row per execution.

Provenance and masking behavior changes: Function execute no longer gates __resolvedSecretNames on secret values appearing in output; referenced {{KEY}} bindings and statically recognized direct reads (environmentVariables['KEY'], shell $KEY, etc.) activate masking and usage reporting even for silent API auth or transformed exfiltration. The placeholder compiler gains per-language direct-read detection (JS AST, Python/shell with fail-closed quoting rules). ResolvedSecretTraceRegistry exposes getResolvedSecretUsage() with personal-owner attribution for shared keys.

Product surface: Credential detail adds See usage (same access as viewing the value) with GET /api/secrets/usage, a usage panel in ActivityLog (new trailing View log column), and docs/blog updates on recognition limits.

Writes are fire-and-forget upserts from workflow completion (skips paused), Copilot function_execute / integration tools, and MCP config resolution—without double-counting handler-owned paths.

Reviewed by Cursor Bugbot for commit 01fa6ca. Configure here.

@greptile-apps

greptile-apps Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds daily secret-usage attribution across workflow, Copilot, and MCP execution surfaces, together with direct environment-read detection and an authorized usage-detail UI.

  • Adds the secret_usage schema, migration, upsert writer, bounded query, API contract, and secret-detail activity panel.
  • Carries resolved-secret ownership and actor context through execution finalization, Copilot tools, and MCP configuration.
  • Extends JavaScript, Python, and shell compilation to recognize static direct environment reads for masking and attribution.
  • Updates tests and documentation for usage visibility, execution-log protection, and scanner limitations.

Confidence Score: 4/5

The PR does not yet appear safe to merge because locally rebound shell variables can still produce incorrect secret-usage records.

The earlier shell-shadowing issue remains despite the reply claiming it was fixed: the current scanner records any configured $NAME expansion based on quote context without checking whether the script assigned that name locally, so the usage trail can attribute a secret that was never read.

Files Needing Attention: apps/sim/lib/execution/code-placeholders/shell.ts

Important Files Changed

Filename Overview
apps/sim/lib/secrets/usage/record.ts Adds a fire-and-forget daily rollup upsert whose latest timestamp, execution ID, and trigger remain aligned under out-of-order completion.
apps/sim/lib/execution/code-placeholders/javascript.ts Adds AST-based attribution for literal member access, element access, destructuring, computed literal keys, and rest patterns.
apps/sim/lib/execution/code-placeholders/python.ts Adds lexer-filtered recognition of literal dictionary and get-based environment reads while conservatively retaining write and delete accesses.
apps/sim/lib/execution/code-placeholders/shell.ts Adds quote-, heredoc-, and escape-aware shell expansion attribution, but the previously reported local-rebinding false attribution remains.
apps/sim/app/api/secrets/usage/route.ts Adds the authorized API adapter for querying a secret's usage history.
packages/db/schema.ts Defines the secret-usage rollup table, null-free bucket key, indexes, and related enum.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Code[Workflow, Copilot, or MCP execution] --> Resolve[Resolve configured secret]
  Resolve --> Registry[ResolvedSecretTraceRegistry]
  Registry --> Finalize[Terminal execution finalization]
  Finalize --> Upsert[Daily secret_usage upsert]
  Upsert --> API[Authorized secret usage API]
  API --> Panel[Secret usage activity panel]
Loading

Reviews (16): Last reviewed commit: "fix(secrets): a dot in prose is not a qu..." | Re-trigger Greptile

Comment thread apps/sim/lib/secrets/usage/record.ts Outdated
Comment thread apps/sim/lib/execution/code-placeholders/javascript.ts
Comment thread apps/sim/lib/secrets/usage/record.ts Outdated
Comment thread apps/sim/lib/execution/code-placeholders/python.ts Outdated
…ndings faking usage

Review round 1.

- record.ts: last_execution_id/last_trigger were assigned unconditionally while
  last_used_at was chosen by greatest(), so two runs completing out of order split
  one row between them — the newer run's timestamp beside the older run's execution
  id, making "View log" open a run the row does not describe. Both are now guarded
  on the timestamp actually advancing, so the row's metadata always belongs to the
  run that owns its timestamp.
- javascript.ts: a local binding named environmentVariables (declaration, parameter,
  destructured binding, or bare reassignment) made reads off the user's own object
  look like mounted-secret reads. Any such binding now disables detection for the
  file; the AST already had parent pointers, so this is a kind check during the
  existing walk.
- python.ts: same class of bug with no parser available, so the rule is an allowlist
  — every mention of the binding must be a literal subscript or .get(), otherwise
  detection is off for the file. This also subsumes the cross-line attribute case
  (other.\n environmentVariables['K']), which the previous space-and-tab look-behind
  missed.

Under-reporting is the safe direction here: a trail that claims a use that never
happened is worse than one that misses a use.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@cursor review

CI runs lint:check across every workspace; the drizzle-kit output in packages/db
had never been through biome, so the branch was green locally (where lint had
only been run inside apps/sim) and red on CI. Whitespace only — both files are
byte-for-byte identical once parsed, and drizzle-kit still reports no pending
schema diff against the reformatted snapshot.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comment thread apps/sim/lib/execution/code-placeholders/javascript.ts Outdated
…ot just declarations

Review round 2. A bare `for (environmentVariables of rows)` has no declaration to
key off, so the previous check missed it and reads of the loop value were still
recorded as secret usage.

Rather than extend the hand-rolled node-kind list, this reuses the pair the same
file already applies to reject a placeholder in a write position:
isDeclarationIdentifier covers declarations, parameters, destructured bindings and
imports, and isWriteIdentifier covers every assignment operator, ++/--,
destructuring targets, and for-in / for-of initializers.

That also closes four forms neither the review nor the original check named:
logical (||=) and nullish (??=) assignment, and object and array destructuring
assignment. Six of the eight added cases fail against the previous check.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit a7a5fff. Configure here.

Comment thread apps/sim/lib/execution/code-placeholders/shell.ts Outdated
icecrasher321 and others added 2 commits August 18, 2026 15:46
… log is gone

Review round 3, plus the docs that were left claiming the old behavior.

- shell.ts: a script that writes a configured name (API_KEY=local, export/local/
  readonly, read, for, unset) expands its own value from that point on, not the
  mounted secret, so recording it claimed a use that never happened. Every mention
  of the name must now be a `$NAME` / `${NAME}` expansion, matching the allowlist
  shape the Python detector already uses. Applied per name rather than per file:
  JavaScript and Python shadow one object holding every secret, whereas rebinding
  one shell variable says nothing about the rest.

- The usage trail deliberately outlives execution logs, so a row routinely names a
  run whose log has been pruned. The read now left-joins workflow_execution_logs on
  its unique execution_id and reports availability, and the panel renders the chip
  disabled with the platform tooltip instead of linking into an empty Logs view.
  Three states: no run to link, a run whose log is gone, and a live link.

- Docs said a direct environmentVariables/$KEY read does not activate masking,
  which this branch changes. Corrected in credentials.mdx, function.mdx and the
  logging FAQ, and the recognition limits are now written down: runtime-built
  names, reassigned bindings, and reads that cannot be told apart from text.
  Added a "See usage" section covering who can see it and why an empty trail
  means "nothing recognized" rather than "never used".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…audit-trail

The api-validation route baseline needed a manual union rather than the auto-merge:
staging raised it to 1121 for its own new route and this branch had already raised
it to 1121 for the secret-usage route, so git merged the identical text cleanly and
silently dropped one of them. The true count is 1122.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/lib/execution/code-placeholders/shell.ts
Comment thread apps/sim/lib/execution/code-placeholders/javascript.ts
… not a rebinding

Review round 4.

- javascript.ts / python.ts: `environmentVariables.API_KEY = 'x'` and
  `delete environmentVariables.API_KEY` touch the name without ever reading the
  mounted value, but the detectors matched the member access and recorded a use
  that never happened. JavaScript now asks the same isWriteIdentifier the
  placeholder rewriter uses (its parameter is widened to ts.Node — the body
  already walked generic nodes, so this is a type change, not a behaviour one)
  plus a delete check; Python excludes a subscript followed by `=` and a `del`
  target.

- shell.ts: requiring every mention of a name to be an expansion also fired on
  text that binds nothing — a comment naming the key, or `echo "API_KEY=$API_KEY"`
  where the literal is an argument rather than an assignment — and dropping those
  cost masking on a genuine read. It now looks for actual writes: an assignment at
  command-word position, a binding builtin, `printf -v`, or a `for` target.

  The two directions are not symmetric, which is why this errs toward detecting
  the read: missing a write records a use of a secret the script only had in its
  environment, a misleading audit row and nothing more, since masking still
  searches for the real value and will not find it. Over-detecting a write
  suppresses masking on a value that does reach the log.

  This also makes the code match what the docs already described — skipping after
  a rebinding, not after any mention.

13 tests added; 11 fail against the previous code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review round 8. `\\$API_KEY` is an escaped backslash followed by a LIVE expansion
— bash prints `\` plus the value — while `\$API_KEY` is an escaped dollar and
stays literal. Checking only the character adjacent to `$` read every even run as
escaped, dropping a real read from usage and masking alike; verified against
bash before fixing.

The scanner now counts the run of backslashes before the `$` and skips only odd
runs, the same parity rule logicalLineEndAfterContinuations in this file already
applies to line continuations. Six-case parity table added; the three even-run
cases fail against the previous check.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…audit-trail

Same route-baseline collision as the previous staging merge: staging moved to
1122 for its account-deletion route while this branch was already at 1122 for
the secret-usage route, so the auto-merge kept identical text and lost one.
The union is 1123, verified by running the audit rather than trusting the merge.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/lib/mcp/resolve-config.ts
Comment thread apps/sim/lib/execution/code-placeholders/javascript.ts
Review round 9. `const { API_KEY } = environmentVariables` delivers the value by
name with no property- or element-access node in the AST, so the member-access
walk missed it entirely — and a missed read leaves an emitted value unmasked,
the dangerous direction.

The AST walk now also recognizes the declaration form (shorthand, renames,
defaults, string-literal keys), the assignment form ({ KEY } = env), and a
...rest element — which names no key but takes every value, so it reports every
configured name; the alternative left `const { ...all } = env; return all`
entirely unmasked. A computed key stays unrecognized, the same runtime-name
boundary as a computed subscript, and a receiver that is not the bare identifier
is not attributed.

Nine cases added; the six positive ones fail against the previous walk.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/lib/execution/code-placeholders/javascript.ts
Comment thread apps/sim/lib/execution/code-placeholders/javascript.ts
Comment thread apps/sim/lib/execution/code-placeholders/javascript.ts
@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@greptile

Comment thread apps/sim/lib/execution/code-placeholders/javascript.ts Outdated
…ncluded

Review round 10. Two accurate findings, folded into a generalization instead of
two more special cases:

- A parameter default (function f({ API_KEY } = environmentVariables)) and a
  binding-element default are the same by-name delivery as a variable
  declaration. The detector now keys on the ObjectBindingPattern itself and
  checks its parent's initializer, so every declaration position follows one
  rule instead of per-kind arms.
- Parentheses group without changing the receiver, so (environmentVariables) is
  unwrapped before the identifier check — in the destructuring arm AND the
  member-access arm, which had the same hole unreported.

Declined the for-of-over-array-literal finding: the receiver there is a
container, not the environment object, and following data flow through
containers has no fixed point — the same documented boundary as aliasing and
computed keys. A test pins the boundary so it reads as chosen, not missed.

Eight cases added; the seven receiver-rule cases fail against the previous code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/lib/execution/code-placeholders/python.ts Outdated
Comment thread apps/sim/lib/execution/code-placeholders/javascript.ts

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit a040b2d. Configure here.

…ed key is a subscript

Review round 11. Both findings were implementation-narrower-than-rule, fixed by
consulting authorities the detectors already had rather than adding new ones:

- python.ts: the receiver walk crosses whitespace so a parenthesized `other.` on
  a previous line is seen — but it landed on a comment's final period
  (`# Load the value.`) and discarded the genuine read on the next line. The
  landing position is now checked against the same lexer ranges that filter the
  candidates, which is also why the receiver check moves after lexing.
- javascript.ts: `const { ['API_KEY']: key } = environmentVariables` is the
  element-access rule in pattern position, so a computed key holding a string
  literal resolves like a literal subscript; any other computed key keeps the
  runtime-name boundary a computed subscript already has.

Eight cases added; the comment-period case and all three literal-computed-key
cases fail against the previous code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 01fa6ca. Configure here.

@icecrasher321
icecrasher321 merged commit 521348b into staging Aug 19, 2026
30 checks passed
@icecrasher321
icecrasher321 deleted the feat/secret-usage-audit-trail branch August 19, 2026 01:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant