Skip to content

Default to lean Logfire telemetry - #224

Merged
jgpruitt merged 1 commit into
mainfrom
reduce-logfire-telemetry
Oct 16, 2026
Merged

jgpruitt merged 1 commit into
mainfrom
reduce-logfire-telemetry

Conversation

@jgpruitt

Copy link
Copy Markdown
Collaborator

Summary

  • Default API and standalone workers to a trace-first lean profile: no metric exports and one numeric application-memory/tokenizer snapshot per minute plus startup.
  • Remove cgroup sampling, automatic runtime-node/V8 metrics, periodic provider-wide flushing, and routine health/auth-success logs. Kubernetes/Prometheus remains the infrastructure monitoring source.
  • Preserve HTTP/RPC/error traces, search/response sizes, authenticated identity, mutation counts, and bounded SDK shutdown.
  • Add LOGFIRE_TELEMETRY_PROFILE=diagnostic for explicit application metric opt-in and 10-second snapshots; both profiles keep automatic V8/cgroup telemetry disabled.
  • Add bounded LOGFIRE_RUNTIME_SAMPLE_INTERVAL_MS, deployment guidance, and isolated real-SDK export tests.

Motivation

Today’s Logfire ingestion review measured 233,633 metric points/hour across dev and prod, versus 2,120/hour before the rollout. Automatic V8 heap-space metrics accounted for 61.5% of the later hour; 48 zero-only series represented 56.8%. Existing Kubernetes metrics cover infrastructure health without duplicating it in Logfire. These are observed ingestion counts, not billing estimates.

Validation

  • ./bun run check:full passed against local PostgreSQL, including 33 CLI end-to-end tests.
  • Real-SDK loopback tests verify zero metric requests in lean mode, retained RPC attributes, diagnostic metric opt-in without V8/cgroup series, and native reader cadence.
  • Loopback tests also pass with inherited trace-sampling/min-level/SDK-disabled overrides.
  • git diff --check passed.

Rollout

Deploy this PR SHA to dev after CI passes, then verify pod health and fresh Logfire ingestion. Production and version release are intentionally separate steps.

Copilot AI balanced review requested due to automatic review settings October 16, 2026 20:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No blocking issues remain, and the configuration, lifecycle behavior, tests, and documentation consistently support the intended telemetry profiles.

0 open findings

What changed in this PR

Reduces default Logfire ingestion for the API and standalone workers while preserving application traces and bounded shutdown.

Changes:

  • Defaults to lean telemetry with minute-resolution snapshots and no metric exports.
  • Adds diagnostic metric opt-in and bounded sampling overrides; removes cgroup sampling and routine success logs.
  • Updates export tests and operational guidance.
File Description
packages/​server/​telemetry.ts Applies profiles to shared telemetry setup.
packages/​server/​telemetry-settings.ts Resolves profiles and sampling intervals.
packages/​server/​telemetry-settings.test.ts Tests defaults, opt-in, and interval bounds.
packages/​server/​telemetry-sdk-compatibility.test.ts Isolates SDK compatibility checks.
packages/​server/​telemetry-export.test.ts Verifies real-SDK exports for both profiles.
packages/​server/​runtime-telemetry.ts Simplifies sampling and makes gauges optional.
packages/​server/​runtime-telemetry.test.ts Tests sampling, gauges, and cleanup.
packages/​server/​process-telemetry.test.ts Checks shared lean configuration.
packages/​server/​middleware/​authenticate-space.ts Removes authentication-success logging.
packages/​server/​handlers/​health.ts Removes heartbeat logging.
packages/​server/​handlers/​health.test.ts Verifies health checks emit no routine logs.
DEVELOPMENT.md Adds profile deployment guidance.
design/​runtime-observability.md Documents profiles and infrastructure correlation.
design/​README.md Updates observability document scope.
design/​embedding-queue-and-worker.md Clarifies tokenizer telemetry behavior.
.env.sample Documents profile and interval settings.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@jgpruitt jgpruitt self-assigned this Oct 16, 2026
@jgpruitt
jgpruitt merged commit 0796e65 into main Oct 16, 2026
7 checks passed
@jgpruitt
jgpruitt deleted the reduce-logfire-telemetry branch October 16, 2026 20:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants