Skip to content

feat(Telemetry): Emit CLI usage events from an API middleware - #9010

Merged
khvn26 merged 3 commits into
mainfrom
feat/cli-usage-events
Nov 18, 2026
Merged

khvn26 merged 3 commits into
mainfrom
feat/cli-usage-events

Conversation

@khvn26

@khvn26 khvn26 commented Nov 18, 2026 •

Copy link
Copy Markdown
Member
  • I have read the Contributing Guide.
  • I have added information to docs/ if required so people know about the feature.
  • I have filled in the "Changes" section below.
  • I have filled in the "How did you test this code" section below.

Changes

Closes #8319

In this PR, we adapt the existing MCPUsageLoggerMiddleware to also log CLI usage, and rename it accordingly.

The middleware now works out which client sent the request (MCP baggage first, then the CLI user-agent) before it reads request.user.

cli.request.made:

  • cli.version, cli.os, cli.arch, and cli.version come from the user-agent.
  • amplitude.user_id is str(request.user.uuid). It is absent for master API keys.
  • status is error for any response with a status code of 400 or above, success otherwise. The event is emitted after the response is obtained, so a 500 is recorded too.
  • organisation.id is resolved as for MCP.

Nothing is logged for unauthenticated requests, SDK requests, or other user-agents.

How did you test this code?

Added new unit tests.

Cover the cli.request.made event emitted for authenticated Flagsmith CLI
requests: the user-agent pattern, status, organisation resolution for
users and master API keys, and that the key paths read no database.
@vercel

vercel Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Nov 18, 2026 5:52pm UTC
2 Skipped Deployments
Project Deployment Actions Updated
flagsmith-frontend-preview Ignored Ignored Preview Nov 18, 2026 5:52pm UTC
flagsmith-frontend-staging Ignored Ignored Preview Nov 18, 2026 5:52pm UTC

Request Review

@github-actions github-actions Bot added the api Issue related to the REST API label Nov 18, 2026
@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The middleware is renamed to ClientUsageLoggerMiddleware and handles authenticated MCP and CLI requests. It recognises CLI requests from a complete flagsmith-cli/<version> (<os>/<arch>) user-agent. CLI and MCP events include response-derived status and organisation details. CLI events include parsed client details and include the user UUID except for master API key users. Tests and the event catalogue cover the updated logging behaviour.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: Minimal · up to e61a8

This change adds CLI usage telemetry to the existing middleware and renames it consistently. No concrete merge-blocking risk was found.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 0262ab45-1d14-42f6-bbe5-c1f2c73a2905
Commits

Reviewing files that changed from the base of the PR and between d465e74 and e61a892.

Files selected for processing: 4

  • api/app/settings/common.py
  • api/telemetry/middleware.py
  • api/tests/unit/telemetry/test_unit_telemetry_middleware.py
  • docs/docs/deployment-self-hosting/observability/_events-catalogue.md

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

@github-actions github-actions Bot added the feature New feature or request label Nov 18, 2026
Log unresolved organisations as None at WARNING, as the MCP middleware
does, cover SDK requests carrying a CLI user-agent, and drop the
query-count tests now that organisation resolution follows MCP.
@github-actions github-actions Bot added feature New feature or request and removed feature New feature or request labels Nov 18, 2026
Rename MCPUsageLoggerMiddleware to ClientUsageLoggerMiddleware and have
it also emit cli.request.made for authenticated requests whose
user-agent is the Flagsmith CLI's. Both events share authentication,
status and organisation resolution, so they keep a single, last slot in
MIDDLEWARE.

Emit both events from literal log calls so the events catalogue picks
them up, which also documents the status attribute of mcp.tool.called.
@github-actions github-actions Bot added docs Documentation updates feature New feature or request and removed feature New feature or request docs Documentation updates labels Nov 18, 2026
@khvn26
khvn26 marked this pull request as ready for review November 18, 2026 17:57
@khvn26
khvn26 requested review from a team as code owners November 18, 2026 17:57
@khvn26
khvn26 requested review from emyller and removed request for a team November 18, 2026 17:57
@github-actions github-actions Bot removed the feature New feature or request label Nov 18, 2026
@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Docker builds report

Image Build Status Security report
ghcr.io/flagsmith/flagsmith-e2e:pr-9010 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-frontend:pr-9010 Finished ✅ Results ✅
ghcr.io/flagsmith/flagsmith-api:pr-9010 Finished ✅ Results ✅
ghcr.io/flagsmith/flagsmith-api-test:pr-9010 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith:pr-9010 Finished ✅ Results ✅
ghcr.io/flagsmith/flagsmith-private-cloud:pr-9010 Finished ✅ Results ✅

@github-actions github-actions Bot added the feature New feature or request label Nov 18, 2026
@codecov

codecov Bot commented Nov 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.92%. Comparing base (e9acd24) to head (e61a892).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #9010   +/-   ##
=======================================
  Coverage   98.92%   98.92%           
=======================================
  Files        1713     1713           
  Lines       72482    72566   +84     
=======================================
+ Hits        71705    71789   +84     
  Misses        777      777           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor
✅ private-cloud · depot-ubuntu-latest-arm-16 — run #23434 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  7 passed

Details

stats  7 tests across 7 suites
duration  37.1 seconds
commit  e61a892
info  🔄 Run: #23434 (attempt 1)

🗂️ Previous results
✅ private-cloud · depot-ubuntu-latest-16 — run #23434 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  7 passed

Details

stats  7 tests across 4 suites
duration  1 minute
commit  e61a892
info  🔄 Run: #23434 (attempt 1)

✅ oss · depot-ubuntu-latest-16 — run #23434 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  2 passed

Details

stats  2 tests across 2 suites
duration  4.3 seconds
commit  e61a892
info  🔄 Run: #23434 (attempt 1)

✅ oss · depot-ubuntu-latest-arm-16 — run #23434 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  35.1 seconds
commit  e61a892
info  🔄 Run: #23434 (attempt 1)

@github-actions

Copy link
Copy Markdown
Contributor

Visual Regression

25 screenshots compared. See report for details.
View full report

@emyller emyller left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

I'd be curious to track how many warnings we get on SaaS, and revisit the MultipleObjectsReturned case for multi-org users if necessary. I think it's not.

@khvn26
khvn26 merged commit 7ec6a0e into main Nov 18, 2026
57 checks passed
@khvn26
khvn26 deleted the feat/cli-usage-events branch November 18, 2026 18:32
@linear-code

linear-code Bot commented Nov 19, 2026

Copy link
Copy Markdown

ENG-647

This branch was successfully deployed

1 active deployment
Preview – docs — e61a8922 Deployed Nov 18, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Issue related to the REST API feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Emit CLI usage events from an API middleware

2 participants