Skip to content

Share MCP proxy backend across agent sessions - #1395

Merged
sunishsheth2009 merged 4 commits into
databricks:mainfrom
sunishsheth2009:shared-mcp-backend
Nov 3, 2026
Merged

sunishsheth2009 merged 4 commits into
databricks:mainfrom
sunishsheth2009:shared-mcp-backend

Conversation

@sunishsheth2009

@sunishsheth2009 sunishsheth2009 commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Each agent/MCP registration currently loads a separate MCP/HTTP stack, increasing memory with every connection.

BEFORE: eight active local clients used 663 MiB aggregate RSS.
AFTER: lightweight stdio relays share one backend and use 218 MiB (67% less). Existing registrations work unchanged.

Each connection retains its own session, credentials, headers, and browser-login output. Different credential environments and selected workspaces use separate backends. The backend starts on demand, authenticates local relays, and exits after 60 seconds idle. A crash asks clients to reconnect; in-flight calls are never replayed.

Both CLI paths use the same option/default resolution. Customer documentation covers the benefit and reconnect action; implementation details stay in code. No new dependencies.

Validation: 1,098 relevant tests passed; 147 transport, header and shared-proxy tests passed after the final header cleanup. Ruff lint/format and ty passed. Subprocess tests cover concurrent startup, session/auth isolation, crash recovery, and idle shutdown against a local HTTP MCP peer. Memory figures include all relay/backend processes, excluding the driver/peer in both variants (PSS 500 → 124 MiB). Native Windows and live Databricks validation remain outstanding.

@sunishsheth2009
sunishsheth2009 marked this pull request as ready for review November 2, 2026 19:19
@sunishsheth2009
sunishsheth2009 requested a review from xi-db November 2, 2026 19:20

@xi-db xi-db left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sunishsheth2009
sunishsheth2009 added this pull request to the merge queue Nov 3, 2026
Merged via the queue into databricks:main with commit 8283b38 Nov 3, 2026
39 checks passed
@sunishsheth2009
sunishsheth2009 deleted the shared-mcp-backend branch November 3, 2026 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants