Skip to content

Add progressive-discovery MCP proxy to hyperforge - #160

Draft
vangheem wants to merge 1 commit into
mainfrom
feat/hyperforge-mcp-proxy
Draft

vangheem wants to merge 1 commit into
mainfrom
feat/hyperforge-mcp-proxy

Conversation

@vangheem

Copy link
Copy Markdown
Collaborator

Summary

Adds hyperforge.mcp_proxy, an MCP proxy for progressive discovery. It sits in front of any number of upstream MCP servers. Instead of combining all of their tools into one long list, it exposes only two tools:

  • search: full-text search over every upstream tool. It returns qualified names (server.tool), descriptions and input schemas.
  • execute: calls a discovered tool by its qualified name with params.

This keeps the agent's tool list and context small no matter how many upstream servers are connected.

Architecture

ConfigProvider ──► MCPProxy ──► UpstreamServer (stdio | streamable http)  × N
                     │  └──► ToolIndex (BM25F)
                     ▼
             build_mcp_server()  ──► stdio  (commands.serve_stdio)
                                  └► HTTP   (app.create_app, Starlette + StreamableHTTPSessionManager)
  • config.py: Pydantic ProxyConfig / UpstreamServerConfig. Each upstream sets its transport, its stdio command or HTTP URL, a timeout, include/exclude tool filters and tags. It also accepts the common mcpServers: {name: {...}} mapping, from YAML or JSON.
  • provider.py: the ConfigProvider protocol, with static and file implementations. This is the extension point for later DB-backed or per-tenant configs.
  • upstream.py: one long-lived client session per upstream. Each session is owned by its own asyncio task, so the MCP client's anyio cancel scopes are entered and exited in the same task. A failed upstream does not stop the proxy. Its error is recorded and shown as unavailable_servers.
  • index.py: an in-memory BM25F-style index. Fields are weighted (tool name > server/tags > description > params > server description). It also uses camelCase/snake_case tokenization and simple suffix stemming. An empty query lists tools.
  • proxy.py: MCPProxy handles the lifecycle (start/stop/reload, async context manager), builds the index, and runs search and execute. If a tool name is unknown, it raises ToolNotFound with search-based suggestions.
  • server.py / app.py / commands.py: wrap the proxy as a low-level MCP Server, served over stdio or stateless streamable HTTP (with /health/ready). They are exposed through the new hyperforge-mcp-proxy entry point. Settings use the MCP_PROXY_ environment prefix.

Other changes

  • Adds a pyyaml dependency.
  • hyperforge/examples/mcp_proxy: an opencode setup that proxies a local FastMCP inventory server, server-everything and server-filesystem, plus a smoke-test client. node_modules is gitignored.
  • hyperforge/tests/mcp_proxy: config, proxy and end-to-end tests against real stdio fixture servers.

Notes / follow-ups

  • The proxy builds its index once at startup. Upstream tools/list_changed notifications are not handled yet; use reload().
  • If an upstream drops after startup, its tools stay indexed. Calls to them return a "not connected" error, and the proxy does not reconnect automatically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant