Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
193 changes: 193 additions & 0 deletions .github/workflows/python-native-qualification.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
name: Python native qualification

on:
workflow_dispatch:
pull_request:
paths:
- "bindings/python/**"
- "src/**"
- "Cargo.toml"
- "Cargo.lock"
- "deny.toml"
- ".github/workflows/python-native-qualification.yml"
push:
branches:
- main
- plan/python-native-library
paths:
- "bindings/python/**"
- "src/**"
- "Cargo.toml"
- "Cargo.lock"
- "deny.toml"
- ".github/workflows/python-native-qualification.yml"

permissions:
contents: read

jobs:
build-wheel:
name: Build ${{ matrix.target }} CPython ${{ matrix.python }}
runs-on: ${{ matrix.runner }}
strategy:
fail-fast: false
matrix:
include:
- { id: linux-x86_64, runner: ubuntu-24.04, target: x86_64-unknown-linux-gnu, python: "3.11", linux: true }
- { id: linux-aarch64, runner: ubuntu-24.04-arm, target: aarch64-unknown-linux-gnu, python: "3.11", linux: true }
- { id: macos-x86_64, runner: macos-15-intel, target: x86_64-apple-darwin, python: "3.11", linux: false }
- { id: macos-arm64, runner: macos-15, target: aarch64-apple-darwin, python: "3.11", linux: false }
- { id: windows-x86_64, runner: windows-2022, target: x86_64-pc-windows-msvc, python: "3.11", linux: false }
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065
with:
python-version: ${{ matrix.python }}
- name: Check candidate package metadata
run: |
python -c "import tomllib; from pathlib import Path; project=tomllib.loads(Path('bindings/python/pyproject.toml').read_text())['project']; binding=tomllib.loads(Path('bindings/python/Cargo.toml').read_text())['package']; root=tomllib.loads(Path('Cargo.toml').read_text())['package']; assert project['name'] == 'eggsearch'; assert project['license'] == 'MIT'; assert project['version'] == binding['version'] == root['version']; print(project['name'], project['version'], project['license'])"
- if: matrix.linux
uses: PyO3/maturin-action@e83996d129638aa358a18fbd1dfb82f0b0fb5d3b
with:
maturin-version: v1.9.5
rust-toolchain: 1.89.0
manylinux: "2014"
args: --manifest-path bindings/python/Cargo.toml --release --locked --out dist -i python${{ matrix.python }}
- if: ${{ !matrix.linux }}
uses: dtolnay/rust-toolchain@6bed0761d98439e5a578e2877258200ad565ba87
with:
toolchain: 1.89.0
targets: ${{ matrix.target }}
- if: ${{ !matrix.linux }}
run: python -m pip install maturin==1.9.5
- if: ${{ !matrix.linux }}
run: maturin build --manifest-path bindings/python/Cargo.toml --release --locked --out dist --interpreter python
- name: Record wheel metadata and digest
shell: bash
run: |
python -m pip install --no-index --no-deps --only-binary=:all: dist/*.whl
python -c "from importlib.metadata import distribution; d=distribution('eggsearch'); print(d.metadata['Name'], d.version); print(*d.files, sep='\\n')" > dist/metadata.txt
sha256sum dist/*.whl | tee dist/SHA256SUMS
cat dist/SHA256SUMS >> "$GITHUB_STEP_SUMMARY"
- id: upload
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02
with:
name: wheel-${{ matrix.id }}
path: dist/
if-no-files-found: error
retention-days: 14
- env:
ARTIFACT_DIGEST: ${{ steps.upload.outputs.artifact-digest }}
run: |
python -c "import os; open(os.environ['GITHUB_STEP_SUMMARY'], 'a').write('GitHub artifact digest: ' + os.environ['ARTIFACT_DIGEST'] + '\\n')"

wheel-smoke:
name: Smoke ${{ matrix.target }} CPython ${{ matrix.python }}
needs: build-wheel
runs-on: ${{ matrix.runner }}
strategy:
fail-fast: false
matrix:
include:
- { id: linux-x86_64, runner: ubuntu-24.04, target: x86_64-unknown-linux-gnu, python: "3.11" }
- { id: linux-x86_64, runner: ubuntu-24.04, target: x86_64-unknown-linux-gnu, python: "3.12" }
- { id: linux-x86_64, runner: ubuntu-24.04, target: x86_64-unknown-linux-gnu, python: "3.13" }
- { id: linux-x86_64, runner: ubuntu-24.04, target: x86_64-unknown-linux-gnu, python: "3.14" }
- { id: linux-aarch64, runner: ubuntu-24.04-arm, target: aarch64-unknown-linux-gnu, python: "3.11" }
- { id: linux-aarch64, runner: ubuntu-24.04-arm, target: aarch64-unknown-linux-gnu, python: "3.12" }
- { id: linux-aarch64, runner: ubuntu-24.04-arm, target: aarch64-unknown-linux-gnu, python: "3.13" }
- { id: linux-aarch64, runner: ubuntu-24.04-arm, target: aarch64-unknown-linux-gnu, python: "3.14" }
- { id: macos-x86_64, runner: macos-15-intel, target: x86_64-apple-darwin, python: "3.11" }
- { id: macos-x86_64, runner: macos-15-intel, target: x86_64-apple-darwin, python: "3.12" }
- { id: macos-x86_64, runner: macos-15-intel, target: x86_64-apple-darwin, python: "3.13" }
- { id: macos-x86_64, runner: macos-15-intel, target: x86_64-apple-darwin, python: "3.14" }
- { id: macos-arm64, runner: macos-15, target: aarch64-apple-darwin, python: "3.11" }
- { id: macos-arm64, runner: macos-15, target: aarch64-apple-darwin, python: "3.12" }
- { id: macos-arm64, runner: macos-15, target: aarch64-apple-darwin, python: "3.13" }
- { id: macos-arm64, runner: macos-15, target: aarch64-apple-darwin, python: "3.14" }
- { id: windows-x86_64, runner: windows-2022, target: x86_64-pc-windows-msvc, python: "3.11" }
- { id: windows-x86_64, runner: windows-2022, target: x86_64-pc-windows-msvc, python: "3.12" }
- { id: windows-x86_64, runner: windows-2022, target: x86_64-pc-windows-msvc, python: "3.13" }
- { id: windows-x86_64, runner: windows-2022, target: x86_64-pc-windows-msvc, python: "3.14" }
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065
with:
python-version: ${{ matrix.python }}
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093
with:
name: wheel-${{ matrix.id }}
path: ${{ runner.temp }}/wheelhouse
- name: Create a clean consumer environment
run: python -m venv "$RUNNER_TEMP/.venv"
shell: bash
- name: Test installed wheel outside checkout
run: |
if [ "$RUNNER_OS" = "Windows" ]; then
python_bin="$(cygpath -u "$RUNNER_TEMP")/.venv/Scripts/python.exe"
else
python_bin="$RUNNER_TEMP/.venv/bin/python"
fi
"$python_bin" -m pip install --no-index --no-deps --only-binary=:all: wheelhouse/*.whl
"$python_bin" -m pip install pytest==8.4.2
"$python_bin" -m pytest -q "$GITHUB_WORKSPACE/bindings/python/tests"
shell: bash
working-directory: ${{ runner.temp }}

source-distribution:
name: Rebuild and smoke extracted sdist
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- uses: dtolnay/rust-toolchain@6bed0761d98439e5a578e2877258200ad565ba87
with:
toolchain: 1.89.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065
with:
python-version: "3.14"
- run: python -m pip install maturin==1.9.5
- name: Build sdist, prefetch locked crates, and rebuild offline
shell: bash
run: |
maturin build --manifest-path bindings/python/Cargo.toml --release --sdist --locked --out dist
cargo fetch --locked --manifest-path bindings/python/Cargo.toml
mkdir extracted
tar -xzf dist/*.tar.gz -C extracted
source_dir="$(find extracted -mindepth 1 -maxdepth 1 -type d)"
source_dir="$(cd "$source_dir" && pwd)"
(cd "$source_dir" && CARGO_NET_OFFLINE=true maturin build --release --locked --out "$GITHUB_WORKSPACE/extracted-wheel")
python -m venv "$RUNNER_TEMP/runtime-venv"
"$RUNNER_TEMP/runtime-venv/bin/python" -m pip install --no-index --no-deps extracted-wheel/*.whl
"$RUNNER_TEMP/runtime-venv/bin/python" -m pip install pytest==8.4.2
cd "$RUNNER_TEMP"
"$RUNNER_TEMP/runtime-venv/bin/python" -m pytest -q "$source_dir/bindings/python/tests"
"$RUNNER_TEMP/runtime-venv/bin/python" -c "import eggsearch; from importlib.metadata import distribution; assert len(eggsearch.tool_names()) == 10; assert distribution('eggsearch').version == '0.4.3'"
sha256sum "$GITHUB_WORKSPACE"/dist/* "$GITHUB_WORKSPACE"/extracted-wheel/* | tee "$GITHUB_WORKSPACE/dist/SHA256SUMS"
cat "$GITHUB_WORKSPACE/dist/SHA256SUMS" >> "$GITHUB_STEP_SUMMARY"
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02
with:
name: python-sdist-and-extracted-wheel
path: |
dist/*.tar.gz
dist/SHA256SUMS
extracted-wheel/*.whl
if-no-files-found: error
retention-days: 14

dependency-audit:
name: Audit binding dependency graph
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- uses: dtolnay/rust-toolchain@6bed0761d98439e5a578e2877258200ad565ba87
with:
toolchain: 1.89.0
- run: cargo install cargo-deny --version 0.20.2 --locked
- run: cargo deny check
working-directory: bindings/python
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065
with:
python-version: "3.14"
- run: python -m pip install --upgrade pip==26.2.1
- run: python -m pip install pip-audit==2.10.1 maturin==1.9.5
- run: pip-audit --local --progress-spinner off
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

eggsearch is a lightweight MCP search/fetch server for AI agents (live metasearch with RRF dedup, bounded fetch, deterministic evidence bundles). Transports: client-owned `mcp stdio` or explicit loopback-only `mcp serve`.

Single library + binary crate, not a workspace. Stable contract is MCP tools (10) plus CLI; the Rust module tree is an implementation detail with no semver library guarantee (see `src/lib.rs`). Dependency flow: `core <- meta <- mcp <- commands`; `fetch` is independent of `meta`. Start with `src/lib.rs` (module map) and `architecture/overview.md` (component index); operator docs live in `docs/`.
Single library + binary crate, not a workspace. Stable contract is MCP tools (10) plus CLI, with a narrow native operation facade in `src/sdk.rs`; the remaining Rust module tree has no general semver library guarantee (see `src/lib.rs` and `architecture/python-library.md`). Dependency flow: `core <- meta <- mcp <- commands`; `fetch` is independent of `meta`. Start with `src/lib.rs` (module map) and `architecture/overview.md` (component index); operator docs live in `docs/`.

## Build & verification

Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,10 @@ See [Safety](docs/safety.md) and [Threat model](docs/threat-model.md).
[CodeGG harness integration](docs/codegg-integration.md) ·
[Test inventory](docs/test-inventory.md)

The in-process asyncio Python package is under `bindings/python` and is not
published yet. See the [Python library guide](docs/python-library.md) for
local build instructions and current support limits.

## Build

```bash
Expand Down
3 changes: 2 additions & 1 deletion architecture/maintenance.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

**Purpose:** Keep the post-consolidation repository from re-concentrating into new monoliths. Contributor contract for adding tools, providers, workflows, parsers, and tests.

The crate is application-first: the stable contract is MCP tools (10) plus CLI. The Rust module tree is an implementation detail for the binary, integration tests, and fuzz harnesses. Dependency flow: `core <- meta <- mcp <- commands`; `fetch` is independent of `meta`. Factual claims about tools and providers must agree with the code; inventories below are guard-enforced, the rest is kept in sync by discipline.
The crate is application-first: the stable contract is MCP tools (10) plus CLI, with the narrow `src/sdk.rs` application facade for native consumers. The remaining Rust module tree is an implementation detail for the binary, integration tests, and fuzz harnesses. Dependency flow: `core <- meta <- mcp <- commands`; `fetch` is independent of `meta`. Factual claims about tools and providers must agree with the code; inventories below are guard-enforced, the rest is kept in sync by discipline.

---

Expand All @@ -26,6 +26,7 @@ Guard-enforced by `tests/static_guards.rs` (fail-closed). Transports call tool s
| Fetch execution | `src/fetch/` (client, limits, cache, origin, browser) | Search ranking, evidence roles |
| Outbound HTTP transport | `eggfetch-core` (transport, pooling, TLS, decompression, deadlines, bounded reads, pinned routing, redirect mechanics, typed failures); optional `src/fetch/egress.rs` (`EggressDialer` beneath the eggfetch `Dialer` seam, provider-only, fail-closed, credential-indirected); rmcp transitively owns only its Streamable HTTP client transport | eggsearch-local reqwest clients or facades; SSRF/retry/truncation policy stays in eggsearch; dynamic `FetchClient` targets never use egress |
| Release/deploy surface | `src/platform.rs`, `src/update.rs`, `src/startup.rs`, `packaging/` | Search/fetch policy |
| Native SDK facade | `src/sdk.rs` delegates to the ten `src/mcp/tools/` handlers and owns only dispatch, config-backed shared state, and structured errors | Provider selection, validation, policy, projection, and tool semantics |

Overlap ratchets (each a named guard):

Expand Down
2 changes: 2 additions & 0 deletions architecture/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,8 @@ fetch ↗
| Browser rendering & profiles | `src/fetch/browser/` (8 files) | Optional headless Chrome/Chromium via CDP; persistent origin-scoped login profiles | [fetch.md](fetch.md#9-browser-rendering-browser-8-files-feature-browser) |
| HTML rendering | `src/fetch/render/` (8 files) | Structural rendering: blocks, text, markdown, code, CSV, notebooks | [fetch.md](fetch.md#6-render-subsystem-render-8-files) |
| MCP server & tools | `src/mcp/` (7 modules + `tools/` with 10 tool modules plus `mod`/`common`/`canonical`/`tests`) | rmcp ServerHandler with 10 tools, stdio and loopback Streamable HTTP | [mcp.md](mcp.md) |
| Native SDK facade | `src/sdk.rs` | Narrow direct Rust operation API delegating to the same ten MCP handlers | [python-library.md](python-library.md) |
| Python native package | `bindings/python/` | PyO3 asyncio client, named Python methods, and typed agent-facing JSON contracts | [python-library.md](python-library.md) |
| CLI commands | `src/commands/` (binary-only, `mod` + 9 modules) | Subcommand wiring: doctor, search, fetch, providers, update, integrate, startup, restart, croncheck, mcp stdio/serve, browser-login/profiles | [commands.md](commands.md) |
| Agent/IDE integrations | `src/integrations/` (10 files, 7 clients) | Client-specific render/apply adapters with atomic JSON edits and protocol verification | [integrations.md](integrations.md) |
| Startup supervision | `src/startup.rs`, `packaging/systemd/`, `packaging/launchd/`, `packaging/windows/` | Canonical persistent runtime, manager detection/rendering, cron watchdog, identity-safe restart, and service state | [startup.md](startup.md) |
Expand Down
68 changes: 68 additions & 0 deletions architecture/python-library.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Python Native Library

The Python library is an additive native consumer of the ten existing tool
operations. Its ownership boundary is registered in
`plans/adrs/ADR-0005-python-native-library-boundary.md` and sequenced by
`plans/subsystems/python-native-library-roadmap.md`.

Current handoff: M001's shared Rust facade and ten-operation parity are
closed at `plans/closure/python-native-library/001-status.md`; M002's native
asyncio bridge is closed at `plans/closure/python-native-library/002-status.md`;
M003's Python API, types, examples, and semantic parity work is closed at
`plans/closure/python-native-library/003-status.md`; M004 distribution
qualification is active.

## Rust operation facade

`src/sdk.rs` owns the narrow `EggsearchClient`, canonical `ToolName` mapping,
shared `ServerState` lifetime, and structured `SdkError` conversion. It
delegates every operation to the matching function under `src/mcp/tools/`.
Those handlers remain the only owner of argument semantics, provider and
fetch policy, sanitization, evidence identity, warnings, and result shape.
The facade returns tool JSON payloads without MCP envelopes. Cloned clients
share one state; the caller supplies the Tokio runtime.

The facade does not make the rest of the crate's module tree a stable general
Rust library. It introduces no Python dependency, runtime, process, listener,
or background task into the root crate. `ToolName` stays aligned with the
code-derived `tool_contract::ALL_CONTRACTS` inventory.

## Python binding and distribution boundary

The Python package lives in an independent `bindings/python` Cargo package.
PyO3 and Python packaging dependencies must not enter the root crate's
default dependency graph or change its binary artifact inventory. Python
import is side-effect-free. The binding owns Python types and lifecycle
adaptation; all execution delegates through `src/sdk.rs`. The PyO3 layer
initializes one process-scoped Tokio runtime on first operation, caps shared
in-flight calls at 32, and bounds JSON conversion to 1 MiB, 64 levels, and
10,000 values. `aclose()` rejects new work, cancels active operations, and
waits for their Rust futures to drop; Python task cancellation drops its
corresponding future. The runtime intentionally remains process-scoped.

Milestone status and exact-candidate evidence live in
`plans/registry.md` and `plans/closure/python-native-library/`. A compiled
extension alone is not evidence of a supported Python consumer. Wheel support
is limited to targets with native install and runtime qualification, and
publication requires separate authorization.

M004's candidate binding uses PyO3's `abi3-py311` stable ABI. Its release
matrix builds a manylinux2014 Linux x86_64/aarch64 wheel, macOS x86_64/arm64
wheels, and a Windows x86_64 wheel, then installs each exact artifact on
CPython 3.11–3.14 in separate consumer jobs. These platform claims remain
conditional on the matching exact-candidate matrix jobs passing. An sdist
rebuild uses the binding and root Cargo manifests from clean extraction; it can
build offline with locked crates already cached, otherwise Cargo registry
access is required. PyPy, free-threaded CPython, ARMv7, and Windows ARM64 are
outside the support matrix.

The public Python `AsyncEggsearch` wrapper adds `async with`, `aclose()`, and
the ten named keyword methods over the generic native `execute` call. Request
and response `TypedDict` definitions live in `bindings/python/python/eggsearch/types.py`;
they describe common JSON fields while runtime dictionaries retain additive
keys. The `py.typed` marker and native extension stub make the package
discoverable to type checkers. `tool_names()` comes from the Rust SDK's
code-derived canonical inventory. `tool_contracts()` returns input/output
schemas and the MCP contract fingerprint from the existing server definitions;
the Python test suite checks request type coverage against advertised schema
properties.
Loading
Loading