Skip to content

perf(bench): end-to-end overhead harness and Phase 0 gate results - #50

Merged
shreyanshjain7174 merged 2 commits into
mainfrom
perf/phase0-exit-bench
Oct 18, 2026
Merged

shreyanshjain7174 merged 2 commits into
mainfrom
perf/phase0-exit-bench

Conversation

@shreyanshjain7174

Copy link
Copy Markdown
Contributor

Plan

Phase 0 exit gate from the AgentGate scale-out prompt: an end-to-end handleAct benchmark, a k6 script, and results in bench/RESULTS.md. The target is under 1 ms p99 added overhead, excluding upstream time, on 4 vCPU.

Gate status: not met. This PR adds the harness and records honest numbers. It changes no production code.

What changed

  • bench/ end-to-end benchmark (build tag bench). The full real stack runs in-process: SQLite with migrations, HMAC keys, group-commit ledger and signer, cached vault, upstream pool, rate limiter, and a registry pointing at an httptest upstream.
    • Requests go over loopback through the real handler.
    • Overhead is total latency minus the upstream time measured per request.
    • Closed-loop runs at concurrency 1, 16 and 64.
    • Open-loop runs at 25%, 50% and 75% of max throughput, timed from the scheduled send so coordinated omission does not hide the tail.
    • Upstream delay 0 and 5 ms. GOMAXPROCS=4.
  • bench/k6/act.js and bench/mockupstream/ for an out-of-process run. k6 is not installed here, so k6 numbers are pending.
  • make bench-e2e.

Results (Apple M4, GOMAXPROCS=4 emulation, 20,000 requests per row)

Mode Delay Load Overhead p99 Gate
closed 0 ms c1 0.34 ms pass
closed 0 ms c16 3.42 ms fail
closed 0 ms c64 6.22 ms fail
open 0 ms 25% (5.4K rps) 7.00 ms fail
open 5 ms 25% (5.4K rps) 2.33 ms fail

Full percentiles, throughput, the profile and the remaining rows are in bench/RESULTS.md.

Why this is not a final verdict

  • The load generator runs in the same process and shares the same 4 Ps as the gateway, so it competes for CPU.
  • Open-loop results are not monotonic in load (50% is worse than 75% at 0 ms), which points at the method as much as the gateway.
  • macOS power management and F_FULLFSYNC behavior differ from the Linux 4-vCPU target.

Next step, in a follow-up: an out-of-process load generator on Linux with 4 dedicated vCPUs, plus profiling of the tail at 25% load.

Validation

  • go build ./...: passed
  • go vet ./...: passed
  • go test -race ./...: passed. The benchmark is excluded by build tag.

shreyanshjain7174 and others added 2 commits October 18, 2026 06:09
Record the missed p99 gate without production tuning.

Signed-off-by: Shreyansh Sancheti <43677304+shreyanshjain7174@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep scheduled timestamps so queueing and late sends remain visible.

Signed-off-by: Shreyansh Sancheti <43677304+shreyanshjain7174@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@shreyanshjain7174
shreyanshjain7174 merged commit 19084f8 into main Oct 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant