Skip to content

Prototype portable KV-backed Geo lookup - #454

Draft
ChristianPavilonis wants to merge 1 commit into
feature/pluggable-geo-providersfrom
feature/kv-geoip
Draft

ChristianPavilonis wants to merge 1 commit into
feature/pluggable-geo-providersfrom
feature/kv-geoip

Conversation

@ChristianPavilonis

Copy link
Copy Markdown
Collaborator

Summary

  • Explore a portable Geo implementation that needs neither platform-provided geography nor a runtime filesystem, using GeoLite CSV import and exact-key KV lookup.
  • Add a standalone format prototype through the real EdgeZero KvHandle, with versioned binary range objects, digest validation and bounded optional object caching.
  • Record real-data measurements and remaining design decisions. This is a draft experiment, not production Geo integration.

Stack

Stacked on #439, with base feature/pluggable-geo-providers at 72ceed7e.

This is a sibling alternative to the MMDB experiment in #453, not stacked on its implementation. Both retain the same pre-MMDB base. No existing stack branch is rebased or retargeted.

Changes

File Change
examples/geo-kv-prototype/ Isolated Rust workspace and lockfile, City/Country CSV projection, deterministic index export, asynchronous KvHandle lookup, cache and CSV-oracle verification.
examples/geo-kv-prototype/src/store.rs Local artifact-file KV harness for export/read-back and measurements. Synchronous filesystem operations here are not a production async backend.
docs/superpowers/specs/2026-11-05-kv-geo-design.md Portable async provider proposal, checked reader-IP ownership, KV indexing, publication and adapter acceptance requirements.
docs/superpowers/plans/2026-11-05-kv-geo-prototype-results.md Reproduction details, source fingerprints, measured results, verification record and unresolved policy decisions.

Production crates, workspace dependencies, request dispatch and existing Geo APIs are unchanged. CSV/import dependencies stay in the standalone prototype. No production dataset, generated index or download credentials are committed.

Measurements

Imported the permitted GeoLite2-City-CSV_20261103 release, containing 3,632,756 IPv4 and 2,091,942 IPv6 networks.

  • A 64 KiB object budget produced 40.302 MiB across 649 objects, including the manifest. A lookup needs at most two object reads after manifest acquisition.
  • Tested 16/64/256/384 KiB budgets, complete object read-back, deterministic re-publication and cache-disabled/enabled/repeated modes.
  • All 49,152 comparisons matched the source-CIDR oracle. These reuse a 4,096-point boundary/random workload across sizes and cache modes, not exhaustive coverage or representative user traffic.
  • A fresh lookup process used no CSV/inventory preload and peaked at about 2.9 MiB RSS for the small measured workload. This is not a maximum serving-memory guarantee.

Measurements use local artifact-file KV through KvHandle. They do not establish cloud KV latency, cost, concurrency, propagation or cold-isolate behavior. See the measurement record and usage instructions.

Issue scope

Refs #438 and #391. This draft only explores format and lookup; it does not complete the production GeoIP acceptance work or close an issue.

Test plan

Tested implementation: e4ede101. Runtime source fingerprint is unchanged from the measured prototype; pre-PR updates only extend its verification record.

  • cargo test --offline --locked --manifest-path examples/geo-kv-prototype/Cargo.toml: 10 passed.
  • Prototype all-target Clippy with -D warnings, formatting, and locked release build.
  • cargo test --offline --locked --workspace --all-targets: 2,115 passed, six existing tests ignored.
  • cargo clippy --offline --locked --workspace --all-targets --all-features -- -D warnings.
  • cargo fmt --all -- --check and staged whitespace checks.
  • cargo check --offline --locked --workspace --all-targets --features 'fastly cloudflare spin'.
  • cargo check --offline --locked -p edgezero-adapter-spin --target wasm32-wasip2 --features spin.
  • Core unit tests and enabled doctests; 13 existing doctests remain ignored.
  • Real City import, all-object digest read-back, repeat-build identity, CSV oracle and fresh-process lookup.
  • README formatting and Markdown relative-link/code-fence checks.
  • Remote KV and platform uploader acceptance, production request integration, cancellation and checked reader-IP integration on edge adapters.
  • New Fastly/Cloudflare WASM runtime execution or a real Country release benchmark.

The current GitHub workflow base filters do not include feature/pluggable-geo-providers, and the root workspace excludes this standalone crate. The commands above are local verification, not a claim of automatic GitHub prototype coverage. Automated coverage needs wiring before this becomes merge-ready production work.

Open decisions and limitations

  1. The release contains XK in 393 IPv4 and 68 IPv6 source networks, but core's country allowlist rejects it. The prototype preserves it in a source-format DTO and reports core_country_supported: false, without changing production policy or silently dropping intervals. Decide production outcome handling explicitly.
  2. The binary format, object budget, configuration API and publishing interfaces are not frozen. 64 KiB is a candidate for remote-KV experiments, not a settled production default.
  3. The local harness is not a crash-safe publisher or a globally consistent deployment mechanism. Remote publication, propagation and dataset/cache retirement still need proof.
  4. This does not add an async production Geo provider, reader-IP trust to other adapters, jurisdiction/consent policy or provider fallback.

Checklist

  • Scope is an isolated prototype and design/evidence documents.
  • New behavior has synthetic tests and real-data verification.
  • No Tokio dependency added to core or adapters; no production store wiring changed.
  • No dataset, generated index, secrets or credentials committed.
  • Production acceptance complete. Keep this PR draft.

@ChristianPavilonis ChristianPavilonis self-assigned this Nov 5, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant